How to tune/edit/add/change the kafka docker container parameters outside from the container

jessica

We have kafka docker container as the following ( on Linux redhat 7.5 )

We created the kafka service on the container according to - https://docs.confluent.io/5.0.0/installation/docker/docs/installation/single-node-client.html

docker-compose ps
               Name                           Command            State                     Ports
-------------------------------------------------------------------------------------------------------------------
kafka-single-node_kafka_1            /etc/confluent/docker/run   Up      0.0.0.0:9092->9092/tcp

docker ps
CONTAINER ID        IMAGE                                             COMMAND                  CREATED             STATUS              PORTS                                        NAMES
     0.0.0.0:8081->8081/tcp                       kafka-single-node_schemaregistry_1
de584963bb5a        confluentinc/cp-kafka:latest                      "/etc/confluent/dock…"   6 hours ago         Up 6 hours          0.0.0.0:9092->9092/tcp                       kafka-single-node_kafka_1

about - docker-compose.yml

pwd
/home/cp-docker-images/examples/kafka-single-node

more docker-compose.yml

kafka:
    image: confluentinc/cp-kafka:latest
    depends_on:
      - zookeeper
    ports:
      - 9092:9092
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1

We want to control the kafka docker container configuration - /etc/kafka/server.properties outside from the container , ( or from the OS itself )

Let say I want to add some parameters or to change some parameters in server.properties ( of the kafka container )

so what is the best why to do the configuration outside from the kafka container

In the following example I show how to access the server.properties , inside The docker kafka container

docker exec -it de584963bb5a    bash

Now we can access/read the file - server.properties

root@de584963bb5a:/# more /etc/kafka/server.properties
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# see kafka.server.KafkaConfig for additional details and defaults

############################# Server Basics #############################

# The id of the broker. This must be set to a unique integer for each broker.
broker.id=0

############################# Socket Server Settings #############################

# The address the socket server listens on. It will get the value returned from
# java.net.InetAddress.getCanonicalHostName() if not configured.
#   FORMAT:
#     listeners = listener_name://host_name:port
#   EXAMPLE:
#     listeners = PLAINTEXT://your.host.name:9092
#listeners=PLAINTEXT://:9092
.
.
.

##################### Confluent Proactive Support ######################
# If set to true, and confluent-support-metrics package is installed
# then the feature to collect and report support metrics
# ("Metrics") is enabled.  If set to false, the feature is disabled.
#
confluent.support.metrics.enable=true


# The customer ID under which support metrics will be collected and
# reported.
#
# When the customer ID is set to "anonymous" (the default), then only a
# reduced set of metrics is being collected and reported.
#
# Confluent customers
# -------------------
# If you are a Confluent customer, then you should replace the default
# value with your actual Confluent customer ID.  Doing so will ensure
# that additional support metrics will be collected and reported.
#
confluent.support.customer.id=anonymous

############################# Group Coordinator Settings #############################

# The following configuration specifies the time, in milliseconds, that the GroupCoordinator will delay the initial consumer rebalance.
# The rebalance will be further delayed by the value of group.initial.rebalance.delay.ms as new members join the group, up to a maximum of max.poll.interval.ms.
# The default value for this is 3 seconds.
# We override this to 0 here as it makes for a better out-of-the-box experience for development and testing.
# However, in production environments the default value of 3 seconds is more suitable as this will help to avoid unnecessary, and potentially expensive, rebalances during application startu
p.
group.initial.rebalance.delay.ms=0
OneCricketeer

We want to control the kafka docker container configuration - /etc/kafka/server.properties outside from the container

I would avoid mounting a server.properties file from the host since every setting within the container can be configured by environment variables like KAFKA_(variable) - https://docs.confluent.io/current/installation/docker/config-reference.html#confluent-kafka-configuration

Rather, you can use env_file for the compose file and move all your variables there

kafka:
   image: confluentinc/cp-kafka:latest
    depends_on:
      - zookeeper
    ports:
      - 9092:9092
    env_file:
      - broker1.env

broker1.env

KAFKA_BROKER_ID=1
KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181
KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://kafka:9092
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
KAFKA_INTER_BROKER_LISTENER_NAME=PLAINTEXT
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1

And you can add any broker config with this pattern - https://kafka.apache.org/documentation/#brokerconfigs

この記事はインターネットから収集されたものであり、転載の際にはソースを示してください。

侵害の場合は、連絡してください[email protected]

編集
0

コメントを追加

0

関連記事

分類Dev

How to mount volume from container to host in Docker?

分類Dev

How to redirect command output from docker container

分類Dev

Giving Docker access to db file outside container

分類Dev

how to ssh docker container

分類Dev

how to ssh docker container

分類Dev

how to ssh docker container

分類Dev

docker for windows how to access docker daemon from container

分類Dev

How to stop Docker process from docker hub container busybox?

分類Dev

How to use confluent/cp-kafka image in docker compose with advertising on localhost and my network container name kafka?

分類Dev

How to exclude app settings from a docker image (and add them to the container)?

分類Dev

Web container cannot call backend container from docker compose

分類Dev

The children outside container

分類Dev

How to add a docker container to an existing docker network

分類Dev

Docker, How to get a container IP inside another container in PHP?

分類Dev

Access host database from a docker container

分類Dev

Docker replicate UID/GID in container from host

分類Dev

Copy folder with wildcard from docker container to host

分類Dev

docker stop spark container from exiting

分類Dev

docker copy file from one container to another?

分類Dev

Access vagrant VMs from inside docker container

分類Dev

Can't connect to db from docker container

分類Dev

Cloud sql proxy not working from docker container

分類Dev

Docker: Unable to connect to container from host

分類Dev

div container floats outside of parent container

分類Dev

container auto hide when clicked outside the container

分類Dev

Can I seed a mongodb docker container from an other docker container without docker-compose?

分類Dev

How to run a cron job inside a docker container?

分類Dev

How to have root permission in the Grafana Docker container?

分類Dev

How to connect nodeJS docker container to mongoDB

Related 関連記事

  1. 1

    How to mount volume from container to host in Docker?

  2. 2

    How to redirect command output from docker container

  3. 3

    Giving Docker access to db file outside container

  4. 4

    how to ssh docker container

  5. 5

    how to ssh docker container

  6. 6

    how to ssh docker container

  7. 7

    docker for windows how to access docker daemon from container

  8. 8

    How to stop Docker process from docker hub container busybox?

  9. 9

    How to use confluent/cp-kafka image in docker compose with advertising on localhost and my network container name kafka?

  10. 10

    How to exclude app settings from a docker image (and add them to the container)?

  11. 11

    Web container cannot call backend container from docker compose

  12. 12

    The children outside container

  13. 13

    How to add a docker container to an existing docker network

  14. 14

    Docker, How to get a container IP inside another container in PHP?

  15. 15

    Access host database from a docker container

  16. 16

    Docker replicate UID/GID in container from host

  17. 17

    Copy folder with wildcard from docker container to host

  18. 18

    docker stop spark container from exiting

  19. 19

    docker copy file from one container to another?

  20. 20

    Access vagrant VMs from inside docker container

  21. 21

    Can't connect to db from docker container

  22. 22

    Cloud sql proxy not working from docker container

  23. 23

    Docker: Unable to connect to container from host

  24. 24

    div container floats outside of parent container

  25. 25

    container auto hide when clicked outside the container

  26. 26

    Can I seed a mongodb docker container from an other docker container without docker-compose?

  27. 27

    How to run a cron job inside a docker container?

  28. 28

    How to have root permission in the Grafana Docker container?

  29. 29

    How to connect nodeJS docker container to mongoDB

ホットタグ

アーカイブ