Incorrect Kafka offset across consumer groups

whysoseriousson

I have a topic which is consumed by two consumer groups. There are 10 messages in the topic.

Application 1 (Consumer group 1)has finished processing of messages from topic and when i describe kafka-consumer-groups (with --group consumerGroup1) then it correctly shows CURRENT-OFFSET =10 and LOG-END-OFFSET =10.

Now i start application 2 (Consumer group 2) which is consuming same topic. It is not processing messages . When i describe kafka-consumer-groups (with --group consumerGroup2) , it surprisingly shows CURRENT-OFFSET =10 and LOG-END-OFFSET =10.

Ideally this should not happen and kafka should be able to recognize that for the consumer group 2 there are no message processed and it should process all messages.

What could be wrong here.

Bartosz Wardziński

You have to set earliest property in your KafkaStreams Properties

Kafka consumer not returning any events

この記事はインターネットから収集されたものであり、転載の際にはソースを示してください。

侵害の場合は、連絡してください[email protected]

編集
0

コメントを追加

0

関連記事

分類Dev

Kafka consumer manual commit offset

分類Dev

Kafka - Assign messages to specific Consumer Groups

分類Dev

Spark 1.6 Streaming consumer reading in kafka offset stuck at createDirectStream

分類Dev

Who keeps track of the last read message offset of the consumer in Apache Kafka?

分類Dev

How to have multiple kafka consumer groups in application properties

分類Dev

Kafka consumer groups suddenly stopped balancing messages among instances

分類Dev

Kafka consumer groups suddenly stopped balancing messages among instances

分類Dev

Kafka Consumer not able to read all message after offset commit (error=OFFSET_OUT_OF_RANGE

分類Dev

Kafka - How to commit offset after every message using High-Level consumer?

分類Dev

How to set offset committed by the consumer group using Spark's Direct Stream for Kafka?

分類Dev

Is consumer offset commited even when failing to post to output topic in Kafka Streams?

分類Dev

Kafka Consumer configuration - How does auto.offset.reset controls the message consumption

分類Dev

kafka-consumer-groupsコマンドの問題

分類Dev

kafka-consumer-groupsコマンドは、spark構造化ストリーミングアプリケーション(コンシューマー)のLAGとCURRENT-OFFSETを表示しません

分類Dev

Kafka MirrorMakerが__consumer_offsetトピックの複製に失敗する

分類Dev

Kafka Consumer with JAVA

分類Dev

Spring boot Kafka consumer

分類Dev

kafka kafka-consumer-groups.sh --describeは、コンシューマーグループの出力を返しません

分類Dev

Kafka Stream: Consumer commit frequency

分類Dev

Partition specific flink kafka consumer

分類Dev

Kafka elixir consumer keeps crashing

分類Dev

kafka consumer code is not running completely

分類Dev

Kafka consumer hangs on poll when kafka is down

分類Dev

Kafka Offset after retention period

分類Dev

kafka: keep track of producer offset

分類Dev

Kafka consumer - what's the relation of consumer processes and threads with topic partitions

分類Dev

Kafka consumer group script to see all consumer group not working

分類Dev

How to calculate common values across different groups?

分類Dev

Kafka Avro Consumer with Decoderの問題

Related 関連記事

  1. 1

    Kafka consumer manual commit offset

  2. 2

    Kafka - Assign messages to specific Consumer Groups

  3. 3

    Spark 1.6 Streaming consumer reading in kafka offset stuck at createDirectStream

  4. 4

    Who keeps track of the last read message offset of the consumer in Apache Kafka?

  5. 5

    How to have multiple kafka consumer groups in application properties

  6. 6

    Kafka consumer groups suddenly stopped balancing messages among instances

  7. 7

    Kafka consumer groups suddenly stopped balancing messages among instances

  8. 8

    Kafka Consumer not able to read all message after offset commit (error=OFFSET_OUT_OF_RANGE

  9. 9

    Kafka - How to commit offset after every message using High-Level consumer?

  10. 10

    How to set offset committed by the consumer group using Spark's Direct Stream for Kafka?

  11. 11

    Is consumer offset commited even when failing to post to output topic in Kafka Streams?

  12. 12

    Kafka Consumer configuration - How does auto.offset.reset controls the message consumption

  13. 13

    kafka-consumer-groupsコマンドの問題

  14. 14

    kafka-consumer-groupsコマンドは、spark構造化ストリーミングアプリケーション(コンシューマー)のLAGとCURRENT-OFFSETを表示しません

  15. 15

    Kafka MirrorMakerが__consumer_offsetトピックの複製に失敗する

  16. 16

    Kafka Consumer with JAVA

  17. 17

    Spring boot Kafka consumer

  18. 18

    kafka kafka-consumer-groups.sh --describeは、コンシューマーグループの出力を返しません

  19. 19

    Kafka Stream: Consumer commit frequency

  20. 20

    Partition specific flink kafka consumer

  21. 21

    Kafka elixir consumer keeps crashing

  22. 22

    kafka consumer code is not running completely

  23. 23

    Kafka consumer hangs on poll when kafka is down

  24. 24

    Kafka Offset after retention period

  25. 25

    kafka: keep track of producer offset

  26. 26

    Kafka consumer - what's the relation of consumer processes and threads with topic partitions

  27. 27

    Kafka consumer group script to see all consumer group not working

  28. 28

    How to calculate common values across different groups?

  29. 29

    Kafka Avro Consumer with Decoderの問題

ホットタグ

アーカイブ