Kafka streams on spring, trouble with exactly once ACL: TransactionalIdAuthorizationException

Martin'sRun

I'm having trouble to connect to a topic on Confluent cloud from a springboot kafka-streams application, the connection failing with a single error line org.apache.kafka.common.errors.TransactionalIdAuthorizationException: Transactional Id authorization failed.

I believe this is because the topic is configured to allow transactions with a specified transactial-id (https://docs.confluent.io/current/kafka/authorization.html#enabling-authorization-for-idempotent-and-transactional-apis). My service being a streaming application, generates a transactional-id in each StreamThread (create producer method in https://github.com/apache/kafka/blob/2.4.1/streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java) which is defined as applicationId + "-" + taskId and this does not match the one defined on topic ACL.

Is this a correct, and if so is there a workaround?

Matthias J. Sax

You would need to use wildcard ACLs by only specifying the prefix, ie, <application.id>- for the corresponding ACL, using --resource-pattern-type prefixed

Cf. https://docs.confluent.io/current/streams/developer-guide/security.html

この記事はインターネットから収集されたものであり、転載の際にはソースを示してください。

侵害の場合は、連絡してください[email protected]

編集
0

コメントを追加

0

関連記事

分類Dev

Kafkaが春にストリーミング、ACLが1回だけ問題になる:TransactionalIdAuthorizationException

分類Dev

Kafka ACL-LEADER_NOT_AVAILABLE

分類Dev

Kafka Streams:RocksDB TTL

分類Dev

Kafka Connect and Streams

分類Dev

Kafka Streams TimestampExtractor

分類Dev

Retrieve data in Firebase exactly once

分類Dev

Spring Cloud Kafka StreamsバインダーでStateRestoreListenerを使用する

分類Dev

Spring Kafka Streams:タイプ情報がありません

分類Dev

How can I test a Spring Cloud Stream Kafka Streams application that uses Avro and the Confluent Schema Registry?

分類Dev

Is it possible to use @KafkaStreamsStateStore annotation on Spring Cloud Stream Kafka Streams 3.0 Binder style API methods?

分類Dev

Kafka Streams local state stores

分類Dev

Kafka streams aggregate throwing Exception

分類Dev

Kafka Streams GlobalKTable sychronization to applications

分類Dev

Kafka Streams: POJO serialization/deserialization

分類Dev

Kafka Connect vs Streams for Sinks

分類Dev

Kafka Streams - missing source topic

分類Dev

Trouble organizing perforce streams to accommodate multiple main branches with streams

分類Dev

python 3 regex: match a character exactly once

分類Dev

Kafka Streams API:KStreamからKTable

分類Dev

Kafka Streams API:KStreamからKTable

分類Dev

Kafka Streams API:KStreamからKTable

分類Dev

Kafka Streams:ConsumerRebalanceListenerの実装

分類Dev

Why is windowing now working for Kafka Streams?

分類Dev

NPE while deserializing avro messages in kafka streams

分類Dev

Kafka: SASL_SSL + ACL can produce but not consume

分類Dev

PROCESSING_GUARANTEE_CONFIGがEXACTLY_ONCEに設定されている場合、Kafkaはリバランスに失敗しました

分類Dev

Spring Boot 1.5を使用したテストでKafka Streamsが開始しないようにする

分類Dev

Kafka Streams:at_least_onceを使用する場合、州のストアへの保存の順序に関する保証はありますか?

分類Dev

Find shortest distance, while visiting every node exactly once

Related 関連記事

  1. 1

    Kafkaが春にストリーミング、ACLが1回だけ問題になる:TransactionalIdAuthorizationException

  2. 2

    Kafka ACL-LEADER_NOT_AVAILABLE

  3. 3

    Kafka Streams:RocksDB TTL

  4. 4

    Kafka Connect and Streams

  5. 5

    Kafka Streams TimestampExtractor

  6. 6

    Retrieve data in Firebase exactly once

  7. 7

    Spring Cloud Kafka StreamsバインダーでStateRestoreListenerを使用する

  8. 8

    Spring Kafka Streams:タイプ情報がありません

  9. 9

    How can I test a Spring Cloud Stream Kafka Streams application that uses Avro and the Confluent Schema Registry?

  10. 10

    Is it possible to use @KafkaStreamsStateStore annotation on Spring Cloud Stream Kafka Streams 3.0 Binder style API methods?

  11. 11

    Kafka Streams local state stores

  12. 12

    Kafka streams aggregate throwing Exception

  13. 13

    Kafka Streams GlobalKTable sychronization to applications

  14. 14

    Kafka Streams: POJO serialization/deserialization

  15. 15

    Kafka Connect vs Streams for Sinks

  16. 16

    Kafka Streams - missing source topic

  17. 17

    Trouble organizing perforce streams to accommodate multiple main branches with streams

  18. 18

    python 3 regex: match a character exactly once

  19. 19

    Kafka Streams API:KStreamからKTable

  20. 20

    Kafka Streams API:KStreamからKTable

  21. 21

    Kafka Streams API:KStreamからKTable

  22. 22

    Kafka Streams:ConsumerRebalanceListenerの実装

  23. 23

    Why is windowing now working for Kafka Streams?

  24. 24

    NPE while deserializing avro messages in kafka streams

  25. 25

    Kafka: SASL_SSL + ACL can produce but not consume

  26. 26

    PROCESSING_GUARANTEE_CONFIGがEXACTLY_ONCEに設定されている場合、Kafkaはリバランスに失敗しました

  27. 27

    Spring Boot 1.5を使用したテストでKafka Streamsが開始しないようにする

  28. 28

    Kafka Streams:at_least_onceを使用する場合、州のストアへの保存の順序に関する保証はありますか?

  29. 29

    Find shortest distance, while visiting every node exactly once

ホットタグ

アーカイブ