Выполняется постепенное обновление с Apache Kafka (kafka_2.10-0.10.1.1) до Confluent Platform 4.0

Мы пытаемся выполнить последовательное обновление с Apache Kafka версии kafka_2.10-0.10.1.1 до Kafka, предоставляемого Confluent 4.0, и мы также хотим сохранить текущие данные. .

Вот шаги, которые мы выполнили:

  1. Завершите работу брокера Apache Kafka.
  2. Скопируйте server.properties из текущего в новый старый, чтобы убедиться, что он совпадает.
  3. Запустите новый узел Kafka из установки Confluent.

Однако мы сталкиваемся с тем, что новый узел пытается присоединиться к кластеру Kafka, поскольку я не мог получить реплику с других узлов в кластере Kafka.

Ошибка, как показано ниже:

GC_USR-0 as the leader reported an error: UNKNOWN_SERVER_ERROR (kafka.server.ReplicaFetcherThread)
[2017-12-21 02:51:33,869] INFO [ReplicaFetcher replicaId=1, leaderId=0, fetcherId=0] Retrying leaderEpoch request for partition __consumer_offsets-48 as the leader reported an error: UNKNOWN_SERVER_ERROR (kafka.server.ReplicaFetcherThread)
[2017-12-21 02:51:33,869] INFO [ReplicaFetcher replicaId=1, leaderId=0, fetcherId=0] Retrying leaderEpoch request for partition DCN_USR_RETRY-0 as the leader reported an error: UNKNOWN_SERVER_ERROR (kafka.server.ReplicaFetcherThread)
[2017-12-21 02:51:33,869] INFO [ReplicaFetcher replicaId=1, leaderId=0, fetcherId=0] Retrying leaderEpoch request for partition __consumer_offsets-19 as the leader reported an error: UNKNOWN_SERVER_ERROR (kafka.server.ReplicaFetcherThread)
[2017-12-21 02:51:33,869] INFO [ReplicaFetcher replicaId=1, leaderId=0, fetcherId=0] Retrying leaderEpoch request for partition GC_ACCT-0 as the leader reported an error: UNKNOWN_SERVER_ERROR (kafka.server.ReplicaFetcherThread)
[2017-12-21 02:51:33,869] INFO [ReplicaFetcher replicaId=1, leaderId=0, fetcherId=0] Retrying leaderEpoch request for partition __consumer_offsets-11 as the leader reported an error: UNKNOWN_SERVER_ERROR (kafka.server.ReplicaFetcherThread)
[2017-12-21 02:51:33,869] INFO [ReplicaFetcher replicaId=1, leaderId=0, fetcherId=0] Retrying leaderEpoch request for partition DCN_ERROR_TEST-0 as the leader reported an error: UNKNOWN_SERVER_ERROR (kafka.server.ReplicaFetcherThread)
[2017-12-21 02:51:33,869] INFO [ReplicaFetcher replicaId=1, leaderId=0, fetcherId=0] Retrying leaderEpoch request for partition DCN_MONITOR-0 as the leader reported an error: UNKNOWN_SERVER_ERROR (kafka.server.ReplicaFetcherThread)
[2017-12-21 02:51:33,869] INFO [ReplicaFetcher replicaId=1, leaderId=0, fetcherId=0] Retrying leaderEpoch request for partition __consumer_offsets-13 as the leader reported an error: UNKNOWN_SERVER_ERROR (kafka.server.ReplicaFetcherThread)
[2017-12-21 02:51:33,869] INFO [ReplicaFetcher replicaId=1, leaderId=0, fetcherId=0] Retrying leaderEpoch request for partition Hello-Yoga-Test-0 as the leader reported an error: UNKNOWN_SERVER_ERROR (kafka.server.ReplicaFetcherThread)
[2017-12-21 02:51:33,869] INFO [ReplicaFetcher replicaId=1, leaderId=0, fetcherId=0] Retrying leaderEpoch request for partition DCN_COUNT_TOPIC-0 as the leader reported an error: UNKNOWN_SERVER_ERROR (kafka.server.ReplicaFetcherThread)
[2017-12-21 02:51:33,869] INFO [ReplicaFetcher replicaId=1, leaderId=0, fetcherId=0] Retrying leaderEpoch request for partition __consumer_offsets-2 as the leader reported an error: UNKNOWN_SERVER_ERROR (kafka.server.ReplicaFetcherThread)
[2017-12-21 02:51:33,869] INFO [ReplicaFetcher replicaId=1, leaderId=0, fetcherId=0] Retrying leaderEpoch request for partition __consumer_offsets-43 as the leader reported an error: UNKNOWN_SERVER_ERROR (kafka.server.ReplicaFetcherThread)
[2017-12-21 02:51:33,869] INFO [ReplicaFetcher replicaId=1, leaderId=0, fetcherId=0] Retrying leaderEpoch request for partition DCN_INFRA_ERROR-0 as the leader reported an error: UNKNOWN_SERVER_ERROR (kafka.server.ReplicaFetcherThread)
[2017-12-21 02:51:33,869] INFO [ReplicaFetcher replicaId=1, leaderId=0, fetcherId=0] Retrying leaderEpoch request for partition __consumer_offsets-6 as the leader reported an error: UNKNOWN_SERVER_ERROR (kafka.server.ReplicaFetcherThread)
[2017-12-21 02:51:33,869] INFO [ReplicaFetcher replicaId=1, leaderId=0, fetcherId=0] Retrying leaderEpoch request for partition GC_ACL-0 as the leader reported an error: UNKNOWN_SERVER_ERROR (kafka.server.ReplicaFetcherThread)
[2017-12-21 02:51:33,869] INFO [ReplicaFetcher replicaId=1, leaderId=0, fetcherId=0] Retrying leaderEpoch request for partition DCN_REQUEST_GROUPS_OWNERSHIP-0 as the leader reported an error: UNKNOWN_SERVER_ERROR (kafka.server.ReplicaFetcherThread)
[2017-12-21 02:51:33,869] INFO [ReplicaFetcher replicaId=1, leaderId=0, fetcherId=0] Retrying leaderEpoch request for partition __consumer_offsets-14 as the leader reported an error: UNKNOWN_SERVER_ERROR (kafka.server.ReplicaFetcherThread)
[2017-12-21 02:51:33,869] INFO [ReplicaFetcher replicaId=1, leaderId=0, fetcherId=0] Retrying leaderEpoch request for partition DCN_MAILBOX_OWNER-0 as the leader reported an error: UNKNOWN_SERVER_ERROR (kafka.server.ReplicaFetcherThread)
[2017-12-21 02:51:33,869] INFO [ReplicaFetcher replicaId=1, leaderId=0, fetcherId=0] Retrying leaderEpoch request for partition TASK_REGISTRY_TEST-0 as the leader reported an error: UNKNOWN_SERVER_ERROR (kafka.server.ReplicaFetcherThread)
[2017-12-21 02:51:33,869] INFO [ReplicaFetcher replicaId=1, leaderId=0, fetcherId=0] Retrying leaderEpoch request for partition uatTest1-0 as the leader reported an error: UNKNOWN_SERVER_ERROR (kafka.server.ReplicaFetcherThread)
[2017-12-21 02:51:33,869] INFO [ReplicaFetcher replicaId=1, leaderId=0, fetcherId=0] Retrying leaderEpoch request for partition DCN_CMDB_TEST-0 as the leader reported an error: UNKNOWN_SERVER_ERROR (kafka.server.ReplicaFetcherThread)

Мы могли бы очистить все существующие данные и настроить новый кластер Confluent Platform, но мы уже использовали Apache Kafka в производственной среде, поэтому мы не хотим терять данные.

Пожалуйста, кто-нибудь может посоветовать?


person Joey Trang    schedule 21.12.2017    source источник


Ответы (1)


Я нашел решение. Следующие конфигурации необходимо добавить в server.properties.

inter.broker.protocol.version=0.10.1.1
log.message.format.version=0.10.1.1
person Joey Trang    schedule 21.12.2017
comment
После обновления всего кластера увеличьте версию протокола, отредактировав inter.broker.protocol.version и установив для него значение 1.0 ... kafka.apache.org/10/documentation.html#upgrade - person OneCricketeer; 01.10.2018