Skip to content

Kafka Partition Reassignment

When a new Kafka node is added, Kafka automatically does not do the partition reassignment. The following steps are used to perform manual reassignment of Kafka partitions for specified topic(s):

1.) Go to Kafka container

docker exec -it kafka_container bash

2.) Create /tmp/topics.json with topics whose partitions should be reassigned in the following format:

cat << EOF | tee /tmp/topics.json
"topics": [
{"topic": ""},
"version": 1

3.) Generate reassignment JSON output from list of topics to be migrated, specify the broker IDs in the broker list:

/usr/bin/kafka-reassign-partitions --zookeeper localhost:2181 --broker-list "121,122,221,222" --generate --topics-to-move-json-file /tmp/topics.json 

The result should be stored in /tmp/reassign.json and look as follows, with all topics and partitions having their new assigment specified:

[appuser@lm11 data]$ cat /tmp/reassign.json 

4.) Use the output from the previous command as input to the execution of the reassignment/rebalance:

/usr/bin/kafka-reassign-partitions --zookeeper localhost:2181  --execute --reassignment-json-file /tmp/reassign.json --additional --bootstrap-server localhost:9092

That's it! Now Kafka should perform the partitions reassignment within the following hours.

For more information, see Reassigning partitions in Apache Kafka Cluster .