Journey of Apache Kafka & Zookeeper Administrator ( Part 13 )

July 2020 ( Ansible support for Kafka 2.6.0 and Java 11 )

In my last article, I explained that how terraform can help with the oracle cloud.

Today, let’s talk about the maintenance of my project and why it’s important.
I have spent more than 1 year on it and I thought that other DevOps Personal can utilize it and have less hazel in supporting their own Kafka setups.

I am trying my best to support this project. here are two examples

1. Support for Kafka 2.6.0 Release
2. Support for Java 11 and higher version

Let’s talk about Kafka 2.6.0 like what is important being added for Kafka Administrator.

• Added support for More Metrics

Below was my PR, it was pretty simple. Only Need to Add new metrics to 116davinder/kafka-cluster-ansible/roles/jmxMonitor/files/kafka-input.txt

Merge pull request #2 from 116davinder/upgrade-2.6.0 · 116davinder/kafka-cluster-ansible@856ca88

Now let’s talk about Support for Java 11 and higher. I am really happy that someone tried using my project and found out that it doesn’t support Java 11 so he created an Issue.

Not working with JAVA 11 · Issue #3 · 116davinder/kafka-cluster-ansible

Honestly saying, I knew that java 11 support was not there so I started checking what is the issue and what exactly is failing. It took me a couple of hours because in my company, I always had a test cluster running and I did a small modification to test. Below were the changes, I had to do so my code can support java 11.

Merge pull request #4 from 116davinder/#3-Java-9-and-higher-gc-settings ·…

Important changes were

1. kafka-server-start.sh Template

# kafka gc logs settings
{% if javaVersion > 9 %}
export KAFKA_GC_LOG_OPTS="-Xlog:gc*:file={{ kafkaLogDir }}/kafkaServer-gc.log:time,tags:filecount=10,filesize=1024"
{% else %}
export KAFKA_GC_LOG_OPTS="-Xloggc:{{ kafkaLogDir }}/kafkaServer-gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=10M"
{% endif %}

2. Java Installation Role

- name: Install Openjdk | {{ javaVersion }} | RedHat | if java version is less than 9
  yum:
    name: "{{ item }}"
    state: "{{ javaUpdateState | default('present') }}"
    update_cache: true
  loop:
    - "java-1.{{ javaVersion }}.0-openjdk"
    - "java-1.{{ javaVersion }}.0-openjdk-devel"
  when:
    - ansible_distribution == 'CentOS' or ansible_distribution == 'Red Hat Enterprise Linux' or ansible_distribution == 'RedHat'
    - javaVersion | int < 9
# TODO: Java 9 & 10 packages are not avilable
- name: Install Openjdk | {{ javaVersion }} | RedHat | if java version is greater than 9
  yum:
    name: "{{ item }}"
    state: "{{ javaUpdateState | default('present') }}"
    update_cache: true
  loop:
    - "java-{{ javaVersion }}-openjdk"
    - "java-{{ javaVersion }}-openjdk-devel"
  when:
    - ansible_distribution == 'CentOS' or ansible_distribution == 'Red Hat Enterprise Linux' or ansible_distribution == 'RedHat'
    - javaVersion | int >= 9

Java 11 and higher packages had changed their names as compared to java 6/7/8 so I had to introduce conditions so it can work with old versions as well.

Request for Everyone, if you are using my Apache Kafka GitHub Project or Any other Project from my Profile, I request you to do the following.

1. Star & Fork the Project.
2. Please create Issues, if more features are required or something unexpected happens.
3. Please create PR, if you have a working solution for your problem.

Why above things, these things give a morale boost to open source project that they are being useful to someone and can use it as well.

Thank You for your Support so far!

The journey will continue on Next Topic ( Consumer Group Monitoring )!