Skip to content

Suffix properties file name with pod name #309

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from
Closed

Suffix properties file name with pod name #309

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
066477a
Makes static (pre-generated) configuration files possible
solsson Mar 6, 2020
bc2896a
Unlike broker config zookeeper's needs to know the replica count
solsson Apr 21, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions kafka/10broker-config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ data:
}
printf '%s\n' "${SEDS[@]}" | sed -f - /etc/kafka-configmap/server.properties > /etc/kafka/server.properties.tmp
[ $? -eq 0 ] && mv /etc/kafka/server.properties.tmp /etc/kafka/server.properties
ln -s /etc/kafka/server.properties /etc/kafka/server.properties.$POD_NAME

server.properties: |-
############################# Log Basics #############################
Expand Down
6 changes: 5 additions & 1 deletion kafka/50kafka.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,10 @@ spec:
- name: broker
image: solsson/kafka:2.4.0@sha256:201a1c7fd378405b6b1bfd801127c8a530ed7a971282bfcee4ec731bc0c50ad2
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: CLASSPATH
value: /opt/kafka/libs/extensions/*
- name: KAFKA_LOG4J_OPTS
Expand All @@ -62,7 +66,7 @@ spec:
containerPort: 5555
command:
- ./bin/kafka-server-start.sh
- /etc/kafka/server.properties
- /etc/kafka/server.properties.$(POD_NAME)
lifecycle:
preStop:
exec:
Expand Down
2 changes: 2 additions & 0 deletions zookeeper/10zookeeper-config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,14 @@ data:
export ZOOKEEPER_SERVER_ID=$((${HOSTNAME##*-} + $ID_OFFSET))
echo "${ZOOKEEPER_SERVER_ID:-1}" | tee /var/lib/zookeeper/data/myid
cp -Lur /etc/kafka-configmap/* /etc/kafka/
REPLICAS=$(( PZOO_REPLICAS + ZOO_REPLICAS ))
[ ! -z "$PZOO_REPLICAS" ] && [ ! -z "$ZOO_REPLICAS" ] && {
sed -i "s/^server\\./#server./" /etc/kafka/zookeeper.properties
for N in $(seq $PZOO_REPLICAS); do echo "server.$N=pzoo-$(( $N - 1 )).pzoo:2888:3888:participant" >> /etc/kafka/zookeeper.properties; done
for N in $(seq $ZOO_REPLICAS); do echo "server.$(( $PZOO_REPLICAS + $N ))=zoo-$(( $N - 1 )).zoo:2888:3888:participant" >> /etc/kafka/zookeeper.properties; done
}
sed -i "s/server\.$ZOOKEEPER_SERVER_ID\=[a-z0-9.-]*/server.$ZOOKEEPER_SERVER_ID=0.0.0.0/" /etc/kafka/zookeeper.properties
ln -s /etc/kafka/zookeeper.properties /etc/kafka/zookeeper.properties.scale-$REPLICAS.$POD_NAME

zookeeper.properties: |
4lw.commands.whitelist=ruok
Expand Down
11 changes: 10 additions & 1 deletion zookeeper/50pzoo.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,11 @@ spec:
- name: init-config
image: solsson/kafka-initutils@sha256:f6d9850c6c3ad5ecc35e717308fddb47daffbde18eb93e98e031128fe8b899ef
command: ['/bin/bash', '/etc/kafka-configmap/init.sh']
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
volumeMounts:
- name: configmap
mountPath: /etc/kafka-configmap
Expand All @@ -36,11 +41,15 @@ spec:
- name: zookeeper
image: solsson/kafka:2.4.0@sha256:201a1c7fd378405b6b1bfd801127c8a530ed7a971282bfcee4ec731bc0c50ad2
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: KAFKA_LOG4J_OPTS
value: -Dlog4j.configuration=file:/etc/kafka/log4j.properties
command:
- ./bin/zookeeper-server-start.sh
- /etc/kafka/zookeeper.properties
- /etc/kafka/zookeeper.properties.scale-5.$(POD_NAME)
lifecycle:
preStop:
exec:
Expand Down
10 changes: 9 additions & 1 deletion zookeeper/51zoo.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,10 @@ spec:
image: solsson/kafka-initutils@sha256:f6d9850c6c3ad5ecc35e717308fddb47daffbde18eb93e98e031128fe8b899ef
command: ['/bin/bash', '/etc/kafka-configmap/init.sh']
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: ID_OFFSET
value: "4"
volumeMounts:
Expand All @@ -39,11 +43,15 @@ spec:
- name: zookeeper
image: solsson/kafka:2.4.0@sha256:201a1c7fd378405b6b1bfd801127c8a530ed7a971282bfcee4ec731bc0c50ad2
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: KAFKA_LOG4J_OPTS
value: -Dlog4j.configuration=file:/etc/kafka/log4j.properties
command:
- ./bin/zookeeper-server-start.sh
- /etc/kafka/zookeeper.properties
- /etc/kafka/zookeeper.properties.scale-5.$(POD_NAME)
lifecycle:
preStop:
exec:
Expand Down