5a842ae155
This patch switches the kafka python client from kafka-python to confluent-kafka due to documented threading issues with the kafka-python consumer and the recommendation to use multiplrocessing. The confluent-kafka client leverages the high performance librdkafka C client and is safe for multiple thread use. This patch: * switches to confluent-kafka library * revises consumer and producer message operations * utilizes event.tpool method for confluent-kafka blocking calls * updates unit tests * adds kafka specific timeouts for functional tests * adds release note Depends-On: Ice374dca539b8ed1b1965b75379bad5140121483 Change-Id: Idfb9fe3700d882c8285c6dc56b0620951178eba2
21 lines
591 B
Bash
Executable File
21 lines
591 B
Bash
Executable File
#!/bin/bash
|
|
set -e
|
|
|
|
. tools/functions.sh
|
|
|
|
SCALA_VERSION=${SCALA_VERSION:-"2.12"}
|
|
KAFKA_VERSION=${KAFKA_VERSION:-"2.0.0"}
|
|
|
|
if [[ -z "$(which kafka-server-start)" ]] && [[ -z $(which kafka-server-start.sh) ]]; then
|
|
DATADIR=$(mktemp -d /tmp/OSLOMSG-KAFKA.XXXXX)
|
|
trap "clean_exit $DATADIR" EXIT
|
|
|
|
tarball=kafka_${SCALA_VERSION}-${KAFKA_VERSION}.tgz
|
|
|
|
wget http://www.apache.org/dist/kafka/${KAFKA_VERSION}/$tarball -O $DATADIR/$tarball
|
|
tar -xzf $DATADIR/$tarball -C $DATADIR
|
|
export PATH=$DATADIR/kafka_${SCALA_VERSION}-${KAFKA_VERSION}/bin:$PATH
|
|
fi
|
|
|
|
pifpaf run kafka -- $*
|