GSSAPI support for Kafka in Python

[Andrew Smith]

When attempting to consume a topic from a Windows hosted Kafka server the connection fails due to the authority mechanisms supported by the server.

(kafka-consumer) N00SYS02:/home/NAVASM/kafka-consumer/src # ./consume-test.py
Traceback (most recent call last):
File "./consume-test.py", line 9, in <module>
auto_offset_reset = 'earliest', sasl_mechanism = 'GSSAPI', security_protocol = 'SASL_PLAINTEXT')
File "/home/NAVASM/kafka-consumer/lib/python3.6/site-packages/kafka/consumer/group.py", line 356, in init
self._client = KafkaClient(metrics=self._metrics, **self.config)
File "/home/NAVASM/kafka-consumer/lib/python3.6/site-packages/kafka/client_async.py", line 244, in init
self.config['api_version'] = self.check_version(timeout=check_timeout)
File "/home/NAVASM/kafka-consumer/lib/python3.6/site-packages/kafka/client_async.py", line 901, in check_version
self._maybe_connect(try_node)
File "/home/NAVASM/kafka-consumer/lib/python3.6/site-packages/kafka/client_async.py", line 381, in _maybe_connect
**self.config)
File "/home/NAVASM/kafka-consumer/lib/python3.6/site-packages/kafka/conn.py", line 273, in init
assert gssapi is not None, 'GSSAPI lib not available'
AssertionError: GSSAPI lib not available

Using KafkaConsumer in Python 3.6 and missing the GSSAPI library.

When attempting to install GSSAPI libs, the krb5-config command is missing on IBM i.

Is this available from the IBM repo? Can’t see krb5-libs on the ACS OSS management tool and not able to install via yum

This is running on an IBM i with V7R3.

Any ideas? They’d be really appreciated

‌

[Kevin Adler]

Kerberos and Generic Security Services are currently not supported in PASE, though it is something we’re investigating.

[Andrew Smith]

Thanks Kevin - at least now I know.

I’ll see if there are workarounds that I can use instead.

Thanks again

‌

[Andrew Smith]

Not currently supported in PASE

[Jesse G]

Your best bet is probably to use the Java client that comes with the main Kafka project from Apache. That will require some Java Kerberos setup, which can get messy (or not, depending on your background), but the GSS support should be there. And you may be also able to wire it into the Kafka component of Apache camel, which has options for GSS.