Details of MarkLogic Connector for Kafka

How MarkLogic Kafka connector works and what it entails? 

MarkLogic has provided a connector to consume the messages out of a Kafka topic, calling as SINK process. 

Main Features:

- It's a Java utility and provided with community support, no enterprise support available.

- Connector consumes the messages out of Kafka topic and MarkLogic document names are unique but automatically generated UUIDs and cannot be changed. Although uri prefix and document extension is supported.

Example documents: AABC-WERE-UUID-THIS.json, uri/AABC-WERE-UUID-THIS.json

Had a conversation with MarkLogic original developer and he mentioned that the code can be updated to mend as needed but MarkLogic won't do it.

- Connector consumes the messages in batches periodically which can be configured, how frequently and how many to consume each time.

- It supports 2 Way SSL with candidate application

- It does not support AVRO format as of today and cannot validate the messages based on a schema

Documentation: https://www.marklogic.com/blog/kafka-marklogic-connector/

Github link: https://github.com/marklogic-community/kafka-marklogic-connector