Apache Kafka Producer Reference
Stereotype Attributes
Kafka Producer Alias
Attribute | Description | Kafka Property | Allowed Values / Examples | |
|---|---|---|---|---|
Name | Specify a name for the alias. | - | any string | |
Standard | ||||
servers | Provide a list of Kafka brokers with (optional) ports. | bootstrap.servers | kafka1.acme.com, kafka2.acme.com:9092 | |
enableIdempotence | Enables the producer to write a message to Kafka exactly once. | enable.idempotence | true | Write message exactly once to the specified partition of the topic during the lifetime of the producer - without data loss and order per partition (default). |
false | Message may be delivered multiple times on e.g. connection failures. | |||
Advanced | ||||
additionalProperties | Use this to directly set properties of the Kafka producer. Using this attribute, you can also overwrite the dedicated settings above. In this case,
| buffer.memory=67108864, retries=4294967294 | ||
Compression | ||||
compressionType | Provide a compression codec for compressing message sets. | compression.type | none | No compression (default). |
gzip | Compress with gzip. | |||
Security | ||||
protocolSecurity | Provide the protocol used to communicate with the Kafka brokers. | security.protocol | plaintext | Use plain text (default). |
ssl | Use SSL. | |||
sslPrivateKeyFile | Provide the path to the client's private key file (PEM) to be used for authentication. | ssl.key.location | a valid path | |
sslPrivateKeyPassword | Provide the password for the private key file. | ssl.key.password | a valid password | |
sslPrivateKeyString | Provide the client's private key string (PEM format) to be used for authentication. | ssl.key.pem | a valid key string | |
sslClientCertificateFile | Provide the path to the client's public key file (PEM) to be used for authentication. | ssl.certificate.location | a valid path | |
sslClientCertificateString | Provide the client's public key string (PEM format) to be used for authentication. | ssl.certificate.pem | a valid key string | |
sslCALocation | Specify how to find CA certificate(s) for verifying the key of the Kafka broker. | ssl.ca.location | <path> | File or directory path to the CA certificate(s). |
probe | Guess the location of the CA store and try to find the CA certificate(s) automatically. | |||
bundled | Use the CA bundle provided with PAS. | |||
sslCAString | Provide the CA certificate string (PEM format) to verify the key of the Kafka broker. | ssl.ca.pem | a valid CA certificate string | |
Kafka Producer Adapter
Attribute | Description | Allowed Values |
|---|---|---|
alias | Specify the Kafka Producer alias the adapter should use to establish the connection. | any valid Kafka Producer alias |
KafkaProducerAdapter Operations
produce
Attribute | Type | Direction | Description |
|---|---|---|---|
producerRecord | in | Provide the message and the target for the adapter to send the message to a Kafka topic. | |
recordMetaData | out | Returns some meta data about the processing the Kafka broker has done. |
KafkaProducerAdapter Parameter Types
MessageHeader
Attribute | Type | Description |
|---|---|---|
key | String | Key of the header. |
value | Blob | Value of the header. |
RecordMetaData
Attribute | Type | Mandatory | Description | Allowed Values / Examples | |
|---|---|---|---|---|---|
topic | String |
| Kafka topic the message has been sent to. | ||
partition | Integer |
| Topic partition the message has been sent to. | ||
offset | Integer |
| Offset of the message within the partition. | ||
recordId | Integer |
| The message ID as provided with the call, or as generated if it has not been specified. | ||
keySize | Integer |
| Actual size of the key in bytes. | ||
valueSize | Integer |
| Actual size of the value in bytes. | ||
timestamp | DateTime |
| A processing timestamp (UTC). See timestampType for what the timestamp refers to. | ||
timestampType | TimestampType |
| Indicates what the timestamp refers to. | NotAvailable | Timestamp is unknown. |
CreateTime | Timestamp relates to message creation time as set by a Kafka broker. | ||||
LogAppendTime | Timestamp relates to the time a message was appended to a Kafka log. | ||||
persistedStatus | PersistedStatus |
| Status of the message. | NotPersisted | Message was never transmitted to the broker, or failed with an error indicating it was not written to the log. |
Persisted | Message was written to the log and acknowledged by the broker. | ||||
PossiblyPersisted | Message was transmitted to broker, but no acknowledgement was received. | ||||
ProducerRecord
Attribute | Type | Mandatory | Description |
|---|---|---|---|
topic | String |
| Specify the topic the message should be sent do. |
partition | Integer |
| Specify the partition the message should be send to. |
key | Blob |
| Specify a key the message should be filed to. |
id | Integer |
| Will be generated. We recommend not to set one manually. |
headers | Array of MessageHeader |
| Specify headers if you want to provide additional information on the message content for e.g. routing or filtering use cases. |
value | Blob |
| Specify the actual message content. Only payload of type Blob is supported. |
Related Documentation: