LinkedIn
Copied!

Table of Contents

Configuring Avro schema for Kafka data set

When you configure a Kafka data set, you can choose Apache Avro as your data format for the Kafka message values and message keys. Avro is a lightweight binary message encoding that is at least two times smaller than regular JSON.

Avro relies on schemas, which are held in external storage. Your keys and values may have different schemas, which will be stored in different registries. Configure an Avro schema by following the steps below.
Open your Kafka data set creation ruleform. If you do not have the Kafka data set creation ruleform opened, follow the instructions here: Creating a Kafka data set.
  1. In your Kafka data set creation ruleform, in the Message values or Message keys section, select Avro from the Data format list.

  2. Click the Open icon next to the Avro schema field.

  3. In the Short description field enter a short description or title.

  4. In the Avro Schema field, enter an Avro schema name.

    Your Avro schema name must start with a letter.
  5. Click Create and Open.

  6. In the Schema registry URL field, enter the URL where the schema will be stored.

  7. Choose Authentication source.

  8. Optional:

    To associate keys and values with your Avro schema, click Add key-value pair.

  9. Optional:

    To upload your Avro schema, click the Upload schema button.

  10. Optional:

    Select the Perform schema evolution when reading messages check box, to use versioning of the schemas. Click Upload different schema button, to upload your second schema.

  11. Click Save.

Have a question? Get answers now.

Visit the Collaboration Center to ask questions, engage in discussions, share ideas, and help others.