Avro Conversions

Provides parsers to read various formats as Avro records (currently JSON and XML), as well as tools to manipulate and/or describe schemas. The latter includes the conversion of XML Schema definitions (XSD) and JSON Schema into Avro schemas.

Documentation

The documentation is split into two parts. The impatient among us can start with the "Quickstart" section below. For the others, there is more elaborate documentation.

Quickstart

Want to jump right in? Here's a simple example to read a JSON schema, manipulate in into an Avro schema, and then parse a JSON file into the (different) Avro records:

import opwvhk.avro.SchemaManipulator;
import opwvhk.avro.json.JsonAsAvroParser;
import opwvhk.avro.util.NamingConvention;
import org.apache.avro.Schema;
import org.apache.avro.generic.GenericRecord;

class Example {
	public static void main(String[] args) throws Exception {
		// Expected are 2 arguments:
		// * the location of a JSON schema file
		// * the location of a JSON file
		URL jsonSchemaLocation = new URL(args[0]);
		URL jsonFileLocation = new URL(args[1]);

		StringBuilder buffer = new StringBuilder();
		// Read an Avro schema; other methods can convert an XSD or JSON schema.
		Schema readSchema = SchemaManipulator.startFromJsonSchema(jsonSchemaLocation)
				.useSchemaNamingConvention(NamingConvention.PASCAL_CASE)
				.useFieldNamingConvention(NamingConvention.CAMEL_CASE)
				.renameField("newName", "path", "to", "field")
				.renameSchema("OldName", "NewName")
				.alsoDocumentAsMarkdownTable(buffer)
				.finish();
		// Print the schema as a Markdown table
		System.out.println(buffer);

		// Create a parser that reads JSON, validated according to the JSON schema, into Avro
		// records using a read schema.
		GenericData model = GenericData.get();
		JsonAsAvroParser parser = new JsonAsAvroParser(jsonSchemaLocation, readSchema, model);

		// The record type depends on the class generated by GenericData.get() (you can also use SpecificData or ReflectiveData).
		GenericRecord record = parser.parse(jsonFileLocation);
		System.out.println(record);
	}
}

Upgrade notes

From 1.3 to 2.0

1. XML validation has been changed. Previously you could choose whether to validate per input. Now, whether inputs will be validated is a fixed setting per parser instance. See the constructor parameters for details.
2. Constructing a JSON parser can no longer throw a (library specific) GenerationException; this has become a runtime exception instead.

Logging

This library uses SLF4J, redirects Commons-Logging over SLF4J, and includes a dependency to direct the JDK 1.4 logging framework to SLF4J as well (one of the dependencies uses it). To correctly log via SLF4J, use the provided file logging.properties to configure java.util.logging.

Contributing ✨

Are you interested in contributing? These links may be of interest:

• Contributing Guidelines
• Good First Issues