Skip to content

fix: avro-to-json operator fails to start when avroSchema property is JSON-string-encoded #335

@katriendg

Description

@katriendg

Summary

Application src/500-application/512-avro-to-json/operators/avro-to-json fails to initialize when the avroSchema configuration property is delivered as a JSON-encoded string (i.e. the schema JSON is wrapped in an extra pair of quotes). The operator logs Failed to parse provided Avro schema: Invalid schema name and returns false from avro_init, preventing the dataflow graph from loading.

Symptoms

In the AIO dataflow debug logs:

avro-to-json Failed to parse provided Avro schema: Invalid schema name
  {"type":"record","name":"ExampleRecord","namespace":"com.example.namespace"...}
  It must match the regex '^((?P<namespace>([A-Za-z_]...
avro-to-json Cannot start: avroSchema configuration is invalid.
Node 'avro_to_json' init call returned false for operator 'Map'
Failed to insert wasm component avro-to-json ...

Root Cause

The AIO runtime serialises module configuration properties as JSON strings. When a user sets
avroSchema to an Avro schema JSON object, the runtime may deliver the property value as a
JSON-string — the entire schema object encoded inside a surrounding JSON string literal:

"avroSchema" -> "{\"type\":\"record\",\"name\":\"ExampleRecord\", ...}"

Inside avro_init, the operator calls Schema::parse_str(v) directly on that value. The
apache-avro 0.17 parser (Schema::parse_str) first calls serde_json::from_str, which
correctly deserialises the outer string literal into a serde_json::Value::String containing
the schema text. The parser then attempts to match a Value::String as a known Avro primitive
type or a named schema reference — it calls Name::new() on the full JSON text as if it were a
type name. That fails with InvalidSchemaName because the Avro name validator regex rejects
anything containing {, ", or :.

The affected path in src/lib.rs (function avro_init):

Schema::parse_str(v)   // v is already a JSON-string wrapping the real schema JSON

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingrustPull requests that update rust code

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions