-
Notifications
You must be signed in to change notification settings - Fork 33
fix: avro-to-json operator fails to start when avroSchema property is JSON-string-encoded #335
Description
Summary
Application src/500-application/512-avro-to-json/operators/avro-to-json fails to initialize when the avroSchema configuration property is delivered as a JSON-encoded string (i.e. the schema JSON is wrapped in an extra pair of quotes). The operator logs Failed to parse provided Avro schema: Invalid schema name and returns false from avro_init, preventing the dataflow graph from loading.
Symptoms
In the AIO dataflow debug logs:
avro-to-json Failed to parse provided Avro schema: Invalid schema name
{"type":"record","name":"ExampleRecord","namespace":"com.example.namespace"...}
It must match the regex '^((?P<namespace>([A-Za-z_]...
avro-to-json Cannot start: avroSchema configuration is invalid.
Node 'avro_to_json' init call returned false for operator 'Map'
Failed to insert wasm component avro-to-json ...
Root Cause
The AIO runtime serialises module configuration properties as JSON strings. When a user sets
avroSchema to an Avro schema JSON object, the runtime may deliver the property value as a
JSON-string — the entire schema object encoded inside a surrounding JSON string literal:
"avroSchema" -> "{\"type\":\"record\",\"name\":\"ExampleRecord\", ...}"
Inside avro_init, the operator calls Schema::parse_str(v) directly on that value. The
apache-avro 0.17 parser (Schema::parse_str) first calls serde_json::from_str, which
correctly deserialises the outer string literal into a serde_json::Value::String containing
the schema text. The parser then attempts to match a Value::String as a known Avro primitive
type or a named schema reference — it calls Name::new() on the full JSON text as if it were a
type name. That fails with InvalidSchemaName because the Avro name validator regex rejects
anything containing {, ", or :.
The affected path in src/lib.rs (function avro_init):
Schema::parse_str(v) // v is already a JSON-string wrapping the real schema JSON