Question: using sedona-functions directly as UDFs in datafusion

[nevi-me]

Hello

We have some hand-rolled geospatial functions in our codebase at work, we use datafusion as our execution engine with some custom data sources. The hand-rolled functions mimic PostGIS's ST_* functions, and we store geospatial data as BinaryArray that contains WKB types.

Question: do your UDFs only support SedonaType that's not SedonaType::Arrow(_)? I see that the UDF signatures require ArgMatcher::is_geometry() which isn't implemented for BinaryArray or BinaryViewArray.

Context

I tried replacing our functions with sedona-functions by registering all the exported functions as UDFs, however simple queries like:

select st_centroid(geom) from table;

fail with

Error performing query: "This feature is not implemented: st_centroid([Arrow(Binary)]): No kernel matching arguments"

For reference, the schema of the table looks something like:

+------------------------------------------+------------------------------------+----------+
| field name                               | type                               | nullable |
+------------------------------------------+------------------------------------+----------+
| id                                       | Int32                              | true     |
| name                                     | Utf8                               | true     |
| geom                                     | Binary                             | true     |
+------------------------------------------+------------------------------------+----------+

[paleolimbot]

Question: do your UDFs only support SedonaType that's not SedonaType::Arrow(_)? I see that the UDF signatures require ArgMatcher::is_geometry() which isn't implemented for BinaryArray or BinaryViewArray.

Yes, functions that operate on geometry require extension metadata to be present.

We will probably keep that for the internals because it's reasonably helpful for us and our users to know when extension metadata has been dropped. The short-term workaround would be to use ST_Centroid(ST_GeomFromWKB(geom)) but that is obviously not what you want your users to type if you're creating these expressions from DataFusion's built-in SQL parser. A slightly more involved workaround involves writing a wrapper function like:

#[derive(Debug, Hash, PartialEq, Eq)]
struct UdfWrapper {
    inner: ScalarUDF,
    signature: Signature,
    return_type: DataType,
    name: String,
}

impl ScalarUDFImpl for UdfWrapper {
    fn as_any(&self) -> &dyn std::any::Any {
        self
    }

    fn name(&self) -> &str {
        &self.name
    }

    fn signature(&self) -> &Signature {
        &self.signature
    }

    fn return_type(&self, _arg_types: &[DataType]) -> Result<DataType> {
        Ok(self.return_type.clone())
    }

    fn invoke_with_args(&self, args: ScalarFunctionArgs) -> Result<ColumnarValue> {
        let mut inner_args = args.clone();
       
        inner_args.arg_fields = inner_args
            .arg_fields
            .iter()
            // Add extension metadata using e.g., WKB_GEOMETRY.to_storage_field()
            .map(|f| ...)
            .collect();
        self.inner.invoke_with_args(args)
    }
}

If there's something we can do to make that process easier I'm happy to consider!