Sprint Goal #2 Feedback about connecting OGC API - Features Clients to OGC API - Connected Systems Servers

[Sam-Bolling]

CSAPI Sprint Goal #2 for Code Sprint 26:
Confirm that a generic API – Features client (such as QGIS without a plugin) can connect to a CSAPI server as expected without issues (i.e., discover and explore the feature resources)

Please post here any and all feedback you have relating to OGC Features clients interoperating with OGC CSAPI servers. Opinions, testimonials, experiences, complaints, concerns, issues, challenges, recommendations, and ideas of all kinds are welcome.

[SpeckiJ]

Goal

QGIS only understands OGC API Features, no notion about CSAPI. Despite this, as CSAPI is an extension of the OAPIF, it should support at least viewing of data.

General Observations:

• QGIS fetches a lot of metadata from the / and /conformance Endpoints
  • we should probably properly fill this out in the server implementations
• As QGIS only uses the /collections endpoint, all data that should be accessible must be in at least one collection
• All responses from the API expected as json and GeoJson FeatureCollection
• In CSAPI not all endpoints/collection actually have geometry attached (in Part 1 only Deployments actually have geometries)
  • QGIS can also handle non-spatial data fine, does not show in the map view (for obvious reasons) but can be seen + used using the attribute Table

Identified Issues/Problems:

• QGIS does not do handling/setting of response formats (via f parameter), just assumes valid response types

  • Can be enforced manually by appending the configured root url with <url>/?f=application/json
  • Note: All endpoints (incl. openapi) must then support the set mimetype. This is difficult as we have endpoints supporting geojson only and some supporting json only.
  • Would be solved by adjusting/setting default MIME Type to GeoJson (if available) and Json otherwise. AFAIK the spec does not mandate a default type.

• Data provided in Arrays is not available

  • For unknown reasons anything provided in an Array cannot be retrieved in QGIS (e.g. if there are multiple deployment->deployedSystems they are just listed as NULL)
    • This is normal JSON Syntax, why does this not work?

Requests:

List of requests that are sent out by QGIS (time-ordered):

Discovery of available datasets:

0. http://localhost:5000/?f=json&SERVICE=WFS&REQUEST=GetCapabilities&ACCEPTVERSIONS=2.0.0,1.1.0,1.0.0
--> We should tolerate such wierd requests with parameters we do not actually support and not throw an error here.
--> This is probably "detection" whether we are a WFS or OGC Feature Service (they are not separated in the GUI)

1. http://localhost:5000/?f=json
--> Check whether the OGC API Features Root works

2. http://localhost:5000/openapi?f=json
--> No idea what is used from here

3. http://localhost:5000/collections
--> Fetching all available collections

After choosing a collection (example here: all_deployments):

4. http://localhost:5000/openapi?f=json
--> No idea what is used from here, probably some metadata?

5. http://localhost:5000/collections/?f=json
--> fetching data already known, probably to refresh the cache?

6. http://localhost:5000/conformance
--> probably more metadata?

7. http://localhost:5000/collections/all_deployments/items?limit=10&f=json
--> actual data

8. http://localhost:5000/collections/all_deployments/items?limit=1000&bbox=12.08022794101994002,54.12694513166707111,12.08022945898005851,54.12694626833292944&f=json
--> actual data with bounding box corresponding to the current map view 

Examples:

{
  "type": "FeatureCollection",
  "features": [
    {
      "type": "Feature",
      "id": "462f8498-d95e-484e-9914-5617c2572078",
      "geometry": {
        "type": "Point",
        "coordinates": [
          12.0802287,
          54.1269457
        ]
      },
      "properties": {
        "uid": "urn:messtonne:1:2025-demo",
        "name": "Messtonne 1 - 2025 Test",
        "featureType": "http://www.w3.org/ns/sosa/Deployment",
        "validTime": null,
        "platform@link": {
          "href": "urn:platform:5300-909"
        },
        "deployedSystems@link": [
          {
            "name": "EXO3_Sonde",
            "system": {
              "href": "http://localhost:5000/systems/YSI599xxx-00-1",
              "urn": "urn:sensor:YSI599xxx-00-1"
            }
          },
          {
            "name": "DCPS_Sensor",
            "system": {
              "href": "http://localhost:5000/systems/5400-xxx",
              "urn": "urn:sensor:5400-526"
            }
          }
        ],
        "description": "Buoy was turned on and tested in the manufacturing hangar",
        "contacts": [
          {
            "role": "http://sensorml.com/ont/swe/property/Operator",
            "organisationName": "xxx",
            "contactInfo": {
              "website": "https://www.example.de"
            }
          }
        ],
        "platform": {
          "system": {
            "href": "urn:platform:5300-909"
          }
        }
      }
    }
  ],
  "links": []
}

Output in QGIS:

[SpeckiJ]

Latest Update:

For unknown reasons anything provided in an Array cannot be retrieved in QGIS (e.g. if there are multiple deployment->deployedSystems they are just listed as NULL)

By default QGIS guesses what types the individual properties have, as there is no embedded information about them in the actual entity (collection item) itself.

If the appropriate Conformance Classes & links (rel: http://www.opengis.net/def/rel/ogc/1.0/schema) are provided, QGIS queries and retrieves the schema of the collection items beforehand via http://localhost:5000/collections/all_deployments/schema?f=json

Unfortunately it refuses to accept the schema (shortened to the relevant components):

{
        "$schema": "https://json-schema.org/draft/2020-12/schema",
        "$id": "https://xxxx/schema",
        "title": "Ligplaats",
        "description": "asdf",
        "type": "object",
        "required": [
            "id"
        ],
        "properties": {
            "id": {
                "readOnly": True,
                "x-ogc-role": "id",
                "description": "The unique identifier of this feature. This ID should be stable over time and always refer to the same feature.",
                "type": "string",
                "format": "uuid"
            },
            "geometry": {
                "x-ogc-role": "primary-geometry",
                "format": "geometry-polygon"
            },
            "contacts": {
                "type": "array",
                "items": {
                  "type": "object"
                }
            },
            "deployedSystems@link": {
                "type": "array",
                "items": {
                  "type": "object"
                }
            }
        },
        "additionalProperties": True
    }

instead opting to throw even more NULL errors when trying to parse the actual data. Currently still unknown whether I provide something invalid or there is an (unknown to me) QGIS restriction

[SpeckiJ]

Relevant QGIS Code: https://github.com/qgis/QGIS/blob/4180bbf187853d5546151669d271a39ae015a0d1/src/providers/wfs/oapif/qgsoapifschemarequest.cpp#L121

[SpeckiJ]

We are not the only ones that have problems with this, e.g. see here: qgis/QGIS#51911 (comment)

I have checked the specification https://docs.ogc.org/DRAFTS/23-058r1.html#rc_schemas and apparently the supported types are not even defined, but only a recommendation (Recommendation 1 B).
Unfortunately we do not really have much choice in how we structure the properties as we already defined them in Section 19.2.6. here: https://docs.ogc.org/is/23-001/23-001.html#clause-sensorml-deployment. I hope other OGC API Features Clients are a bit more advanced in their json Parsing and can actually handle these objects.

Options I can see:

1. We document the restrictions with QGIS ala "due to technical limitations of QGIS some properties cannot be used there".

2. We develop some alternative Mapping that flattens the nested array-of-json-objects-with-properties structure into simple "string": "string" mappings. I'm not sure whether we actually want to do this, feels like quite a hacky job for not much benefit

[Sam-Bolling]

Greetings @SpeckiJ, and thank you for making https://csa.demo.52north.org/ available as a persistent, live reference implementation for other implementers to test with! I did some initial testing, trying to retrace the steps from your end of sprint demo. For context, I am not a seasoned, or even very experienced QGIS user. Here are my current results:

After adding an OpenStreetsMap (OSM) base layer through the QuickMapServices, I opened a Data Source Manager | WFS / OGC API - Features/Server Connections window by selecting Layer/Add Layer/Add WFS / OGC API Features Layer... from the primary menus ribbon. Here, I selected "new" which opened a Create a New WFS Connection window. I entered the URL https://csa.demo.52north.org/ and gave it the name 52 North CSAPI Server and clicked the OK button. I connected to it, and QGIS identified collections. I highlighted all of them and clicked the Add button.

This is where things seemed to run into issues. The all_datastreams collection appears to work. I can view metadata and its non-spatial entities in the attribute table with their metadata. However, the rest of the collections, although listed, did not appear to function. Instead, there is a warning icon by them that reads "Unavailable Layer! Layer data source could not be found. Click to set a new data source." When I click on the warning icon, I get a Repair Data Source pop-up menu, and at the bottom, it reads:

Original source URI: pagingEnabled='default' preferCoordinatesForWfsT11='false' restrictToRequestBBOX='1' srsname='OGC:CRS84' typename='dutch_windmills' url='https://csa.demo.52north.org/' version='auto'

I also visited the landing page directly in Microsoft Edge browser at the URL https://csa.demo.52north.org/. I noticed that the conformance page only lists https://www.opengis.net/spec/ogcapi-common-1/1.0/conf/core, but I think some generic features clients may be expecting declarations for OGC API - Features conformance classes here, and some CSAPI clients may expect declarations to OGC API - Connected Systems Parts 1 and 2 conformance classes.

I also noted that some pages try to resolve to a local host URL, and this may be related to some of the collections not working in QGIS.

Respectfully,

Sam

[SpeckiJ]

Sorry for the confusion, i forgot to explain that I actually tested it using the url:

https://csa.demo.52north.org/?f=application/geo%2Bjson

Which hardcodes the feature-type to geojson - this bypasses the Content-Type parsing of pygeoapi which does not really handle our mimetypes well. Specifically our implementation does not really deal well with the Accept Header containing multiple different keys in the case that they are all valid, but content is only available in some types.

I will try to develop a fix during this week and deploy a fixed version to the server, until then you need to deploy it using this workaround

[Sam-Bolling]

Thanks, Jan. I was able to reproduce almost all of your demo results with the new link you provided:

The only remaining issue was the inability to explore the collection of Dutch windmill feature resources, which may be related to links pointing to localhost. Overall, we can now reproduce this activity of connecting Feature clients to CSAPI servers, and we better understand the nuances and practices associated with it, thanks to your work! Another next step I look forward to is expanding the activity with additional feature clients beyond QGIS.

Respectfully,

Sam