JSON-LD context for search operations, pagination

[rduyme]

At the moment the OpenAPI search operations do not contain any JSON-LD context.

I plan to create a JSON-LD context with paging elements using approach done in IIIF

https://iiif.io/api/search/2.0/

It is based on https://www.w3.org/ns/activitystreams# vocabulary.

(see response examples and context in https://iiif.io/api/search/2.0/#responses

Idea is to :

• create a separated search context json file to map json fields (meta, count, page, page_size, results...).
• use id and type the same way.

context : "https://w3id.org/skg-if/context/1.0.1/skg-if.json"  +  search context
"id": "https://example.org/service/manifest/search?q=bird&motivation=painting",
 "type": "AnnotationPage",  ( as:OrderedCollection )

we could keep this search context file in api github repository and also mapped in w3id.org

Is this approach ok ?
@essepuntato, @andremann

[rduyme]

here is proposition, including ActivityStream context elements. https://www.w3.org/TR/activitystreams-core/

This is JSON-LD result for get list of products endPoint : https://example.org/api/products?filters=cf.search.title:bird&page=1

{
"@context": [
        "https://w3id.org/skg-if/context/1.0.1/skg-if.json",
        {
            "@base": "http://example.com/",
            "as": "http://www.w3.org/ns/activitystreams#",
            "OrderedCollectionPage": {
                "@id": "as:OrderedCollectionPage"
            },
            "partOf": {
                "@id": "as:partOf"
            },
            "totalItems": {
                "@id": "as:totalItems"
            },
            "nextPage": {
                "@id": "as:next"
            },
            "firstPage": {
                "@id": "as:first"
            },
            "lastPage": {
                "@id": "as:last"
            },
            "prevPage": {
                "@id": "as:prev"
            },
            "items": {
                "@type": "@id",
                "@id": "as:items",
                "@container": "@list"
            }
        }
    ],
    "type": "OrderedCollectionPage",
    "local_identifier": "https://example.org/api/products?filters=cf.search.title:bird&page=1",
    "partOf": {
        "local_identifier": "https://example.org/api/products?filters=cf.search.title:bird",
        "type": "OrderedCollectionPage",
        "totalItems": 125,
        "firstPage": {
            "local_identifier": "https://example.org/api/products?filters=cf.search.title:bird&page=1",
            "type": "OrderedCollectionPage"
        },
        "lastPage": {
            "local_identifier": "https://example.org/api/products?filters=cf.search.title:bird&page=13",
            "type": "OrderedCollectionPage"
        }
    },
    "nextPage": {
        "local_identifier": "https://example.org/api/products?filters=cf.search.title:bird&page=2",
        "type": "OrderedCollectionPage"
    },
    "items": [
        {
            "local_identifier": "xxx",
            "entity_type": "product",
            "product_type": "literature"
        },
        {
            "local_identifier": "yyy",
            "entity_type": "product",
            "product_type": "research data"
        }
    ]
}

https://json-ld.org/playground/

as: xxxxx context elements will be externalized in a dedicated JSON file.

"@context": [
        "https://w3id.org/skg-if/context/1.0.1/skg-if.json",
        "https://w3id.org/skg-if/api/context/skg-if-search.json", <= 
        {
            "@base": "http://example.com/",
         }
]

[essepuntato]

Hi @rduyme,

Just a quick comment here: in order to keep the same structure of the format created for SKG-IF, would it be better to use "@graph" instead of "items"? In this way, it would already be compliant with what we have done so far, and does not seem distant from SKG-IF at all - and, in addition, we avoid to use rdf:List that are not OWL2 compliant.

In addition, in order to distinguish even better the part related to the metadata of the request and the part related to the actual data returned, it would be reasonable for me to group the metadata of the request in a syntactic block using the JSON-LD nesting. Just to make an example:

{
"@context": [
    "https://w3id.org/skg-if/context/1.0.1/skg-if.json",
    "https://w3id.org/skg-if/api/context/skg-if-search.json",
    {
        "@base": "http://example.com/",
    }
],
"meta": {
    "type": "OrderedCollectionPage",
    "local_identifier": "https://example.org/api/products?filters=cf.search.title:bird&page=1",
    "partOf": {
        "local_identifier": "https://example.org/api/products?filters=cf.search.title:bird",
        ... 
    }, ...
},
"@graph": [
    {
        "local_identifier": "xxx",
        "entity_type": "product",
        "product_type": "literature"
    },
    {
        "local_identifier": "yyy",
        "entity_type": "product",
        "product_type": "research data"
    }, ...
]
}

In addition, I think it would be appropriate to keep the same convention for naming the JSON properties when they are composed words, i.e. using the _ for separating them (e.g. use part_of instead of partOf).

What do you think?

[rduyme]

Thanks for the feedback Silvio,

• meta nested => yes
• underscore (e.g. use part_of instead of partOf). => yes

For "@graph"

• Could we hide it in JSON under an alias like "items" this way ?

 "items": {
                "@type": "@id",
                "@id": "as:items",
                "@container": "@graph"
          }  

• Don't we loose the "order" semantic ( OrderedCollectionPage ) using @graph ?

[essepuntato]

In principle, we can hide it using "items" as an alias of "@graph", yes (but still using the square brackets for containment). My worry here is that we are going to change the IF, and we have to adapt all the various examples already developed so far. @andremann, what do you think?

About the "semantic order", yes: indeed, we are going to lose it, but I fear that presenting the data returned by the API using a different format to those prepared for the dumps (as prescribed via the IF) is far more cognitively complicated and adds also complexities related to the OWL vs RDF behaviour I was mentioning above.

[rduyme]

I plan to use a dedicated search context file, only used and mentioned in OpenAPI specifications. The YAML OpenAPI specification is by definition forcing the structure the JSON output.

For example, the only single acceptable JSON output in the OpenAPI operation GetProductById is this one for Agent section :

    "by": {
      "local_identifier":"xxxxxx" 
      "name": "John Doe"
     }

It is embedded, it is the YAML specification. You have many OpenAPI tools that can validate a json output is aligned with the YAML OpenAPI. This choice of flat embedding is to have something similar to all existing standard API (crossref, datacite, openAIRE, pubmed, OpenAlex etc...). and hide any track of RDF in the context ( no @id no @type => no @graph ). For me that is the promise of JSON-LD. My fear is to have implementers running away when seeing an @graph, instead of a common json array :-)

For your concern @essepuntato.
Most of examples provided ex: https://github.com/skg-if/examples/blob/main/OpenCitations/oc_1.jsonld are not following this Agent embedding pattern, and still are OK, because we don't state anywhere they are examples of the OpenAPI. These are just JSON-LD examples compatible with the skg-if context they include. We add this discussion here : #4

You can put these examples in https://json-ld.org/playground/, they will not crash.
These examples are ok if you have triple store users : server side and client side.
These examples are not totally aligned with the snapshots guidelines : https://skg-if.github.io/snapshot/ (products and agents should be in separated files)

And in the end, no problem, these examples are not actually related to items array of this future search context : https://w3id.org/skg-if/api/context/skg-if-search.json (we try to create) : the examples don't include this search context, and are not about any searching feature. They are standalone examples.

Sorry Silvio, too long again :-)

[rduyme]

Here is proposition for the context elements, based on Silvio's feedback.

{
"@context": [
        "https://w3id.org/skg-if/context/1.1.0/skg-if.json",
        {
            "@base": "http://example.com/",

           // all search context terms below should finally be in https://w3id.org/skg-if/context/1.1.0/skg-if.json and not in a separated search json context file. 
            "as": "http://www.w3.org/ns/activitystreams#",
            "search_result_page": {
                "@id": "as:CollectionPage"
            },
          	"search_result": {
                "@id": "as:Collection" // Collection and CollectionPage, no order.
            },
            "part_of": {
                "@id": "as:partOf"
            },
          	"count": {
                "@id": "as:totalItems"
            },
          	"page_size": {
                "@id": "as:totalItems"
            },
          	"page": {
                "@id": "schema:position" // no page number semantic in activitystreams we can use schema.org position
            },
            "meta":"@nest" // nested as suggested by Silvio.
        }],
    "meta": {
      	"@id": "https://example.org/api/products?filters=cf.search.title:bird&page=1&page_size=10", // note : to be discussed : ok to use @id and not SKG-IF alias local_identifier ?
      	"entity_type": "search_result_page", // we hide @type in context (easier to maintain by only updating the context and not client implementations)
      	"page_size": 10, // note : to be discussed, what about last page where you can have less ? (do we keep 10 or less ?)
      	"page":1,
      	"part_of": {
          "@id": "https://example.org/api/products?filters=cf.search.title:bird",
          "entity_type": "search_result", // we hide @type in context (easier to maintain)
          "count": 125
      	}
        // Proposition no need for as:nextPage, as:firstPage, as:lastPage concepts in SKG-IF, to keep the 'meta' section simple.
    },
    "@graph": [ // we use @graph as suggested by Silvio.
        {
            "local_identifier": "xxx",
            "entity_type": "product",
            "product_type": "literature"
        },
	{
            "local_identifier": "yyy",
            "entity_type": "product",
            "product_type": "literature"
        }

    ]
}

Copy paste example without comments for https://json-ld.org/playground/

{
"@context": [
        "https://w3id.org/skg-if/context/1.1.0/skg-if.json",
        {
            "@base": "http://example.com/",
            "as": "http://www.w3.org/ns/activitystreams#",
            "search_result_page": {
                "@id": "as:CollectionPage"
            },
          	"search_result": {
                "@id": "as:Collection"
            },
            "part_of": {
                "@id": "as:partOf"
            },
          	"count": {
                "@id": "as:totalItems"
            },
          	"page_size": {
                "@id": "as:totalItems"
            },
          	"page": {
                "@id": "schema:position"
            },
             "meta":"@nest"
        }],
    "meta": {
      	"@id": "https://example.org/api/products?filters=cf.search.title:bird&page=1&page_size=10", 
      	"entity_type": "search_result_page", 
      	"page_size": 10, 
      	"page":1,
      	"part_of": {
          "@id": "https://example.org/api/products?filters=cf.search.title:bird",
          "entity_type": "search_result", 
          "count": 125
      	}
    },
    "@graph": [ 
        {
          "local_identifier": "https://example.org/api/products/xxx",
            "entity_type": "product",
            "product_type": "literature"
        },
	{
            "local_identifier": "https://example.org/api/products/yyy",
            "entity_type": "product",
            "product_type": "literature"
        }

    ]
}

Canonized https://json-ld.org/playground/ output

<https://example.org/api/products/xxx> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://purl.org/spar/fabio/ScholarlyWork> <https://example.org/api/products?filters=cf.search.title:bird&page=1&page_size=10> .
<https://example.org/api/products/xxx> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://purl.org/spar/fabio/Work> <https://example.org/api/products?filters=cf.search.title:bird&page=1&page_size=10> .
<https://example.org/api/products/yyy> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://purl.org/spar/fabio/ScholarlyWork> <https://example.org/api/products?filters=cf.search.title:bird&page=1&page_size=10> .
<https://example.org/api/products/yyy> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://purl.org/spar/fabio/Work> <https://example.org/api/products?filters=cf.search.title:bird&page=1&page_size=10> .
<https://example.org/api/products?filters=cf.search.title:bird&page=1&page_size=10> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://www.w3.org/ns/activitystreams#CollectionPage> .
<https://example.org/api/products?filters=cf.search.title:bird&page=1&page_size=10> <http://www.w3.org/ns/activitystreams#partOf> <https://example.org/api/products?filters=cf.search.title:bird> .
<https://example.org/api/products?filters=cf.search.title:bird&page=1&page_size=10> <http://www.w3.org/ns/activitystreams#totalItems> "10"^^<http://www.w3.org/2001/XMLSchema#integer> .
<https://example.org/api/products?filters=cf.search.title:bird&page=1&page_size=10> <https://schema.org/position> "1"^^<http://www.w3.org/2001/XMLSchema#integer> .
<https://example.org/api/products?filters=cf.search.title:bird> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://www.w3.org/ns/activitystreams#Collection> .
<https://example.org/api/products?filters=cf.search.title:bird> <http://www.w3.org/ns/activitystreams#totalItems> "125"^^<http://www.w3.org/2001/XMLSchema#integer> .

[rduyme]

resolved by PR skg-if/api#48