As a curator, I want to more easily add metadata about related resources so that it's more discoverable

[jggautier]

While working on exporting more dataset metadata in Schema.org (#4371), the team talked about the best way to include the URLs of related publications. (In Schema.org, it's best practice to use a URL, instead of an ID number, for related publications.) But not all depositors/curators enter related publication metadata the way we expect (the way the fields have been designed), which has and will lead to significant curation work to correct the metadata so that it's exported in different metadata formats (I'm thinking of Schema.org and DataCite).

This is what the Dataverse 4 relatedPublication compound fields look like:

For the Schema.org issue (#4371), we decided to use what's entered in the URL field.

But there are plenty of cases where:

• what the depositor enters in the URL field is not the URL form of the included DOI or other PID (which is what the picture above shows)
• nothing is entered in the URL field
• the URL form of a PID is included, but nothing is entered in the ID Type and/or ID Number fields

So not all possible related publication metadata will be exported, and discoverable in other systems, without curation work to update the metadata after it's been published.

For DataCite schema, Dataverse needs to know what are the ID Types and identifiers of related publications, other datasets and software. For Schema.org, Dataverse needs to know what's the URL of related publications. (As of this issue, there's no recommended way to include in Schema.org metadata about other related datasets and software.)

For other datasets and software, Dataverse doesn't have fields that are meant to collect URLs, ID Types or ID Numbers:

Field for related software used to create the data (this might change with a planned software metadata block):

Field for related datasets:

Can we improve the metadata fields (such as the number of fields, how they're labelled and described) and/or how Dataverse uses what's entered in metadata fields (such as text parsing, getting more information from external sources) in order to reduce the curation and training needed now to make sure Dataverse is more capable of exposing information about related resources?

[jggautier]

This is the metadata that the DataCite 4.1 schema accepts for relatedIdentifiers:

<relatedIdentifier resourceTypeGeneral="Dataset" relatedIdentifierType="DOI" relationType="IsIdenticalTo">10.25377/sussex.7133897</relatedIdentifier>

(There are small controlled vocabs for resourceTypeGeneral, relatedIdentifierType, and relationType (see #2778).)

For the relatedIdentifier value, where the identifier 10.25377/sussex.7133897 is, the schema technically allows anything. We could add the URL form of the DOI instead, i.e. https://doi.org/10.25377/sussex.7133897 and it would be valid against the schema:

<relatedIdentifier resourceTypeGeneral="Dataset" relatedIdentifierType="DOI" relationType="IsIdenticalTo">https://doi.org/10.25377/sussex.7133897</relatedIdentifier>

But can DataCite use the URL form (e.g. for exposing related identifiers for MakeDataCount (#4821))? Or are they expecting only the identifier?

[amberleahey]

Just had a conversation with a user who would also like to see the related data set field be more structured to support discovery and readability online (as the related publication one is structured). Currently they are adding HTML tags to format this but could be enhanced directly in the form. Thanks for considering!

[pdurbin]

@amberleahey this is great feedback. Thanks. Please see also some recent discussion about "Related Dataset" in the context of Make Data Count at #4821 (comment)

[jggautier]

"Related material" is as another unstructured metadata field used to record how another research object is related to the dataset. I've been looking into how people depositing datasets in Dataverse installations use that field (what information do they put in it), and what "related material" metadata Harvard Dataverse stores when it imports DDI metadata from non-Dataverse repositories.

DDI's definition of related material sounds like a notes field:

Describes materials related to the study description, such as appendices, additional information on sampling found in other documents, etc. Can take the form of bibliographic citations. This element can contain either PCDATA or a citation or both, and there can be multiple occurrences of both the citation and PCDATA within a single element. May consist of a single URI or a series of URIs comprising a series of citations/references to external materials which can be objects as a whole (journal articles) or parts of objects (chapters or appendices in articles or documents).

Example

<relMat> Full details on the research design and procedures, sampling methodology, content areas, and questionnaire design, as well as percentage distributions by respondent's sex, race, region, college plans, and drug use, appear in the annual ISR volumes MONITORING THE FUTURE: QUESTIONNAIRE RESPONSES FROM THE NATION'S HIGH SCHOOL SENIORS.</relMat> <relMat>Current Population Survey, March 1999: Technical Documentation includes an abstract, pertinent information about the file, a glossary, code lists, and a data dictionary. One copy accompanies each file order. When ordered separately, it is available from Marketing Services Office, Customer Service Center, Bureau of the Census, Washington, D.C. 20233. </relMat> <relMat>A more precise explanation regarding the CPS sample design is provided in Technical Paper 40, The Current Population Survey: Design and Methodology. Chapter 5 of this paper provides documentation on the weighting procedures for the CPS both with and without supplement questions.</relMat>

I've found so far that people who use that field often enter related research objects that could be described as one of DataCite's resourceTypes (pg. 43) and aren't as wordy about describing the resource: https://dataverse.harvard.edu/dataverse/harvard?q=relatedMaterial:*.

What's entered in related material is also less directly related to the dataset, maybe because the tooltip for related publication is "Publications that use the data from this dataset."

[marcomarsella]

I manage a system assigning DOIs to Plant Genetic Resources (PGRs). Many of our stakeholders deposit datasets to Dataverse. It would be very useful to be able to cite the DOIs of the PGRs to which a dataset refers because this connection would be fed to EventData, a service that we already use to discover data citation of our DOIs from papers and other publications.
The cited DOIs should be put in the <relatedIdentifiers> element of the metadata schema with relationType="References". This would make DataCite automatically record the citation in EventData.

[jggautier]

That was fast. Thanks @marcomarsella (for following up from our conversation over email)!

There's been discussion about which of the many relationTypes to use (some of the discussion is in the issue at #2778) and I think that would have to be resolved.

It sounds like you're advocating for always using relationType="References", regardless of the type of related research object. Is that right? Is the system you manage picking up related DOIs only when the relationType is "References"? Would it be okay to use a "one-way" relation or edge to describe every relationship between two datasets?

I forgot to mention that there are some Dataverse repositories that are already sending some of this information to DataCite (the two I'm aware of are QDR and Repositorio de Datos del Consorcio Madroño). But each is using one relationType to indicate a relationship between the datasets they publish and related text-based research objects (mostly journal articles and reports), so using the same relationType makes more sense to me.

I pointed to this issue because it's about all of the types of information Dataverse's metadata fields need to capture but aren't (at least not in a structured way), like persistent IDs. So Dataverse would need more fields, need to be able to parse persistent IDs from unstructured text, or somehow get information from other systems about datasets that are related to the datasets published by the Dataverse repository.

[marcomarsella]

The operators that EventData would capture are listed in <https://support.datacite.org/docs/relationtype_for_citation> However, to avoid confusion and to facilitate querying, I think it would be good to stick to one relation operator only. M

…

On 29 Jun 2020, at 18:44, Julian Gautier ***@***.***> wrote: That was fast. Thanks @marcomarsella <https://github.com/marcomarsella>! There's been discussion about which of the many relationTypes to use (some of the discussion is in the issue at #2778 <#2778>) and I think would have to be resolved. But it sounds like you're advocating for always using relationType="References", regardless of the type of research object. Is that right? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5277 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADNUBUI2BS5UNGCRET67JI3RZDAGXANCNFSM4GB4EP2Q>.

[pkiraly]

@jggautier @pdurbin We have a similar feature request from a Göttingen research group Digital Geochemistry Infrastructure (DIGIS). I would like to work on this feature, but I am unsure which ticket is the best one. There are some parallel which are mentioned in this ticket or in the tickets mention this ticket. I haven't seen any commit message or pull requests in this direction. Are you OK if I start working on it?

The implementation doesn't seem to be rocket sience at the first sight, all we need to do is modify a single method: DOIDataCiteRegisterService.generateRelatedIdentifiers(), and implement XML generation out of fields such as

• Related Publication
• Related Material
• Related Datasets
• Other References
• Data Sources
• Origin of Sources

applying the appropriate Datacite attributes.

[qqmyers]

FWIW: #2778 has lots of discussion about how various Dataverse fields could be mapped to various DataCite relationship types. As you say, implementation isn't hard (see https://github.com/QualitativeDataRepository/dataverse/blob/8a597dbd8c1f5e28f1cf7403efdba642d2bc6751/src/main/java/edu/harvard/iq/dataverse/DOIDataCiteRegisterService.java#L578-L660 for example), but finding mappings that work for everyone (or are configurable, etc.) is the challenge.