Get as much info from Loculus into ENA as possible

[theosanderson]

For now there are some unmapped fields that don't make it through (Theo will update this with more info).

[claude]

This issue may be related to #2983 (Improve assembly field mapping), which also tracks improving the field data that gets passed through to ENA during deposition. Please check if there is overlap.

[theosanderson-agent]

Unmapped Pathoplexus fields in ENA deposition

Based on the current field mappings in ena-submission/config/defaults.yaml and the Pathoplexus metadata schema in loculus_values/values.yaml, the following user-submittable fields are not currently mapped to ENA during deposition:

Sample collection details:

• geoLocSite, sampleType, presamplingActivity, collectionDevice, collectionMethod, cultureId

Specimen details:

• foodProduct, foodProductProperties, specimenProcessing, specimenProcessingDetails, experimentalSpecimenRoleType

Host details:

• hostOriginCountry, hostDisease, hostRole, hostAgeBin, hostTaxonId, isLabHost, hostVaccinationStatus

Exposure & travel:

• travelHistory, exposureSetting, exposureDetails

Infection history:

• previousInfectionDisease, previousInfectionOrganism

Sequencing details:

• sequencingProtocol, sequencingAssayType, sequencingDate, ampliconPcrPrimerScheme, ampliconSize, purposeOfSequencing

Sequencing contact:

• sequencedByContactName, sequencedByContactEmail

Bioinformatics:

• rawSequenceDataProcessingMethod, dehostingMethod, assemblyReferenceGenomeAccession, breadthOfCoverage

Quality control:

• qualityControlMethodName, qualityControlMethodVersion, qualityControlDetermination, qualityControlIssues, qualityControlDetails

Diagnostics:

• diagnosticMeasurementMethod, diagnosticTargetPresence, diagnosticTargetGeneName, diagnosticMeasurementValue, diagnosticMeasurementUnit

Culture/passage:

• cellLine, passageNumber, passageMethod

Other:

• comment

Note: The collector name ENA field is currently mapped to an empty list with a default of "not provided" — sequencedByContactName could potentially fill this. Also, sequencingProtocol is available but only sequencingInstrument is mapped (to PLATFORM in the assembly manifest). The config also has a comment noting #ena_checklist: ERC000033 - do not use until all fields are mapped to ENA accepted options.

Some of these fields (e.g. hostDisease, travelHistory, passageNumber/passageMethod) have direct counterparts in the ENA virus pathogen reporting checklist (ERC000033) and could be mapped relatively straightforwardly.

[theosanderson-agent]

Detail on unmapped fields with ENA ERC000033 equivalents

Cross-referencing the unmapped Pathoplexus fields against the ENA virus pathogen reporting checklist (ERC000033):

Fields with direct or near-direct ERC000033 equivalents

Unmapped Pathoplexus field	ERC000033 field	ENA status	Notes
sequencedByContactName	collector name	mandatory	Currently mapped to [] with default "not provided". This is a mandatory ENA field that we're defaulting away — mapping sequencedByContactName here would improve data quality.
geoLocSite	geographic location (region and locality)	recommended	Currently only geoLocAdmin1, geoLocAdmin2, geoLocCity are mapped. geoLocSite could be appended for more specific location info.
hostDisease	host_description	optional	ERC000033's host_description accepts "other descriptive information relating to the host" — hostDisease fits here.
hostRole	host_description	optional	Could also contribute to host_description.
isLabHost / cellLine	lab_host	optional	ERC000033 has lab_host for "scientific name of laboratory host used for propagation". isLabHost + cellLine could inform this.
exposureSetting	host_habitat	recommended	ERC000033's host_habitat covers "domestic farm types, live market, wild migratory/resident" — exposureSetting overlaps with this.
exposureDetails	subject_exposure	optional	Currently only exposureEvent maps to subject exposure. exposureDetails could be appended for richer data.
travelHistory	subject_exposure	optional	Travel-related exposure context could also contribute to the subject exposure field.
passageNumber / passageMethod	(no exact field)	—	Not in ERC000033, but commonly included as custom sample attributes in ENA viral submissions. Could be passed as free-text attributes.

Fields that could improve existing mappings

Current mapping	Improvement
PLATFORM (assembly manifest) ← sequencingInstrument	sequencingProtocol is also available and the docs list it as mapping to PLATFORM, but the actual config uses sequencingInstrument. Consider whether both should contribute, or if the mapping/docs need reconciling.
PROGRAM (assembly manifest) ← consensusSequenceSoftwareName + version	sequencingAssayType could add context here.
host subject id ← [] (default "not provided")	This is mandatory in ERC000033. No obvious Pathoplexus field maps here, but worth noting as a gap.

Unmapped fields with NO ERC000033 equivalent

These PHA4GE-origin fields don't have a standard ERC000033 counterpart but could be passed as custom ENA sample attributes if desired:

• Sample collection: sampleType, presamplingActivity, collectionDevice, collectionMethod
• Specimen: foodProduct, foodProductProperties, specimenProcessing, specimenProcessingDetails, experimentalSpecimenRoleType
• Host: hostOriginCountry, hostAgeBin, hostTaxonId, hostVaccinationStatus
• Infection history: previousInfectionDisease, previousInfectionOrganism
• Sequencing: sequencingDate, ampliconPcrPrimerScheme, ampliconSize, purposeOfSequencing
• Contact: sequencedByContactEmail
• Bioinformatics: rawSequenceDataProcessingMethod, dehostingMethod, assemblyReferenceGenomeAccession, breadthOfCoverage
• QC: all qualityControl* fields
• Diagnostics: all diagnostic* fields
• Other: comment

Summary of priorities

The biggest wins for ENA data quality would likely be:

1. sequencedByContactName → collector name (filling a mandatory field that's currently defaulted)
2. geoLocSite → geographic location (region and locality) (enriching an existing recommended mapping)
3. exposureSetting → host_habitat (filling a recommended field)
4. hostDisease / hostRole → host_description (adding useful optional context)
5. isLabHost / cellLine → lab_host (adding useful optional context)

Also worth noting: the config has #ena_checklist: ERC000033 - do not use until all fields are mapped to ENA accepted options — enabling the checklist would enforce ENA validation on these fields, which would catch format mismatches but requires ensuring all mapped values conform to ENA's controlled vocabularies.