Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 0 additions & 43 deletions .github/workflows/sphinx_docs.yml

This file was deleted.

116 changes: 102 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,27 +2,115 @@

**Gen3 Validator** is a Python toolkit designed to make working with Gen3 metadata schemas and data validation straightforward for developers.

With this tool, you can:

- **Resolve and flatten Gen3 JSON schemas** so you can work with them programmatically.
- **Validate JSON metadata files** against Gen3 schemas, catching schema violations early in your pipeline.
- **Check linkage integrity** between data nodes (e.g., ensuring all sample-to-subject references are valid).
- **Parse Excel-based metadata templates** and convert them to JSON for Gen3 ingestion.
- **Get detailed validation results and summary stats** as Python data structures or pandas DataFrames, making it easy to integrate with your own scripts or reporting tools.

## Installation
```bash
pip install gen3_validator
pip show gen3_validator
```

## Docs
- [src docs](https://australianbiocommons.github.io/gen3_validator/)
- [example usage](https://github.com/AustralianBioCommons/gen3_validator/blob/main/docs/usage.md)

*Note: *I recommend you clone this repo, and walk through the examples in the [usage](https://github.com/AustralianBioCommons/gen3_validator/blob/main/docs/usage.md) page. The usage examples load data from the `tests/data` [directory](https://github.com/AustralianBioCommons/gen3_validator/tree/main/tests/data) so you can see how the data is structured.*

## Quickstart
```python
import gen3_validator

## Installation
```bash
pip install gen3_validator
pip show gen3_validator
resolver = gen3_validator.ResolveSchema(schema_path = "../tests/schema/gen3_test_schema.json")
resolver.resolve_schema()
schema = resolver.schema_resolved

data = [
{
"baseline_timepoint": True, # variable not in data dictionary
"freeze_thaw_cycles": "10", # should be an integer
"sample_collection_method": "2fddbe7d09",
"sample_id": "d4f31f7bb6",
"sample_in_preservation": "snap Frozen",
"sample_in_storage": "yes",
"sample_provider": "USYD",
"sample_source": "UBERON:3781554",
"sample_storage_method": "not stored",
"sample_type": "59a8fd8005",
"storage_location": "UMELB",
"subjects": {
"submitter_id": "subject_e5616257f8"
},
"submitter_id": "sample_efdbe56d20",
"type": "sample"
},
{
"baseline_timepoint": True,
"freeze_thaw_cycles": 76,
"sample_collection_method": "e2a6403b51",
"sample_id": 3324635, # should be a string
"sample_in_preservation": "not allowed to collect",
"sample_in_storage": "unknown",
"sample_provider": "USYD",
"sample_source": "UBERON:9332357",
"sample_storage_method": "frozen, liquid nitrogen",
"sample_type": "8fd28ec2f3",
"storage_location": "Baker",
"subjects": {
"submitter_id": "subject_071bc3e81a"
},
"submitter_id": "sample_f7645c1221",
"type": "sample"
}
]
results = gen3_validator.validate.validate_list_dict(data, schema)

print(results)
```

Example output:

```python
[
{
'node': 'sample',
'index': 0,
'validation_result': 'FAIL',
'invalid_key': 'freeze_thaw_cycles',
'schema_path': 'properties.freeze_thaw_cycles.type',
'validator': 'type',
'validator_value': 'integer',
'validation_error': "'10' is not of type 'integer'"
},
{
'node': 'sample',
'index': 0,
'validation_result': 'FAIL',
'invalid_key': 'root',
'schema_path': 'additionalProperties',
'validator': 'additionalProperties',
'validator_value': False,
'validation_error': "Additional properties are not allowed ('baseline_timepoint', 'subjects' were unexpected)"
},
{
'node': 'sample',
'index': 1,
'validation_result': 'FAIL',
'invalid_key': 'sample_id',
'schema_path': 'properties.sample_id.type',
'validator': 'type',
'validator_value': 'string',
'validation_error': "3324635 is not of type 'string'"
},
{
'node': 'sample',
'index': 1,
'validation_result': 'FAIL',
'invalid_key': 'root',
'schema_path': 'additionalProperties',
'validator': 'additionalProperties',
'validator_value': False,
'validation_error': "Additional properties are not allowed ('baseline_timepoint', 'subjects' were unexpected)"
}
]

```

---

## Dev Setup
Expand Down
20 changes: 0 additions & 20 deletions docs/Makefile

This file was deleted.

35 changes: 0 additions & 35 deletions docs/make.bat

This file was deleted.

34 changes: 0 additions & 34 deletions docs/source/conf.py

This file was deleted.

29 changes: 0 additions & 29 deletions docs/source/gen3_validator.parsers.rst

This file was deleted.

53 changes: 0 additions & 53 deletions docs/source/gen3_validator.rst

This file was deleted.

Loading