v1.0.0

Vizgen Data Transfer v1.0.0

Features

Vizgen Ultra Support

• Full compatibility with Vizgen Ultra data structures.
• Updated default drive mappings (transitioned from F: to L: for HPC/Isilon integration).

Robust Data Integrity & Validation

• Multi-layered Checks: Implemented comprehensive pre- and post-transfer validation using both Python os.walk and Robocopy list logs.
• Size Verification: Added GB-scale comparison to ensure data parity across high-volume transfers.
• Automated Reporting: Enhanced email reporting with detailed file counts, folder counts, and total execution time tracking.
• Error Handling: Improved log clarity for easier troubleshooting and interpretation of transfer discrepancies.

Windows Optimisation

Long Path Support: Fixed the Windows MAX_PATH 260-character limit issue, ensuring deep directory structures transfer without failure.
Ease of Use: Added a Windows PROFILE and start_eivdt shortcut for quick environment sourcing and execution.

Internal Improvements & Refactoring

• Performance: Defaulted to single-thread execution for maximum stability during high-load transfers.
• Configuration: Migrated to a centralised .vizgen_config.toml system, allowing for easy updates to Robocopy headers, footers, and drive paths.
• Logging: Centralised all transfer logs into a dedicated folder for historical auditing.
• Modern Tooling: * Migrated project management to uv for faster dependency resolution.
  • Implemented bump-my-version for standardised SemVer management.
  • Added support for Python 3.9+ (including tomli for backward compatibility).

Changelog Summary

Added

• Tracking and reporting of total execution time.
• Optional flags: --ignore_python_counts and --ignore_robocopy_counts.
• Robocopy list checks on raw_data, analysis, and output.
• Centralised log storage.
• Support for local .vizgen_config.toml overrides in the working directory.

Changed

• Refactored core logic for better maintainability.
• Updated documentation and README with detailed setup instructions.
• Switched default thread count to 1 for reliability.

Fixed

• Windows 260-character path limit.
• Email reporting formatting for data counts.

Full Changelog: v0.1.0...v1.0.0

v0.1.0

Vizgen Data Transfer v0.1.0 [Initial Release]

Features

• Initial implementation of the Vizgen data transfer tool
• Supports transferring raw_data, analysis, and output types
• Configurable via .vizgen_config.toml
• Includes multi-threaded copying and external disk support
• CLI with detailed help and usage instructions

Full Changelog: https://github.com/EI-CoreBioinformatics/vizgen_data_transfer/commits/v0.1.0