Releases: KosinskiLab/pyTME
v0.3.3
New Features
- On-the-fly score background correction for all commonly used scores
- Extended analysis support for pytme_runner to simplify analysis of large datasets
- Extended analyzer support for Jax backend to other aggregation operations, such as constrained matching.
Internal changes
- Common postprocessing routines for composable filters, i.e., application of phase shifts and cropping, have been delegated to time.filters.Compose and tme.filters.ComposableFilters
- Frequency grids for the creation of composable filters adhere to the convention used in
np.fft.rfftn, i.e., DC component at the origin, to avoid potentially ill-shifted filters. This changes propagates into tme.filters.ReconstructFromTilt, which now expects each element of the input stack to be compliant with this convention. - Rotation matrices are now treated as pull, as opposed to push, to become compliant with the convention used in other EM software.
- All backends now use homogeneous transformation matrices to define rigid transforms and batches of rigid transforms.
v0.3.1
Version 0.3.1 brings improvements to both functionality and code organization, along with several breaking changes.
New Features
- Constrained template matching using rejection sampling
- CTF and wedge masks can now be defined using XML/STAR and MDOC file formats
- Multiple background correction methods and entity distinction capabilities in postprocess.py
- Automated runner for large cluster jobs
- QOL changes to the GUI
API/CLI Changes
- tme.preprocessing has been renamed to tme.filters and internally reworked
- tme.analyzer.py is now located at tme/analyzer and features a new abstract analyzer framework that better separates different analyzer types
- Replaced Fourier padding with edge padding throughout the codebase
Command line tools now use hyphens (-) instead of underscores (_) for better consistency
Migration Notes
Version 0.3.1 continues to use the new coordinate system convention introduced in 0.3.0. Briefly, Euler angles are expressed as intrinsic ZYZ Euler angles and e_z standard orientation. This particularly affects CCP4/MRC files, which are now converted to XYZ internally. Due to the change in coordinate system, previous TM results can not be used with version 0.3.0 onwards
v0.3.b0
v0.2.5
Version 0.2.5 introduces analytical gradients for non-exhaustive scoring methods and adds support for complex template shapes in Fourier shift calculations.
Enhancements
- Cross-correlation-based scores in tme.matching_optimization were given a grad method to compute analytical gradients.
- tme.matching_data.fourier_padding supports computing shifts of complex-shaped templates.
Documentation
- Minor revision of code examples to reflect a change in test data packaging.
v0.2.4
Version 0.2.4 addresses a critical bug, continues the deprecation process of tme.preprocessing, and includes various improvements and fixes. Users are strongly advised to update to this new version.
For users working with 0.2.3, the issue can be avoided by using --pad_filter with match_template.py. Alternatively, using the box size recommended by postprocess.py together with --no_centering can be used.
Bug Fixes
- Fixed a critical bug in
match_template.pythat caused wider cross-correlation peaks when using filters and templates not padded to the recommended shape bypreprocess.py(4d0e01f, Thanks to Min Zhang and #15 for reporting). - Resolved a minor numerical inaccuracy when using weighted tilt masks (5d14e6b).
- Corrected a type error that occurred when using discrete bandpass filters on cupy backends (4d0e01f).
API/CLI Changes
- Continued the deprecation of
tme.preprocessormodule and moved the implementation of wedge masks totme.preprocessing.tilt_series.
Enhancements
- Restructured C extensions and tests to prevent potential circular imports when testing conda/pyenv installations (4d0e01f).
postprocess.pycan automatically determine the ideal box size for a given backend.
v0.2.3
Version 0.2.3 introduces quality-of-life improvements, fixes a range of outstanding issues, and more extensive documentation. Users are recommended to update to the new version.
Enhancements
- Relion star files can be created without extracting subtomograms (#10).
preprocess.pyhas more functionalities for template generation.
API/CLI Changes
- FFT padding was harmonized across backends, defaulting to
tme.backends.NumpyFFTWBackend. tme.preprocessing.WedgeReconstructedreturns a strictly binary mask if weighting is disabled.match_template.pycan no longer perform threshold-based cropping of input data.
Documentation
- Deprecated quickstart in favor of a more extensive and structured user guide.
Bug Fixes
match_template.pywould create CTFs that did not factor in the spatial sampling rate, yielding incorrect CTFs. This has since been fixed andtme.preprocessing.tilt_series.CTFhas been adapted accordingly to avoid such issues in the future (48eadf6).- For certain data shape and parameter combinations (odd/even, fourier padding / edge padding), template matching results would be shifted by a single-voxel. This has been fixed in a793815.
tme.Density.from_filewould incorrectly extract the origin argument from CCP4/MRC files with non-standard MAPC, MAPS, MAPR configuration. This has been fixed in 2a2438b.- Using
tme.analyzer.PeakCallerMaximumFilterwith the pytorch backend would ignore peaks on the boundaries for certain min-distance requirements. This has been fixed in 531ff77. - Fixed an issue where the pytorch backend would not be listed as available in c54ddef.
v0.2.2
Version 0.2.2 introduces improvements to core functionality, new backends, and code refactoring.
Enhancements
- Exposed computation backend in
match_template.py. - Extended
match_template.pyto support n-dimensional matching (previously API-only). - Added background correction feature to
postprocess.py. - Expanded GUI capabilities for manual matching refinement and target mask creation.
- Introduced new computation backends: MLX and Jax.
API Changes
- Deprecated
tme.helpers; functions moved totme.Structureandtme.Preprocessor. - Replaced redundant functions in
tme.preprocessorwith thetme.preprocessingmodule. - Updated shape expressions in
tme.matching_exhaustiveto use fundamental types instead of backend-specific arrays. - Relocated scoring functions from
tme.matching_exhaustivetotme.matching_scores. - Refactored backends using metafunctions to reduce redundancy.
Documentation
- Updated
README.mdand made minor changes to overall documentation.
v0.2.1
Version 0.2.1 introduces a range of new filters, improves interoperability, and addresses minor issues.
Enhancements
- match_template.py can perform 3D CTF correction.
- New reconstruction filters reducing artifact introdution.
- postprocess.py can oversample and locally refine peaks to subvoxel / sub angular sampling precision.
- Improved interoperability between dynamo, relion and pytme via tme.Orientations.
API Changes
- tme.Density.centered now pads to odd box shapes, which improves the stability of peak estimates.
- tme.MatchingData handles additional init parameters to avoid potentially unexpected interactions with setters.
- tme.backends data type naming scheme was changed to more accurately reflect the underlying fundamental data type.
Documentation
- Added code examples to Density / MatchingData / Exhaustive Matching and Orientations
Bug Fixes
- Fixed an issue where match_template.py would crash when specifying target_masks. This was caused by a specific interaction of the OS with the underlying memmap (Thanks to Xiaohan for reporting).
- Fixed an issue where postprocess.py would crash when specifying a minimum score cutoff. To avoid this moving forward, match_template.py and postprocess.py are now tested together with the API.
- Fixed an issue where tme.preprocessing._utils.fftfreqn would create grids of incorrect dimensions.
- skimage.io.imread raises a ValueError or OSError depending on the underlying OS when encountering an incorrect input. This would prevent users from running postprocess.py on atomic structures.
- Fixed an issue where bandpass filter creation would fail when only specifying lowpass or highpass cutoff.
- Fixed a performance issue when peak calling on GPU
v0.2.0
Version 0.2.0 provides a new interface for dealing with orientations in 3D space, quality of life improvements to the GUI and fixes a critical performance issue introduced in a previous version.
Enhancements
- New Orientations class provides a convenient interface for converting between different file formats, namely tsv, relion star and dynamo tbl.
- Severe runtime decrease (2-3 times) for scoring methods affected by an issue in cupy.std, including scores computed using tme.matching_exhaustive.corr_scoring and tme.matching_exhaustive.flc_scoring.
- Revised interface for various command line options. New sampling and filtering functionalities in match_template.py
- The preprocessing GUI can now automatically import pickle files created by match_template.py, read all formats supported by tme.density.Density.from_file and provides additional functionalities for alignment.
API Changes
- Moved Orientations class from postprocess.py to tme.orientations.Orientations.
- tme.matching_optimization was refactored to allow individual scoring objects to be setup more easily and to facilitate the usage of different optimizers.
- tme.preprocessing.Compose introduces composable filters that were made accessible via match_template.py.
Documentation
- init methods of classes defined in tme.matching_optimization are correctly displayed.
- Inheritance structure is made more obvious.
- Attributes are now correctly documented for dataclasses.