Top-Level API and Project Structure of the EasyScience-Based Python Libraries for Data Analysis, on the example of EasyDiffraction

[AndrewSazonov]

Proposed modifications of the names

I propose some minor modifications to the existing names for the data processing workflow steps, library classes, and GUI page titles:

Step	Library Class Name	GUI Page Name
Project	Project & Info	Project
Sample Model	SampleModels (collection) & SampleModel (individual)	Sample Model
Measurement	Measurements (collection) & Measurement (individual)	Measurement
Analysis	Analysis (calculation, fitting/optimization)	Analysis
Summary	Summary (results & reporting)	Summary

These names seem to be general-purpose and domain-agnostic, making them applicable across various scattering techniques while remaining intuitive for users.

Project vs. Job or Study

Job

• ❌ Implies a single-run process, whereas users refine models iteratively.
• ❌ Typically refers to batch processing, where a task is submitted and executed without user interaction.
• ❌ Not ideal for interactive workflows in Jupyter or GUI applications, where users need to modify parameters and recompute results multiple times.
• ❌ Suggests a one-time computation rather than a persistent working environment.

Study

• ❌ Too broad, suggesting a high-level scientific investigation rather than a structured computational workflow.
• ❌ Not specific enough to indicate that data, models, and computations are all part of a unified, structured workflow.
• ❌ May imply a collection of multiple projects rather than a single analysis process.

Project

• ✅ Clearly represents a structured scientific data analysis workflow, where all data, models, and computations are stored together.
• ✅ Avoids confusion with jobs (single-run batch processes) and studies (broad investigations).
• ✅ Works well in both script-based workflows and GUI-based applications.
• ✅ Conveys a persistent working environment, supporting iterative model refinement and result saving.

Sample Model vs. Model, Sample, or Theory

Model

• ❌ Too generic – could refer to instrument models, mathematical models, or fitting models.
• ❌ Does not explicitly indicate that it represents the physical sample being studied.

Sample

• ❌ Too broad – could include sample preparation, composition, or physical properties, which extend beyond structural modeling.
• ❌ Could be confused with real-world sample metadata (e.g., mass, composition), rather than referring to the computational model.

Theory

• ❌ Misleading – suggests a general theoretical approach rather than a specific computational model for a given sample.

Sample Model

• ✅ Explicitly refers to the model of the studied sample.
• ✅ Avoids confusion with instrument models or purely theoretical models.
• ✅ Well-defined across diffraction, reflectometry, QENS, and other scattering techniques.
• ✅ Can be extended in the future to include real-world sample metadata (e.g., mass, composition), if necessary.

Measurement vs. Experiment

Experiment

• ❌ Implies a physical experiment, while the focus here is on data analysis.
• ❌ Can be confused with experimental design, which is unrelated to data processing.
• ❌ Suggests a broader scientific process, rather than just the acquisition and storage of measured data.

Measurement

• ✅ Represents collected data, whether from a real experiment or a simulation.
• ✅ More precise, as it refers to processed, structured data, rather than the act of conducting an experiment.
• ✅ Better aligns with the concept of instrumental setup and measured data storage.
• ✅ Used in many scientific disciplines to refer to raw or processed data, rather than the experimental process itself.

API example

import easydiffraction as ed

# === Step 1: Project ===

# Create a new diffraction analysis project
proj = ed.Project()

# Set project metadata (title and description)
proj.info.title = "My Diffraction Analysis"
proj.info.description = """This project analyzes diffraction data using joint 
refinement of CW and TOF neutron diffraction measurements."""

# Save the project before continuing
proj.save_as("path/to/new_project")

# === Step 2: Sample Model ===

# Load a sample model from a CIF file
# The sample model ID ("sample1") is automatically determined from the CIF file
proj.sample_models.append(cif_path="path/to/sample1.cif")

# Show available sample models in the project
proj.sample_models.show_ids()

# Display sample model parameters (both descriptors and refinable parameters)
proj.sample_models["sample1"].show_params()

# Modify structural model parameters (e.g., space group and lattice parameters)
proj.sample_models["sample1"].space_group.name = "Pnma"  
proj.sample_models["sample1"].cell.length_a = 5.0

# Visualize the sample structure (3D crystal/magnetic structure)
proj.sample_models["sample1"].show_structure()

# === Step 3: Measurement ===

# Load experimental neutron diffraction measurements

# Option 1: Load from a CIF file
# Measurement ID ("meas1") and type ("pd-neut-cwl") are automatically extracted
proj.measurements.append(cif_path="path/to/meas1.cif")

# Option 2: Load from raw data (explicitly define ID and type)
proj.measurements.append("meas2", type="pd-neut-tof", file_path="path/to/meas2.dat")

# Show available measurements in the project
proj.measurements.show_ids()

# Display all measurement parameters (both descriptors and refinable parameters)
proj.measurements.show_params()

# Modify instrument setup parameters
proj.measurements["meas1"].instr_setup.wavelength = 1.54  # CW neutron wavelength (Å)
proj.measurements["meas2"].instr_setup.twotheta_bank = 120.0  # TOF detector bank angle (°)

# Show measured data for each experiment (intensity vs. angle or TOF)
proj.measurements["meas1"].show_data_chart()
proj.measurements["meas2"].show_data_chart()

# === Step 4: Analysis ===

# Display all refinable parameters (parameters that can be either fixed or varied during fitting)
proj.analysis.show_refinable_params()

# Modify instrument calibration parameters
proj.measurements["meas1"].instr_calib.twotheta_offset = 0.02  # CW offset (°)
proj.measurements["meas2"].instr_calib.d_to_tof_offset = 0.01  # TOF offset (µs)

# Adjust peak broadening parameters (instrumental resolution effects)
proj.measurements["meas1"].peak_broad.gauss_u = 0.005  # CW Gaussian broadening
proj.measurements["meas2"].peak_broad.gauss_sigma_0 = 0.003  # TOF Gaussian broadening

# Show comparison between measured and calculated patterns before fitting
proj.analysis.show_meas_vs_calc_chart("meas1")
proj.analysis.show_meas_vs_calc_chart("meas2")

# Select parameters for fitting (allow them to be varied during optimization)
proj.sample_models["sample1"].atom_sites["atom1"].fract_x.vary = True
proj.measurements["meas1"].peak_broad.gauss_u.vary = True
proj.measurements["meas2"].peak_broad.gauss_sigma_0.vary = True

# Show only parameters currently selected for fitting
proj.analysis.show_free_params()

# Run the fitting procedure to optimize selected parameters
proj.analysis.fit()

# Show fitting results (optimized parameters and fit quality metrics)
proj.analysis.show_fit_results()

# Show the comparison of measured and fitted patterns
proj.analysis.show_meas_vs_calc_chart("meas1")
proj.analysis.show_meas_vs_calc_chart("meas2")

# === Step 5: Summary ===

# Generate a final summary report with refined parameters and fitting results
proj.summary.show_report()

# Save the final state of the project
proj.save()

Abbreviations

Experiment type

Parameter	Description	Abbreviations
Diffraction Mode	Specifies whether the diffraction data corresponds to powder diffraction (polycrystalline sample) or single crystal diffraction.	pd, sc
Diffraction Radiation Probe	Specifies whether the measurement uses neutrons or X-rays as the radiation source.	neut, xray
Experiment Mode	Defines whether the measurement is performed using a constant wavelength or a time-of-flight method.	cwl, tof
Polarization Mode	Indicates whether the neutron beam is unpolarized or polarized (Classical, Longitudinal Polarization Analysis, or Spherical Neutron Polarimetry).	unp, pol, lpa, snp

Examples

Description	String-like representation	Class name suffix
Powder neutron diffraction with constant wavelength	pd-neut-cwl	_PdNeutCwl
Powder neutron diffraction with time-of-flight	pd-neut-tof	_PdNeutTof
Powder X-ray diffraction	pd-xray	_PdXray
Single crystal neutron diffraction with constant wavelength	sc-neut-cwl	_ScNeutCwl
Single crystal neutron diffraction with time-of-flight	sc-neut-tof	_ScNeutTof

Class name suffix usage in EasyDiffraction

Class name suffixes are used to distinguish between different types of measurements and logically group them within the library. While this naming convention may not strictly follow typical Pythonic style, it enhances clarity and consistency.

The suffixes are intentionally applied to provide clarity in complex hierarchical structures, where various types of measurements and calibrations must be easily distinguished—especially during introspection, auto-completion, and debugging.

For example:

• InstrumentCalibrationPdNeutTof vs. InstrumentCalibration_PdNeutTof
• InstrumentCalibrationPdNeutTofUnp1d vs. InstrumentCalibration_PdNeutTofUnp1d

The use of suffixes clearly separates the measurement type while keeping the rest of the class name structured and understandable.

These suffixes are solely used to indicate the measurement type, ensuring logical grouping without compromising readability.

Top-level package structure

📦 easydiffraction
├── 📄 project.py
│   ├── 🏷️ class Project                                # Main entry point
│   └── 🏷️ class ProjectInfo                            # Metadata: title, description, path
│
├── 📁 sample_models
│   ├── 📄 models.py                                    # Handles sample model collections and individual models
│   │   ├── 🏷️ class SampleModel                        # Represents a single sample model
│   │   └── 🏷️ class SampleModels                       # Collection of sample models
│   ├── 📄 space_group.py
│   ├── 📄 cell.py
│   └── 📄 atom_sites.py
│
├── 📁 measurements
│   ├── 📄 measurements.py                              # Handles different types of measurements
│   │   ├── 🏷️ class BaseMeasurement                    # Abstract base class for measurements
│   │   ├── 🏷️ class Measurement_Sc                     # Base class for Single-crystal diffraction measurement
│   │   ├── 🏷️ class Measurement_Pd                     # Base class for powder diffraction measurement
│   │   ├── 🏷️ class Measurement_PdCwl                  # Base class for CW neutron & X-ray powder diffraction
│   │   ├── 🏷️ class Measurement_PdTof                  # Base class for TOF powder diffraction
│   │   ├── 🏷️ class Measurement_PdNeutCwl              # CW neutron powder diffraction measurement
│   │   ├── 🏷️ class Measurement_PdNeutTof              # TOF neutron powder diffraction measurement
│   │   ├── 🏷️ class Measurement_PdXrayCwl              # X-ray powder diffraction measurement
│   │   └── 🏷️ class Measurements                       # Collection of measurement objects
│   ├── 📄 instrument_setup.py                          # Instrument setup parameters
│   │   ├── 🏷️ class InstrumentSetup_PdNeutCwl
│   │   └── 🏷️ class InstrumentSetup_PdXrayCwl
│   ├── 📄 instrument_calibration.py                    # Instrument calibration parameters
│   │   ├── 🏷️ class InstrumentCalibration_PdCwl
│   │   └── 🏷️ class InstrumentCalibration_PdNeutTof
│   ├── 📄 peak_profile.py                              # Peak profile functions
│   │   ├── 🏷️ class PeakProfile_PdCwl
│   │   └── 🏷️ class PeakProfile_PdTof
│   ├── 📄 peak_broadening.py                           # Peak broadening effects
│   │   ├── 🏷️ class PeakBroadening_PdCwl
│   │   └── 🏷️ class PeakBroadening_PdTof
│   ├── 📄 peak_asymmetry.py                            # Peak asymmetry effects
│   │   ├── 🏷️ class PeakAsymmetry_PdCwl
│   │   └── 🏷️ class PeakAsymmetry_PdTof
│   ├── 📄 background.py                                # Background subtraction models
│   │   └── 🏷️ class Background_Pd
│   └── 📄 datastore.py                                 # Stores measured & calculated data
│       ├── 🏷️ class MeasuredData_Sc
│       ├── 🏷️ class MeasuredData_Pd
│       ├── 🏷️ class CalculatedData_Sc
│       └── 🏷️ class CalculatedData_Pd
│
├── 📁 analysis
│   ├── 📄 analysis.py                                  # Main analysis API, integrates calculators & fitters
│   │   └── 🏷️ class Analysis
│   ├── 📄 calculation.py                               # Handles diffraction pattern calculations
│   │   └── 🏷️ class DiffractionCalculator
│   ├── 📁 calculators                                  # Calculation engine wrappers
│   │   ├── 📄 factory.py                               # Manages available calculators
│   │   │   └── 🏷️ class CalculatorFactory
│   │   ├── 📄 base.py                                  # Abstract base class for all calculators
│   │   │   └── 🏷️ class CalculatorBase                 # ABC for diffraction calculators
│   │   ├── 📄 crysfml.py                               # Wrapper for CrystFML backend
│   │   │   └── 🏷️ class CrysfmlCalculator
│   │   ├── 📄 cryspy.py                                # Wrapper for CrySPY backend
│   │   │   └── 🏷️ class CryspyCalculator
│   │   └── 📄 pdffit.py                                # Wrapper for PDFfit backend
│   │       └── 🏷️ class PdffitCalculator
│   └── 📄 minimization.py                              # Handles optimization algorithms for parameter fitting
│       └── 🏷️ class DiffractionMinimizer
│
├── 📄 summary.py                                       # Handles reporting & export of results
│   └── 🏷️ class Summary
│
└── 📁 utils                                            # Helper functions
    └── 📄 utils.py                                     # General utility functions
        ├── 🔧 def download_from_repository()
        └── 🔧 def is_notebook()

Overview diagram

This is a top-level diagram that illustrates the concept of the Project as a container for all other objects involved in the data analysis process. This approach enables a simple and intuitive API for data analysis, as demonstrated in the example script above.

The Project object maintains all relationships between its components, ensuring they are consistently connected, while allowing each object to remain independently usable when required.

classDiagram

    namespace easycrystallography {
        class StructuralModel:::model {
            id: str
            space_group: SpaceGroup
            cell: Cell
            atom_sites: AtomSites
            from_cif_file(cif_path: str)
            from_cif_str(cif: str)
            as_cif() str
        }
    }
    
    class Project:::container {
        info: ProjectInfo
        sample_models: SampleModels
        measurements: Measurements
        analysis: Analysis
        summary: Summary
    
        %% Use the directory rather than a file
        %% This directory should have at least project.cif (or info.cif? or main.cif?)
        load(dir_path: str)
        save_as(dir_path: str)
        save()
        reset()
        as_cif() str
    }

    class ProjectInfo:::container {
        title: str
        description: str
        path: str
        as_cif() str
        _created: datetime
        _last_modified: datetime
    }

    class SampleModels:::collection {
        self: list[SampleModel]
    
        append(model_id: str)
        append(cif_path: str)
        append(cif: str)
        append(model: SampleModel)
        remove(model_id: str)
    
        show_ids()
        show_params()
        as_cif() str

        _get_params() dict
    }

    class SampleModel:::model {
        show_structure()
    }

    class Measurements:::collection {
        self: list[measurement_cls]
    
        append(id: str, type: str)
        append(id: str, type: str, file_path: str)
        append(cif_path: str)
        append(cif: str)
        append(meas: measurement_cls)
        remove(meas_id: str)

        show_ids()
        show_params()
        as_cif() str

        _get_params() dict
    }

    class Measurement_PdNeutTof:::measurement {
        instr_calib: InstrumentCalibration_PdNeutTof
    }

    class Measurement_PdNeutCwl:::measurement {
        instr_setup: InstrumentSetup_PdNeutCwl
    }
  
    class Measurement_PdXrayCwl:::measurement {
        instr_setup: InstrumentSetup_PdXrayCwl
    }

    class Measurement_PdTof:::baseclass {
        <<abstract>>
        instr_setup: InstrumentSetup_PdTof
        peak_profile: PeakProfile_PdTof
        peak_broad: PeakBroadening_PdTof
        peak_asym: PeakAsymmetry_PdTof
    }

    class Measurement_PdCwl:::baseclass {
        <<abstract>>
        instr_calib: InstrumentCalibration_PdCwl
        peak_profile: PeakProfile_PdCwl
        peak_broad: PeakBroadening_PdCwl
        peak_asym: PeakAsymmetry_PdCwl
    }

    class Measurement_Pd:::baseclass {
        <<abstract>>
        linked_samples: list[LinkedSample]
        background: Background_Pd
        meas_data: MeasuredData_Pd
        calc_data: CalculatedData_Pd
        show_data_chart()
    }
  
    class Measurement_Sc:::baseclass {
        <<abstract>>
        linked_sample: LinkedSample
        meas_data: MeasuredData_Sc
        calc_data: CalculatedData_Sc
        show_data_chart()
    }
  
    class BaseMeasurement:::baseclass {
        <<abstract>>
        id: str
        expt_type: str
        as_cif() str
    }

    class Summary:::container {
        +as_cif() str
        +as_html() str
        +show_report()
    }

    %% Relationships %%

    Project *-- ProjectInfo
    Project *-- SampleModels
    Project *-- Measurements
    Project *-- Analysis
    Project *-- Summary
  
    SampleModels "1" -- "*" SampleModel : contains
    SampleModel <|-- StructuralModel
  
    Measurements "1" -- "*" Measurement_PdNeutTof : contains
    Measurements "1" -- "*" Measurement_PdNeutCwl : contains
    Measurements "1" -- "*" Measurement_PdXrayCwl : contains

    Measurement_PdNeutTof <|-- Measurement_PdTof
    Measurement_PdNeutCwl <|-- Measurement_PdCwl
    Measurement_PdXrayCwl <|-- Measurement_PdCwl

    Measurement_PdTof <|-- Measurement_Pd
    Measurement_PdCwl <|-- Measurement_Pd
    Measurement_ScTof <|-- Measurement_Sc
    Measurement_ScCwl <|-- Measurement_Sc
 
    Measurement_Pd <|-- BaseMeasurement
    Measurement_Sc <|-- BaseMeasurement

    %%%%%%%%%%%%%
    %% STYLING %%
    %%%%%%%%%%%%%

    %% Abstract Base Classes (BaseMeasurement, Measurement_Pd, etc.)
    classDef baseclass fill:#6A5ACD,stroke:#333,stroke-width:1px,color:white;

    %% Containers (Project, ProjectInfo, Summary, Analysis)
    classDef container fill:#455A64,stroke:#333,stroke-width:1px,color:white;

    %% Collections (SampleModels, Measurements)
    classDef collection fill:#607D8B,stroke:#333,stroke-width:1px,color:white;

    %% Concrete Measurements (PdNeutCwl, PdNeutTof, PdXrayCwl)
    classDef measurement fill:#4682B4,stroke:#0D47A1,stroke-width:1px,color:white;

    %% Models (SampleModel, StructuralModel)
    classDef model fill:#009688,stroke:#004D40,stroke-width:1px,color:white;

Loading

Analysis step diagram

In this diagram, both Minimizers and Calculators are assumed to use a factory-based approach to create their respective objects.

The minimization module is provided by EasyScience and imported into EasyDiffraction as-is. Potential simplifications or customizations can be considered in the future for tighter integration or improved usability.

The structure of the calculation component can follow a similar pattern to minimization in EasyScience and be reused across different techniques. The actual implementation, however, will be technique-specific.

For the calculation component, the abstract base class (CalculatorBase) defines the interface for calculator wrappers. Concrete implementations, such as CryspyCalculator, wrap third-party calculation engine APIs. The term Wrapper is intentionally avoided in class names to prevent confusion when using the API, e.g., proj.analysis.calculator = 'crysfml' instead of proj.analysis.wrapper = 'crysfml'.

classDiagram

    %%%%%%%%%%%%%%%%%%%%%%%%%
    %% easyscience package %%
    %%%%%%%%%%%%%%%%%%%%%%%%%

    namespace easyscience {

        class MultiFitter:::fitter {
        }

        %% Rename to SingleFitter?
        class Fitter:::fitter {
            ...
            %% Rename to _fitter: fitter_cls?
            _minimizer: minimizer_cls
            ...
        }

        %% Rename to AvailableFitters?
        %% Move to FitterFactory class instead?
        class AvailableMinimizers {
            ...
        }

        %% Rename to AvailableFitter?
        %% Move to FitterFactory class instead?
        class AvailableMinimizer {
        <<dataclass>>
        }

        %% Move to FitterFactory class instead?
        class factory:::factory {
            factory(...) minimizer_cls
        }

        %% Rename to LmfitFitter?
        class LMFit:::wrapper {
        }

        %% Rename to BumpsFitter?
        class Bumps:::wrapper {
        }

        %% Rename to DfolsFitter?
        class DFO:::wrapper {
        }

        %% Rename to FitterBase?
        class MinimizerBase:::baseclass {
            <<abstract>>
            /property/ all_constraints
            /property/ enum
            /property/ name
            _get_method_kwargs(...)
            _prepare_parameters(...)
            _fit_function(...)
            _create_signature(...)$
            _error_from_jacobian(...)$
            fit_constraints()
            set_fit_constraint(...)
            add_fit_constraint(...)
            remove_fit_constraint(...)
            evaluate(...)
            convert_to_pars_obj(...)*
            fit(...)*
            supported_methods()$
            all_methods()$
            convert_to_par_object(...)$
        }
    }

    %%%%%%%%%%%%%%%%%%%%%%%%%%%%%
    %% easydiffraction package %%
    %%%%%%%%%%%%%%%%%%%%%%%%%%%%%

    namespace easydiffraction {

        class Project:::container {
        }

        class Analysis:::container {
            /property/ calculator: DiffractionCalculator
            /property/ fitter: DiffractionFitter

            %% "single" (default), "combined", or "sequential"
            /property/ refinement_mode: str

            available_calculators() list[str]
            available_minimizers() list[str]

            calculate_pattern(meas_id: str) ndarray
            fit(...)

            %% Shows all Float parameters that can be fitted (fixed & free)
            show_refinable_params()
            %% Shows only those parameters that are free to be fitted
            show_free_params()
            %% Shows only those parameters that are fixed and are not allowed to be fitted
            show_fixed_params()

            show_calc_chart(meas_id: str)
            show_meas_vs_calc_chart(meas_id: str)
            show_fit_results()

            as_cif() str
        }

        %% Fitting

        class DiffractionFitter:::fitter {
            fitter: MultiFitter
        }

        %%class Minimizer {
        %%    -resid_func(fit_params: FitParameters, meas_data, calculator) ndarray
        %%    -callback_func(fit_params: FitParameters, iter: int, resid_func: ndarray)
        %%    +fit(resid_func, callback_func, fit_params: FitParameters, options: dict)
        %%}

        %% Calculation

        class DiffractionCalculator:::calculator {
            _calculator: calculator_cls
            set_calculator(type: str)
            calculate_hkl(sample_models: SampleModels, experiments: Experiments) ndarray
            calculate_pattern(sample_models: SampleModels, experiments: Experiments) ndarray
        }

        class CalculatorFactory:::factory {
            create_calculator(type: str) calculator_cls
            register_calculator(type: str, calculator_cls)
            available_calculators() list
        }

        class CrysfmlCalculator:::wrapper {
        }

        class CryspyCalculator:::wrapper {
        }

        class PdffitCalculator:::wrapper {
        }

        class CalculatorBase:::baseclass {
            <<abstract>>
            /property/ name: str
            calculate_hkl(...)* ndarray
            calculate_pattern(...)* ndarray
        }
    }

    %%%%%%%%%%%%%%%
    %% RELATIONS %%
    %%%%%%%%%%%%%%%

    %% Major %%

    Project *-- Analysis : contains

    Analysis *-- DiffractionFitter : contains
    Analysis *-- DiffractionCalculator : contains

    %% Refinement/Fitting/Minimization/Optimization %%

    DiffractionFitter *-- MultiFitter : contains

    MultiFitter --|> Fitter : inherit

    Fitter ..> factory : uses

    Fitter *-- LMFit : contains
    Fitter *-- Bumps : contains
    Fitter *-- DFO : contains

    factory ..> LMFit : creates
    factory ..> Bumps : creates
    factory ..> DFO : creates

    LMFit --|> MinimizerBase : inherit
    Bumps --|> MinimizerBase : inherit
    DFO --|> MinimizerBase : inherit

    Fitter .. AvailableMinimizers : uses
    factory .. AvailableMinimizers : uses
    LMFit .. AvailableMinimizers : uses
    Bumps .. AvailableMinimizers : uses
    DFO .. AvailableMinimizers : uses
    MinimizerBase .. AvailableMinimizers : uses

    AvailableMinimizers --|> AvailableMinimizer : inherit

    %% Calculations %%

    DiffractionCalculator ..> CalculatorFactory : uses

    DiffractionCalculator *-- CrysfmlCalculator : contains
    DiffractionCalculator *-- CryspyCalculator : contains
    DiffractionCalculator *-- PdffitCalculator : contains

    CalculatorFactory ..> CrysfmlCalculator : creates
    CalculatorFactory ..> CryspyCalculator : creates
    CalculatorFactory ..> PdffitCalculator : creates

    CrysfmlCalculator --|> CalculatorBase : inherit
    CryspyCalculator --|> CalculatorBase : inherit
    PdffitCalculator --|> CalculatorBase : inherit

    %%%%%%%%%%%%%
    %% STYLING %%
    %%%%%%%%%%%%%

    %% Core Containers (Project, Analysis)
    classDef container fill:#546E7A,stroke:#263238,stroke-width:1px,color:#FFFFFF;

    %% Diffraction Fitter
    classDef fitter fill:#FBC02D,stroke:#F57F17,stroke-width:1px,color:#000000;

    %% Diffraction Calculator (Core Calculation Logic)
    classDef calculator fill:#FBC02D,stroke:#F57F17,stroke-width:1px,color:#000000;

    %% Factories (Factory classes in Minimization/Calculation)
    classDef factory fill:#2E7D32,stroke:#1B5E20,stroke-width:1px,color:#FFFFFF;

    %% Wrappers (Crysfml, Cryspy, Pdffit Calculators)
    classDef wrapper fill:#FB8C00,stroke:#E65100,stroke-width:1px,color:#FFFFFF;

    %% Abstract Base Classes (MinimizerBase, CalculatorBase)
    classDef baseclass fill:#3949AB,stroke:#1A237E,stroke-width:1px,color:#FFFFFF;

Loading

[AndrewSazonov]

Points of preliminary agreement

• Sample Model naming. We will use the name ‘Sample Model’ for the GUI page and SampleModel as the library class, instead of just Model or Sample.
• Experiment naming. We will continue using the term ‘Experiment’ instead of ‘Measurement’, as the object responsible for both instrument parameters and experimentally measured data, if available.
• Project as central object. We agreed to use the Project object (instead of ‘Job’) as the main access point responsible for connecting all other objects.
• Method naming: add instead of append. We will use the method name add (instead of append) in the library when adding sample models and experiments to their respective collections. For consistency, we will also update the GUI button labels from ‘load ...’ and ‘define ...’ to ‘add ...’.
• Flexible ways to add sample models and experiments. We will allow users to add sample models and experiments as objects of their respective classes (as already proposed in the discussion draft). While adding them via file paths (e.g., a CIF file) is not universally supported, it remains a desirable option for simple API usage - particularly for those without Python experience. Adding these objects via string-based description methods will complement this approach.
• Refactoring experiment class structure. We agreed to avoid using class abbreviations for different types of experiments. Instead, we will adopt an approach where each class is responsible for a specific component - e.g., Time-of-Flight, Single-Crystal, X-ray, etc. These components will then be combined into a single experiment class using multiple inheritance. Class names should be descriptive and meaningful, avoiding concatenations of short, cryptic parts.
• Explicit type definition for experiments. Following the above, when creating an experiment object, multiple attributes will be used to explicitly define its type, instead of a single, lengthy string attribute composed of concatenated shortcuts.

Open questions for further discussion

• Displaying the ‘Idealized’ calculated pattern. How important is it to provide the ability to display the idealized calculated pattern directly on the Sample Model page (as a separate tab)? This pattern would be generated either without instrument parameters (if possible), or using default parameters from an imaginary instrument. On the Experiment page, this pattern would take into account actual instrument parameters and be compared with measured data (if available). On the Analysis page, users would perform model refinement to fit calculated data to measured data.
• Defining Experiment Type for the ‘Idealized’ Calculated Pattern. If an idealized calculated pattern is to be generated on the Sample Model page, how do we define which type of experiment the pattern corresponds to? In diffraction, for example, there could be a dozen completely different charts based on different combinations of attributes defining the experiment type.
  Should we allow users to easily switch from one experiment type to another at this stage (e.g., from powder to single crystal)? If so, how do we address the fact that the experiment description block only appears on the next GUI page or becomes available through the experiment class, which has not yet been created in the library?
• Ownership of the Calculator object. Should the calculator object belong to SampleModel, SampleModels, Analysis, or Project?
  • If each SampleModel owns its own calculator and is responsible for calculating the idealized pattern for that individual model, who then is responsible for calculating the combined pattern for all sample models - i.e., the pattern we will actually compare with the measured data?
  • If SampleModels (or multiple SampleModel instances) collectively own the calculator to calculate a combined pattern.
    • How do we handle cases where, for example, three sample models - A, B, and C - are defined, and we generate a combined calculated pattern from all of them, but our experiment datasets do not include contributions from all three phases? For instance: The first dataset includes contributions from SampleModels A and B, the second dataset includes contributions from SampleModels B and C. So, how do we manage situations where A, B, and C are never contributing together, yet the current combined calculation includes all three by default?
    • Who is responsible for updating the calculated pattern after instrumental parameters are defined during the experiment definition step (which takes place after the sample model step)? Is it still the responsibility of SampleModels? If so, the SampleModels object now become depend on the Experiment object to take those parameters into account?
    • Would the same method, such as plot_data_chart(), still be used on the SampleModel to display the updated data chart - now reflecting instrument effects and other experiment parameters - rather than the idealized version shown previously
• We currently have an ADR: Fitting using third-party minimizers. However, several points were either not discussed in depth or remain unclear:
  • The ADR states: MinimizerFactory - Single class / function, but it doesn’t explain why the current implementation uses a function instead of a class. Additionally, there is no reasoning provided for why AvailableMinimizers is implemented as a separate class. As a result, the current setup - with a factory function in one file and the AvailableMinimizers class in another - feels unnecessarily complex. We should revisit this design and consider implementing a unified MinimizerFactory class that consolidates all related functionality, similar to the CalculatorFactory proposed in the diagram.
  • There is also inconsistency in the terminology used throughout the codebase and GUI. We have a mixture of fit- and minimize-based names, which can lead to confusion. We need to decide whether to establish a consistent naming convention for classes, attributes, and GUI elements related to minimization and fitting. Specifically, should we standardize on using fit-based terms - such as Fitter, fit, fitting engines, and available fitters - or adopt minimizer-based names throughout?

[rozyczko]

This shoudl really be split into separate ADRs/discussions:

• general Proposed modifications of the names in the EasyScience space
• calculators/mimimizers/fitters naming and structure - also in the EasyScience space
• things which are EDL specific (Abbreviations, class diagrams) should be in the ED space

This way, we can have more specific discussions on concrete topics rather than very long meetings touching everything.,

[AndrewSazonov]

I agree that splitting the discussion into smaller, more focused topics is necessary.

Regarding the diffraction-specific abbreviations, I fully agree they should be moved into the EasyDiffraction space. However, the class diagram I suggested above is intended as a high-level representation, focusing on the global relationships between core objects such as Project, SampleModel, Experiment, etc. The idea of discussing it here is to determine whether this structure can be generic enough to support other techniques beyond diffraction. Diffraction is used in the example primarily to make the diagram more concrete and realistic.

The same reasoning applies to the Analysis step diagram. It is meant to help us explore a generic approach for handling both Minimizers and Calculators, which could then be extended to other domains.

Of course, more detailed diagrams of individual classes and specific implementations should be discussed within their respective technique-specific spaces.

[AndrewSazonov]

One more question to discuss

I'll post it here for now until we split this discussion into smaller threads.

While thinking about different types of refinement workflows, I’ve identified several use cases relevant to diffraction. These can be grouped into three broad categories, depending on the refinement strategy: single, combined, and sequential refinements.

1. Single Refinement

• 1.1. One sample model, one measured dataset.
  This is the simplest case. Unfortunately, life is often more complicated, but it’s a good starting point.

• 1.2. Multiple sample models, one measured dataset.
  A slightly more complex case. For example, two crystallographic phases both contribute to the measured data. Both sample models need to be considered during refinement.

2. Combined Refinement

• 2.1. One sample model, multiple datasets.
  This scenario covers measurements of the same sample on different instruments or under different measurement conditions. For example:

  • High-intensity vs. high-resolution datasets
  • Focus on magnetic vs. crystal structures
    In this case, the same sample model parameters are refined simultaneously across multiple datasets, while instrument parameters are specific to each dataset.

• 2.2. Multiple sample models, multiple datasets.
  Each dataset may include contributions from different sample models. For example:

  • A neutron diffraction dataset might have the main contribution from the sample of interest, plus a minor contribution from a sample environment (e.g., a cryostat).
  • A synchrotron X-ray dataset of the same composition, from a different preparation batch, could have the main contribution from the sample of interest plus a minor contribution from an impurity phase.

  In this case, all datasets are refined simultaneously as part of a single combined refinement, with both:

  • Shared (constrained) parameters across datasets for the main sample model
  • Independent parameters specific to each dataset, covering minor phases and instrument-specific parameters.

3. Sequential Refinement

• 3.1. Single or multiple sample models contributing to multiple datasets.
  For example, studying temperature dependence in a phase transition by measuring datasets at different temperatures.

  • The refinement is done sequentially: each dataset is refined one after another.
  • The refined parameters from one dataset are used as the starting point for the next.
    This sequential strategy is particularly important when working with large datasets, such as hundreds of measurements collected at small temperature steps.

• 3.2. Multiple sample models tested on a single measured dataset to select the best-fitting model.
  For example, comparing several candidate magnetic structures to determine which one best fits the data.

  • In this case, each model is refined independently; the results can be compared afterward to identify the best solution.
  • This could also be viewed as a type of parallel refinement, where multiple models are processed in sequence (or even simultaneously), but without interaction between their refinements.
    Perhaps this warrants introducing a fourth refinement type: parallel/separate/independent/decoupled refinement?

Additional Considerations

In all of the above cases, there doesn’t appear to be a strong need to specify an active experiment explicitly before the refinement begins. Instead, the user should only need to specify the type of refinement, which would automatically determine the appropriate strategy behind the scenes.

There may be other types of refinement workflows, particularly for other techniques, so it would be great to explore and discuss those as well.

[AndrewSazonov]

Next Portion of Agreed Points

1. Unification of the top levels of the API
   We agreed to unify the top few levels of the API structure:

   1. The Project class will serve as the central entry point and will be responsible for managing the entire workflow.

   2. The next level of classes will include:

      • ProjectInfo
      • SampleModel
      • Experiment
      • Analysis
      • Summary

      These classes will be shared across all techniques, ensuring a consistent framework.

   3. Below this level, things will start to differ between techniques, specifically in the content and structure of individual classes.
      However, we aim to fill them consistently, maintaining a logical separation of concerns:

      • SampleModel will only contain model parameters of the sample.
        For example, in diffraction, these are structural parameters, describing the crystal or magnetic structure.
      • Experiment will contain:
        • Measured data (if available)
        • Instrumental parameters
        • Background (if applicable)
        • A list of associated sample models that contribute to the experimentally measured (and calculated) pattern, along with the quantitative values of their contributions.

   4. In the GUI, more variation between techniques is allowed:

      • The order of steps can differ. For example:

        • In EasyDiffraction, the user defines the Sample Model first, then the Experiment.
        • In EasyImaging, the process is reversed—Experiment first, then Sample Model.

      • Similarly, how associated sample models (from the Experiment class in the library) are represented in the GUI can differ:

        • In EasyDiffraction, they might be shown under the Experiment (Phases) page.
        • In EasyImaging, they might be shown on the Analysis page.

2. Calculator and CalculatorFactory

   • The Calculator will be created by a CalculatorFactory and owned by the Analysis class.

   • However, for cases where the user wants to calculate and display an "idealized" pattern directly from the SampleModel, without involving the Analysis step (e.g., to quickly check if the calculated profile makes sense), a dummy Analysis object will be created under the hood with default parameters to calculate the required pattern.

   • This logic will be hidden from the user and exposed via class methods on the SampleModel class, such as:

     • model1.calculate()
     • model1.plot()

   • This approach preserves the current workflow in EasyReflectometry, where the idealized calculated pattern is displayed already on the Sample page.
     It also enhances EasyDiffraction, by adding an extra tab to the SampleModel page in the Main View. This tab will show the idealized diffraction pattern, in addition to the crystal structure visualization, which doesn't have a direct analogue in EasyReflectometry.

   • Switching to this tab in EasyDiffraction will display an extra group in the sidebar, offering parameters for a "virtual experiment", allowing the user to choose the type of data representation or experiment setup they wish to simulate.
     There could be a dozen visualization options:

     • Single-crystal spots or powder diffraction patterns
     • Neutron or X-ray
     • Time-of-flight or constant wavelength
     • 1D or 2D
     • Polarized or unpolarized
       etc.

Still Open Questions

1. The structure and implementation details of the MinimizerFactory, including naming conventions, are still to be discussed.

2. The discussion of refinement strategies will be moved to another ADR and addressed afterwards.

[damskii9992]

Regarding point 2. : Calculator and CalculatorFactory, the conclusion which I thought we reached was the following:

• The Calculator will be created by a CalculatorFactory which is called only by the Analysis class.
• The Analysis class will implement the class methods:
  • Analysis.plot(model=SampleModel, optional experiment=Experiment)
  • Analysis.calculate(model=SampleModel, optional experiment=Experiment)
    Which requires a SampleModel and optionally an Experiment object with an instrument model and will call the Calculator with these models and return the calculated curve or a plot of it.
• On the SampleModel a SampleModel.plot() and SampleModel.calculate() method will be implemented which simply calls the class methods of the Analysis class to produce the calculated curve and plot of the model itself.

[AndrewSazonov]

I initially had a different understanding of this, but your approach seems to be cleaner and better respects the separation of responsibilities.