Skip to content

[wishlist]: Read calibration data from pypeit instead of downloading it #292

@Hellseher

Description

@Hellseher

Hi,

While preparing Guix Astro Update 2025/11 and refreshing python-specreduce to 1.7.0 I've noticed
that it tried to download calibration data from GitHub for one test:

    def load_pypeit_calibration_lines(
            lamps: Sequence | None = None,
            wave_air: bool = False,
            cache: bool | Literal['update'] = True,
            show_progress: bool = False
    ) -> QTable | None:
        """
        Load reference calibration lines from ``pypeit`` linelists. The ``pypeit`` linelists are
        well-curated and have been tested across a wide range of spectrographs. The available
        linelists are defined by ``PYPEIT_CALIBRATION_LINELISTS``.

        Parameters
        ----------
        lamps : Lamp string, comma-separated list of lamps, or sequence of lamps to include in
            output reference linelist. The parlance of "lamp" is retained here for consistency
            with its use in ``pypeit`` and elsewhere. In several of the supported cases the
            "lamp" is the sky itself (e.g. OH lines in the near-IR).
            The available lamps are defined by ``PYPEIT_CALIBRATION_LINELISTS``.

        wave_air : If True, convert the vacuum wavelengths used by ``pypeit`` to air wavelengths.

        cache : Toggle caching of downloaded data

        show_progress : Show download progress bar

        Returns
        -------
        linelist:
            Table containing the combined calibration line list. ``pypeit`` linelists have the
            following columns:
            * ``ion``: Ion or molecule generating the line.
            * ``wavelength``: Vacuum wavelength of the line in Angstroms.
            * ``NIST``: Flag denoting if NIST is the ultimate reference for the line's wavelength.
            * ``Instr``: ``pypeit``-specific instrument flag.
            * ``amplitude``: Amplitude of the line. Beware, not consistent between lists.
            * ``Source``: Source of the line information.
        """
        if lamps is None:
            return None

        if not isinstance(lamps, Sequence):
            raise ValueError(f"Invalid calibration lamps specification: {lamps}")

        if isinstance(lamps, str):
            if ',' in lamps:
                lamps = [lamp.strip() for lamp in lamps.split(',')]
            else:
                lamps = [lamps]

        linelists = []
        for lamp in lamps:
            if lamp in PYPEIT_CALIBRATION_LINELISTS:
                data_url = f"{PYPEIT_DATA_URL}/arc_lines/lists/{lamp}_lines.dat"
                try:
                    data_path = download_file(data_url, cache=cache,
                                              show_progress=show_progress,
                                              pkgname='specreduce')
                    linelists.append(Table.read(data_path, format='ascii.fixed_width', comment='#'))
                except URLError as e:
>                   warnings.warn(f"Downloading of {data_url} failed: {e}", AstropyUserWarning)
E                   astropy.utils.exceptions.AstropyUserWarning: Downloading of https://raw.githubusercontent.com/pypeit/pypeit/release/pypeit/data//arc_lines/lists/ArI_lines.dat failed: <urlopen error [Errno -3] Temporary failure in name resolution>

specreduce/calibration_data.py:167: AstropyUserWarning
=========================== short test summary info ============================
FAILED specreduce/tests/test_wavecal1d.py::test_init_line_list - astropy.utils.exceptions.AstropyUserWarning: Downloading of https://raw.githubusercontent.com/pypeit/pypeit/release/pypeit/data//arc_lines/lists/ArI_lines.dat failed: <urlopen error [Errno -3] Temporary failure in name resolution>
================= 1 failed, 104 passed, 24 deselected in 6.35s =================

However pypeit provides data in PyPI package, see https://github.com/pypeit/PypeIt/tree/1.17.4/pypeit/data.
May it be helpful to have some environment variable to check for calibration data from or try to read it from the pypeit package?

Thanks,
Oleg

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions