LGR Core library

Core library and LGR data structure used to manipulate an LGR.

This version conforms to the specification RFC7940.

Acknowledgment

This library was implemented by Viagenie (Julien Bernard, Michel Bernier, Guillaume Blanchet, Marc Blanchet, Vincent Gonzalez, Audric Schiltknecht and Alexandre Taillon-Desrochers) and Wil Tan on an ICANN contract.

Pre-Requisites

• Python 3.10 and above
• LibXML2 [MIT License] used by the lxml Python bindings

Python Dependencies

Main Dependencies

The main dependencies are defined in the requirements.txt file.

• language-tags [MIT License]
• lxml [BSD License]
• munidata for Unicode-related properties access [BSD License]
• picu [MIT/X License]
• pycountry [L-GPL License]

Testing Dependencies

The dependencies to run the test suite are defined in the tests requirements.txt.

• idna [BSD License]
• pytest [MIT License]
• pytest-cov [MIT License]

Documentation Dependencies

The dependencies to build the documentation are defined in the doc requirements.txt file.

• Graphviz [Eclipse Public License]
• pylint [GNU GPL License]
• Sphinx [BSD License]

Virtual Environment setup

If your distribution does not package the required dependencies, the easiest way to get a working environment in no-time is to use Python's virtual environments.

• Install virtualenv

• Create a python virtualenv:

    $ virtualenv venv
  

• Activate the environment:

    $ source ./venv/bin/activate
  

• Download dependencies:

    (venv) $ pip install -r requirements.txt
  

Tools

The tools directory contains some tools to exercise the library.

• xml_dump.py parses an XML LGR file, creates the LGR structure, and convert it back to XML.
• make_idna_repertoire.py generates the IDNA2008 starting repertoire, by parsing the IDNA2008 table registry available on IANA website, and collecting all PVALID, CONTEXTO and CONTEXTJ characters.
• lgr_cli.py parsed an XML LGR file, and validate it against configured MSR and Unicode version.
• rfcXXXX_dump.py parses inputs in RFC XXXX format, and generate an XML LGR.
• one_per_line_dump.py parses input in the "one per line" format, and generate an XML LGR.
• rfc7940_validate.py takes an LGR and checks it for compliance with RFC 7940.

Other tools are available to manipulate LGR files and labels:

• lgr_annotate.py takes an LGR and a list of labels and output the list of labels with their respective disposition.
• lgr_collision.py takes an LGR and a list of labels and check for collision(s) between the labels/variants.
• lgr_compare.py is used to compare 2 LGR (output textual diff, merge, intersection).
• lgr_diff_collision.py takes 2 LGR and one set of labels, and test for collisions between labels and generated variants from the 2 LGR.
• lgr_merge_set.py takes some LGRs and create a merged LGR from the provided set.
• lgr_check_harmonized checks that variants code points from a list of LGRs are symmetric and transitive in each LGR, and list missing variants from one LGR to another.
• lgr_cross_script_variants takes some LGRs and generate the list of cross-script variants.
• lgr_idn_table_review.py reviews IDN tables against reference LGRs.

Testing and coverage

Tests and coverage report can be run as follows:

(venv) $ pip install tox
(venv) $ tox

Open htmlcov/index.html in a web browser for the coverage report.

Documentation

To generate the documentation, go to the doc directory and run the following command:

(venv) $ make html

The generated documentation is available in doc/_build/html/index.html.