Package building/Installation error

[Aliasgarsaifee]

Can @karthik-mb you help me with packaging it and installing from source.
Tried installing via - pip install git+https://github.com/Kenpath/indic-text-normalization.git@main

But then getting import errors. Most probably issue with setup.py file.

from indic_text_normalization import Normalizer
Normalizer(lang="en")
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Cell In[2], line 2
      1 from indic_text_normalization import Normalizer
----> 2 Normalizer(lang="en")

File ~/.virtualenvs/venv/lib/python3.11/site-packages/indic_text_normalization/normalize.py:131, in Normalizer.__init__(self, input_case, lang, cache_dir, overwrite_cache, whitelist, post_process, max_number_of_permutations_per_split)
    128 base_path = f'indic_text_normalization.{lang}'
    130 # Load tagger
--> 131 tagger_module = importlib.import_module(f'{base_path}.taggers.tokenize_and_classify')
    132 ClassifyFst = tagger_module.ClassifyFst
    134 # Load verbalizer

File /opt/conda/lib/python3.11/importlib/__init__.py:126, in import_module(name, package)
    124             break
    125         level += 1
--> 126 return _bootstrap._gcd_import(name[level:], package, level)

File <frozen importlib._bootstrap>:1204, in _gcd_import(name, package, level)

File <frozen importlib._bootstrap>:1176, in _find_and_load(name, import_)

File <frozen importlib._bootstrap>:1147, in _find_and_load_unlocked(name, import_)

File <frozen importlib._bootstrap>:690, in _load_unlocked(spec)

File <frozen importlib._bootstrap_external>:940, in exec_module(self, module)

File <frozen importlib._bootstrap>:241, in _call_with_frames_removed(f, *args, **kwds)

File ~/.virtualenvs/venv/lib/python3.11/site-packages/indic_text_normalization/en/taggers/tokenize_and_classify.py:21
     18 import pynini
     19 from pynini.lib import pynutil
---> 21 from indic_text_normalization.en.graph_utils import (
     22     NEMO_WHITE_SPACE,
     23     GraphFst,
     24     delete_extra_space,
     25     delete_space,
     26     generator_main,
     27 )
     28 from indic_text_normalization.en.taggers.abbreviation import AbbreviationFst
     29 from indic_text_normalization.en.taggers.cardinal import CardinalFst

File ~/.virtualenvs/venv/lib/python3.11/site-packages/indic_text_normalization/en/__init__.py:15
      1 # Copyright (c) 2021, NVIDIA CORPORATION.  All rights reserved.
      2 #
      3 # Licensed under the Apache License, Version 2.0 (the "License");
   (...)     12 # See the License for the specific language governing permissions and
     13 # limitations under the License.
---> 15 from indic_text_normalization.en.taggers.tokenize_and_classify import ClassifyFst
     16 from indic_text_normalization.en.verbalizers.verbalize import VerbalizeFst
     17 from indic_text_normalization.en.verbalizers.verbalize_final import VerbalizeFinalFst

ImportError: cannot import name 'ClassifyFst' from partially initialized module 'indic_text_normalization.en.taggers.tokenize_and_classify' (most likely due to a circular import) (/home/jovyan/.virtualenvs/venv/lib/python3.11/site-packages/indic_text_normalization/en/taggers/tokenize_and_classify.py)

[karthik-mb]

Hi @Aliasgarsaifee,

Thanks for reporting this.

We’ve already attempted to properly package the project. However, the main difficulty comes from pynini, which can be challenging to install via pip. While pip works when a compatible prebuilt wheel is available, it often fails when it attempts to build from source, since pynini depends on OpenFst and requires C++ compilation and system-level dependencies.

At the moment, the most reliable installation methods are:

• Conda (recommended), as it installs prebuilt binaries and handles native dependencies automatically.
• uv with a prebuilt wheel, where available.

That said, we will retry the packaging process on our end and explore alternative approaches to make pip-based installation smoother and more reliable.

If you’ve found a stable pip-only solution that works consistently across platforms, we’d be happy to review and consider it.

Thanks again for bringing this up!

[Aliasgarsaifee]

For linux based systems pynini gets installed with pip not any issues are faced.
I have used Nemo-one using

python -m pip install git+https://github.com/NVIDIA/NeMo-text-processing.git@{BRANCH}

works with linux.

[karthik-mb]

Thanks for the clarification.

I'll take another look at this on our end, try to reproduce it in a clean Linux environment, and work on a proper fix. I’ll update the thread once we have more information.

Appreciate your feedback.

[Aliasgarsaifee]

Can we connect on short meet @karthik-mb ? share you email id

[karthik-mb]

Hi @Aliasgarsaifee,

I’ve made changes to the packaging, and you should now be able to install it using:

pip install git+https://github.com/Kenpath/indic-text-normalization.git

The installation is now working correctly on Linux. I’m currently testing on Windows and macOS, as there still seem to be some issues there. I am working on it.

If the issue still persists after this update, we can connect briefly and go through it together.

Thanks for your patience!

[karthik-mb]

Can we connect on short meet @karthik-mb ? share you email id

Hi @Aliasgarsaifee,
Sure, we can connect. You can reach me at karthik.m@kenpath.io.
Looking forward to discussing this further.

[Aliasgarsaifee]

Yes now installation is happening, testing further.
Have send you an email also, please check and let me know.