A super-fast lookup service for canonical names based on tantivy.
juditha wants to solve the noise/garbage problem occurring when working with Named Entity Recognition. Given the availability of huge lists of known names, such as company registries or lists of persons of interest, one could canonize ner-results against this service to check if they are known.
The implementation uses a pre-populated tantivy index. Data is either FollowTheMoney entities or simply list of names.
pip install juditha
echo "Jane Doe\nAlice" | juditha load-names
juditha lookup "jane doe"
"Jane Doe"
To match more fuzzy, reduce the threshold (default 0.97):
juditha lookup "doe, jane" --threshold 0.5
"Jane Doe"
cat entities.ftm.json | juditha load-entities
juditha build
juditha load-names -i s3://my_bucket/names.txt
juditha load-entities -i https://data.ftm.store/eu_authorities/entities.ftm.json
juditha build
Following the nomenklatura specification, a dataset json config needs names.txt or entities.ftm.json in its resources.
juditha load-dataset https://data.ftm.store/eu_authorities/index.json
juditha load-catalog https://data.ftm.store/investigraph/catalog.json
juditha build
from juditha import lookup
assert lookup("jane doe") == "Jane Doe"
assert lookup("doe, jane") is None
assert lookup("doe, jane", threshold=0.5) == "Jane Doe"Juditha Dommer was the daughter of a coppersmith and raised seven children, while her husband Johann Pachelbel wrote a canon.
To mark the compatibility with followthemoney, juditha follows the same major version, which is currently 4.x.x.
juditha, (C) 2024 investigativedata.io
juditha, (C) 2025 Data and Research Center – DARC
juditha is licensed under the AGPLv3 or later license.