Across-the-board performance improvements, from 2x on simple single-term queries to 15x on phrase and proximity searches. On larger corpora (50K+ documents), multi-word query speedups are even greater. Collocations are up to ~100x faster, sorted KWIC up to ~25x faster, time series up to ~9x faster, and frequency reports and result aggregation ~1.5x faster. First page of results now generated server-side in ~100ms regardless of total result count.

Query type	v5.1.1.0	v5.2.0	Speedup
Single term (common)	110ms	51ms	2x
Single term (rare)	110ms	21ms	5x
Phrase search	310ms	21ms	15x
Proximity search	311ms	21ms	15x
Sentence search	336ms	31ms	11x
OR search	110ms	21ms	5x

Benchmarks on ELTeC corpus (100 novels). Larger corpora see proportionally greater gains.

Server architecture

Replaced Apache CGI with Gunicorn (threaded workers, Unix socket, systemd service).
Replaced the custom WSGI dispatcher with the Falcon framework.
Queries now run in-process in a background thread instead of spawning subprocesses.

Search engine

All word lookups, regex expansion, and NOT-term handling now use LMDB instead of subprocess/grep pipelines.
Metadata searches use an LMDB inverted word index instead of subprocess grep.
Multi-word searches (phrase, proximity, co-occurrence) use Numba JIT kernels and a merge-free per-document architecture.
Collocations and sorted KWIC rebuilt on a new columnar numpy database replacing the old sentences.lmdb, with memory-mapped arrays and Numba parallel kernels.

Other odds and ends

Metadata autocomplete in collocation fields.
Comprehensive test suite with gold set regression tests (byte-level hitlist comparison) and performance benchmarks with automated regression detection.
Updated for PhiloLogic5 architecture; GitHub Pages documentation site.

Assets 2

28 Jan 21:18

clovis

v5.1.1.0

ef603b3

PhiloLogic 5.1.1.0

Many accessibility fixes to the Web UI
Various security fixes
Performance fixes
Using a uv version of Python (3.12)
Using a local install of nvm for node/npm

Full Changelog: v5.1.0.1...v5.1.1.0

Assets 2

23 Oct 20:52

clovis

v5.1.0.1

083e6a6

PhiloLogic 5.1.0.1

Bug fixes...

Full Changelog: v5.1.0.0...v5.1.0.1

Assets 2

19 Sep 15:01

clovis

v5.1.0.0

c0e3f65

PhiloLogic 5.1

Many accessibility updates for the front-end. We are working towards WCAG 2.1 AA compliance.
Support for very large databases has been improved, by bypassing LMDB's limits for single value sizes.
Important fixes to metadata searching
Other bugs and performance improvements

Full Changelog: v5.0.0.2...v5.1.0.0

Assets 2