Performance optimisation tracking (v0.9-STABILISATION)

[JayVDZ]

Overview

Parent tracking issue for all performance optimisation work targeted at v0.9-STABILISATION. This provides a single view of what needs to be done, what's in progress, and what's complete.

v0.9-STABILISATION scope

Import pipeline

• perf: batch pre-fetch CSO external ID mappings during full import #440 — FindMatchingCso batch pre-fetch (Phase 1 complete) — Pre-fetched external ID dictionary for O(1) lookup + PK-based hydration. Eliminates the LOWER() string-scan bottleneck. Phase 2 (page-level batch hydration) and Phase 3 (index alignment) remain as future enhancements.

Query and data-access layer

• Systematic database index and query performance analysis #427 — Systematic DB index + query analysis — Identify and fix missing indexes, eliminate unnecessary `.Include()` chains (absorbed Look for EF backlink optimisations #69), fix container processing order to avoid cross-batch fixups (absorbed perf: eliminate post-hoc cross-batch reference fixup by fixing container processing order #397).

Sync pipeline

• perf: investigate export rule evaluation performance (~100ms per rule per MVO) #417 — Export rule evaluation (~100ms per rule per MVO) — Investigate and optimise `EvaluateExportRules` / `CreateAttributeValueChanges` cost.
• Optimisation: Pre-export reconciliation of CREATE followed by DELETE PendingExports #218 — Pre-export CREATE→DELETE reconciliation — Avoid exporting objects that are immediately deleted.

Persistence layer

• Worker performance: MVO update conversion + advanced caching strategies #436 — MVO update conversion + caching strategies — Follow-on from completed Worker database performance optimisation - surgical raw SQL for hot paths #338. Raw SQL for MVO updates, query result caching, DbContext pooling. Note: perf: batch pre-fetch CSO external ID mappings during full import #440's Lookup/Hydrate seam provides the foundation for future caching — see architectural notes on Worker performance: MVO update conversion + advanced caching strategies #436.

Memory and scale

• Worker optimisation for large MVA groups (50K+ members) #383 — Large MVA groups (50K+ members) — Chunked reference processing, projection-based loading, materialised display names.

Connector layer

• perf: optimise AD schema discovery to batch attribute lookups #433 — AD schema discovery batching — Batch attribute lookups instead of N+1 LDAP queries.

Future milestones (labelled `performance` for discoverability)

• Draft strategy for parallel processing in JIM sync operations #214 (v1.0) — Parallel processing strategy for sync operations
• Optimise data generation for very large object counts (1M+) #276 (v2.0) — Data generation optimisation for 1M+ object test templates