feat: OpenLDAP integration testing and multi-directory support#439
Merged
feat: OpenLDAP integration testing and multi-directory support#439
Conversation
Phase 1 of OpenLDAP integration testing plan. Adds Docker infrastructure for an OpenLDAP instance with two naming contexts (dc=regionA,dc=test and dc=regionB,dc=test) to enable true multi-partition import testing. - Dockerfile based on bitnamilegacy/openldap with bootstrap LDIFs - Init script creates second MDB database via cn=config at startup - Build script (Build-OpenLdapImage.ps1) with content-hash labelling - Docker Compose service replaces old osixia/openldap placeholder - Accesslog overlay enabled for future delta import testing - Verified: container starts healthy, both suffixes queryable Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ntions (#72) Establish consistent fictional domain and company names across all integration tests, unit tests, Docker infrastructure, and documentation. Names are inspired by Alastair Reynolds' science fiction universe. Domain mapping: - SUBATOMIC.LOCAL -> PANOPLY.LOCAL (primary AD directory) - SOURCEDOMAIN.LOCAL -> RESURGAM.LOCAL (cross-domain source) - TARGETDOMAIN.LOCAL -> GENTIAN.LOCAL (cross-domain target) - dc=regionA,dc=test -> dc=yellowstone,dc=local (OpenLDAP partition A) - dc=regionB,dc=test -> dc=glitterband,dc=local (OpenLDAP partition B) Company names: - Subatomic / Quantum Dynamics -> Panoply (primary HR company) - Quantum Bridge -> Rockhopper (contractor company) - Orbital Systems -> Akinya (contractor company) - GlobexCorp -> Panoply (example data) Email domains: - @subatomic.local / @testdomain.local -> @panoply.local - @example.com (in tests) -> @panoply.org - @sourcedomain.local -> @resurgam.local - @targetdomain.local -> @gentian.local Also removes stale performance result JSON files (historical data that referenced old domain names and had no ongoing value). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Mark Phase 1 (Docker Infrastructure) as complete with actual deliverables - Update domain naming throughout to match new conventions - Mark bootstrap OU structure, compose config, and accesslog overlay as done - Move plan doc to docs/plans/doing/ (now in progress) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Populate-OpenLDAP.ps1 creates inetOrgPerson users and groupOfNames groups across both OpenLDAP suffixes (Yellowstone and Glitterband). - Users split between suffixes with distinct indices so Scenario 9 can assert partition-scoped import returns only targeted objects - groupOfNames MUST constraint handled: initial member assigned during group creation, additional members via ldapmodify - LDIF piped via stdin to avoid uid 1001 file permission issues - Verified at Nano (3 users) and Micro (10 users) scales Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… 3 (#72) Add -DirectoryType parameter to the integration test framework, enabling the same test scenarios to run against Samba AD or OpenLDAP. - Get-DirectoryConfig in Test-Helpers.ps1: returns directory-specific configuration (container, host, port, bind DN, object classes, etc.) for SambaAD (Primary/Source/Target) or OpenLDAP - Run-IntegrationTests.ps1: -DirectoryType param controls Docker profile selection, health check targets, and OU preparation - LDAP-Helpers.ps1: all functions accept $DirectoryConfig hashtable alongside individual params for backward compatibility - Setup-Scenario1.ps1: LDAP connected system settings (host, port, bind DN, SSL, auth) driven by $DirectoryConfig - Invoke-Scenario1: container name for docker exec driven by config SambaAD remains the default — existing behaviour fully preserved. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When Run-IntegrationTests.ps1 is invoked without -DirectoryType, the interactive menu now presents a directory type selection screen (SambaAD or OpenLDAP) after scenario and template selection. SambaAD remains the default when -DirectoryType is passed explicitly or when the menu is bypassed. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…72) Replace the boolean IsActiveDirectory flag with a proper LdapDirectoryType enum (ActiveDirectory, OpenLDAP, Generic) for cleaner multi-directory support. This is Phase 4, Step 1 — external ID and directory type abstraction. LdapDirectoryType enum: - Detected from rootDSE: AD capability OIDs, vendorName for OpenLDAP - DetectDirectoryType() centralises detection logic in LdapConnectorUtilities Computed properties on LdapConnectorRootDse: - ExternalIdAttributeName: objectGUID (AD) / entryUUID (OpenLDAP/Generic) - ExternalIdDataType: Guid (AD) / Text (OpenLDAP/Generic) - UseUsnDeltaImport: true (AD) / false (others) - EnforcesSamSingleValuedRules: true (AD) / false (others) Schema discovery: - External ID recommendation uses computed property instead of hardcoded objectGUID — gracefully handles missing attribute with warning - Data type override uses computed property - Plurality override takes LdapDirectoryType instead of bool Export: - FetchObjectGuid renamed to FetchExternalId — fetches the correct attribute per directory type and parses accordingly (binary GUID for AD, string UUID for OpenLDAP) - LdapDirectoryType passed to export class via constructor All existing IsActiveDirectory references removed (7 locations). 24 new tests, all existing tests updated and passing. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Samba AD advertises AD capability OIDs but has genuinely different behaviour (broken paging, different error codes). It now gets its own LdapDirectoryType.SambaAD enum value instead of being handled as a post-hoc vendorName string check. Changes: - LdapDirectoryType.SambaAD enum value added - DetectDirectoryType returns SambaAD when AD OIDs + vendorName="Samba" - SupportsPaging is now a computed property (false for SambaAD, true for all others) — removes isSambaAd string checks from Utilities and Import - All computed properties updated: SambaAD shares AD behaviour for external ID (objectGUID), USN delta import, SAM single-valued rules - Export ParseExternalIdFromResponse handles SambaAD case - 8 new tests for SambaAD-specific behaviour Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Reflect the ISyncEngine/ISyncServer/ISyncRepository three-layer sync architecture introduced in the Worker Redesign (Phase 1c). Add new components (SyncEngine, SyncServer, DriftDetectionService, FileSystemServer, ExampleDataServer, ISyncRepository) to C4 model, update Worker processor relationships, and regenerate all SVGs. Update all 10 Mermaid diagrams from v0.3.0 to v0.7.1 with corrected flow annotations showing ISyncEngine for pure domain decisions and ISyncRepository for dedicated data access. Migrate jim-diagrams alias from deprecated structurizr/lite to structurizr/structurizr local. Update PowerShell cmdlet count to 79. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ry (#72) Phase 4 Steps 2-3: Make partition and schema discovery work for non-AD directories (OpenLDAP, 389DS, etc.). Partition discovery (Step 2): - Branch on LdapDirectoryType: AD/SambaAD use existing crossRef/systemFlags path; OpenLDAP/Generic query rootDSE namingContexts (RFC 4512) - All naming contexts returned as non-hidden partitions - Container discovery already works generically Schema discovery (Step 3): - Branch on LdapDirectoryType: AD/SambaAD use existing classSchema/ attributeSchema path; OpenLDAP/Generic query subschemaSubentry - New Rfc4512SchemaParser: tokeniser and parser for RFC 4512 objectClass and attributeType description strings with 37 unit tests - SYNTAX OID → AttributeDataType mapping (14 known RFC 4517 OIDs) - Writability from USAGE field and NO-USER-MODIFICATION flag - Superclass hierarchy walking for both object classes and attribute types - DESC field extraction for attribute/class descriptions - New "Include Auxiliary Classes" connected system setting: allows admins to include auxiliary object classes in schema discovery (both AD and RFC paths), off by default Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ale (#72) End-to-end validation: CSV → JIM → OpenLDAP provisioning works. Connector fix: - Synthesise distinguishedName attribute for all RFC schema object types (OpenLDAP doesn't expose it in subschema — it's implicit in every entry) Integration test parameterisation: - Partition selection uses DirectoryConfig.BaseDN (not hardcoded DC=panoply) - Container selection matches by full DN or short name (People vs Corp) - Object type lookup uses DirectoryConfig.UserObjectClass (inetOrgPerson) - LDAP attributes and export mappings branched by directory type - Expression mappings (DN template) parameterised per directory - Object matching rule uses correct attribute (employeeNumber vs employeeID) - Export rule name derived from ConnectedSystemName - Joiner verification uses Get-LDAPUser helper (not samba-tool) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…pport (#72) Replace all AD-specific verification with directory-agnostic helpers: - Joiner: Get-LDAPUser instead of samba-tool user show - Mover: Get-LDAPUser attribute check instead of samba-tool output parsing - Leaver: Test-LDAPUserExists instead of samba-tool user show - Reconnection: Test-LDAPUserExists for all three checks - Cleanup: ldapdelete for OpenLDAP, samba-tool for Samba AD - docker cp: use DirectoryConfig.ContainerName (not hardcoded samba-ad-primary) - Hardcoded panoply.local references replaced with DirectoryConfig.Domain Fix ldapsearch in LDAP-Helpers.ps1: - Use PowerShell argument splatting to avoid shell glob expansion of '*' - Add -LLL flag to suppress LDIF metadata comments - Filter out '*' from explicit attributes list (LDAP default returns all) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…pport (#72) Connector fix: - Synthesise entryUUID (external ID) attribute on all RFC schema object types — operational attributes aren't in any class's MUST/MAY list but are required for JIM to uniquely identify objects Integration test parameterisation: - All samba-tool verification replaced with Get-LDAPUser/Test-LDAPUserExists - All docker cp targets use DirectoryConfig.ContainerName (not hardcoded) - AD cleanup uses ldapdelete for OpenLDAP, samba-tool for SambaAD - Hardcoded panoply.local email/UPN domains replaced with DirectoryConfig - Mover, Leaver, Reconnection verifications all directory-agnostic Known issue: Mover export sends Create (not Update) against OpenLDAP — confirming delta import doesn't properly reconcile exported objects. Under investigation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…nLDAP confirmation (#72) Two fixes for OpenLDAP integration: 1. LdapConnectorImport: Only query cn=changelog during delta imports (not full imports). OpenLDAP doesn't expose cn=changelog, so the query always fails. Previously ran on every import, causing ERR log noise and activity failures. 2. Integration tests: Use Full Import (not Delta Import) for confirming exports against OpenLDAP. Delta import requires changelog support which OpenLDAP doesn't provide. Added IsOpenLDAP flag to test config. Remaining issue: Full Import confirming creates RPEIs that CompleteActivityBasedOnExecutionResultsAsync evaluates as "all items had errors" despite the import processing 3 objects successfully with 0 errors. The import/reconciliation works correctly but the activity status is incorrectly set to FailedWithError. Under investigation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
OpenLDAP doesn't return distinguishedName as an LDAP attribute — it's the entry's DN property. The import now synthesises it from searchResult.DistinguishedName so export confirmation can match the exported DN value against the imported value. Also fixes variable name bug in Invoke-SyncSequence ($isOpenLDAP → $Config.IsOpenLDAP) and assertion name for confirming import. This unblocks Joiner and Mover (Attribute Change) tests. Mover Rename fails due to Full Import confirmation RPEI error counting — under investigation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Implements proper delta import support for OpenLDAP using the slapo-accesslog overlay instead of the unsupported cn=changelog mechanism. Changes: - New GetDeltaResultsUsingAccesslog method: queries cn=accesslog for write operations (add/modify/delete/modrdn) since the previous watermark timestamp, then fetches current state for each changed object - New LastAccesslogTimestamp watermark on LdapConnectorRootDse: persisted as generalised time string (e.g., "20260326183000.000000Z") - New UseAccesslogDeltaImport computed property: true for OpenLDAP only - New QueryAccesslogForLatestTimestamp: finds latest reqStart by scanning all accesslog entries (no server-side sort required) - Improved OpenLDAP detection: falls back to structuralObjectClass (OpenLDAProotDSE) when vendorName is not set (common with Bitnami images) - Delta import routing: AD→USN, OpenLDAP→accesslog, Generic→changelog - Watermark capture runs during both full and delta imports so the baseline full import establishes the starting position Reverted the Full Import confirmation workaround — delta import now works properly for export confirmation against OpenLDAP. Test results (Nano): Joiner ✓, Mover (Attribute Change) ✓ Remaining: Mover Rename fails (delta import after ModifyDN — under investigation) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The IsRdnAttribute method was hardcoding cn as an RDN attribute, which prevented cn from being modified via LDAP ModifyRequest on OpenLDAP (where the RDN is uid, not CN). Now parses the actual RDN attribute from the object's DN. Also parameterises Mover Rename verification for OpenLDAP — checks cn/displayName update instead of DN change (uid-based DNs don't change when displayName changes). Test results (Nano): Joiner ✓, Mover ✓, Mover-Rename ✓ Remaining: Mover-Move and Leaver verification use AD-specific checks. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Parameterised remaining Scenario 1 verifications for multi-directory:
- Mover Move: checks departmentNumber attribute (not OU in DN) for OpenLDAP
- Disable/Enable: skipped for OpenLDAP (no userAccountControl equivalent)
All test steps now pass at Nano scale against OpenLDAP:
✓ Joiner, Mover, Mover-Rename, Mover-Move, Disable*, Enable*,
Leaver, Reconnection (*skipped — AD-specific)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…#72) Phases 4-5 marked complete with detailed deliverables. Phase 6 roadmap added with all 8 remaining scenarios prioritised by effort vs value, implementation advice for each, and key design decisions needed (Gap 6: groupOfNames constraint for S8). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Scenario 9 (Partition-Scoped Imports) now supports both Samba AD and OpenLDAP via DirectoryConfig. For OpenLDAP, both partitions (Yellowstone + Glitterband) are selected and four run profiles created, enabling true partition isolation assertions — scoped imports to each partition return only that partition's users. Runner updated with Step 4c to populate OpenLDAP with test data before scenarios run. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…erations (#342) Adds "Connecting to connected system" and "Importing objects from connected system" activity messages so administrators can see what phase the import is in before object processing begins. File-based connectors show "Importing objects from file". This completes the phase messaging requested in #342. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…appings (#72) OpenLDAP RFC 4512 schema correctly marks uid, givenName, sn, and departmentNumber as multi-valued (no SINGLE-VALUE keyword). JIM rightly rejects multi-valued→single-valued import flows. Use displayName and employeeNumber (both SINGLE-VALUE) for the import sync rule instead. Also restructure test to run both scoped imports before a single Full Sync, avoiding a concurrency issue with back-to-back sync runs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…263) Move comprehensive, type-aware reconciliation logic from PendingExportReconciliationService into SyncEngine as pure, stateless methods. Both the sync path (EvaluatePendingExportConfirmation) and the import path (ReconcileCsoAgainstPendingExport) now share identical attribute matching that handles all 8 data types. The old AttributeValuesMatch method only covered 5 types — Boolean, Guid, and LongNumber attributes silently failed to match, leaving exports stuck. PendingExportReconciliationService is now a thin async orchestration wrapper (load → delegate to SyncEngine → persist). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…orts (#72) Populate-OpenLDAP.ps1 used indices starting at 1, colliding with CSV-generated user indices (0–99) during Scenario 1 LDAP export. This caused 49 "The object exists" errors on a clean environment. Apply the same $ldapIndexOffset = 500000 strategy already used by Populate-SambaAD.ps1 to guarantee non-overlapping uids at all template sizes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
S7 (Clear Connected System Objects): Add DirectoryConfig parameter and pass through to Setup-Scenario1. No AD-specific logic — scenario is entirely CSV-based. S6 (Scheduler Service): Add DirectoryConfig parameter, replace hardcoded "Panoply AD" with DirectoryConfig.ConnectedSystemName, and use jim.web container for CSV file copies (directory-agnostic). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
CSV test data is bind-mounted from the host into JIM containers. The docker cp to samba-ad-primary was redundant (it copied to a different volume). Removing it fixes OpenLDAP compatibility and simplifies the test — host file changes are immediately visible via the bind mount. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add Source (Yellowstone) and Target (Glitterband) instance configs for OpenLDAP, mirroring the SambaAD Source/Target pattern. Both use the same openldap-primary container but different suffixes, enabling cross-domain sync testing (Scenario 2) without additional containers. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ries (#341) Expression attribute lookups (mv["Department"], cs["sAMAccountName"]) were case-sensitive because the underlying dictionaries used the default StringComparer.Ordinal. This made expressions fail silently when the attribute name casing in the expression didn't exactly match the stored attribute name. Pass StringComparer.OrdinalIgnoreCase to all four BuildAttributeDictionary / BuildCsoAttributeDictionary methods. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Setup and invoke scripts fully parameterised with DirectoryConfig. OpenLDAP uses Yellowstone (source) and Glitterband (target) suffixes on the same container. Connection settings, object types, attributes, DN expressions, user creation, and verification all driven by config. End-to-end test is blocked by #435 (MVA→SVA import handling): OpenLDAP marks uid, sn, givenName as multi-valued per RFC 4512, preventing direct import mappings to single-valued MV attributes. The export side works (SVA→MVA allowed) but confirming imports fail because exported values to multi-valued targets accumulate rather than replace. Once #435 is implemented, S2 OpenLDAP should pass without further script changes — just update the import mappings to include uid, sn, givenName with first-value selection. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
All four phases complete. Move plan to docs/plans/done/. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ng (#72) The deep EF Include chain (CSO → AttributeValues → ReferenceValue → MetaverseObject) produced a cartesian explosion at scale (5000+ CSOs, 80K+ member references) that caused EF Core to intermittently fail to materialise ReferenceValue navigations — different ones each run. This led to false drift corrections that removed group members from Target LDAP during the confirming sync. Replace the Include chain with PopulateReferenceValuesAsync, which uses a direct SQL query to reliably fetch referenced CSO IDs and their MetaverseObjectIds. A new [NotMapped] property ResolvedReferenceMetaverseObjectId on ConnectedSystemObjectAttributeValue carries this data without touching EF navigations or the change tracker. All consumers (DriftDetectionService, SyncEngine, SyncRuleMappingProcessor, SyncTaskProcessorBase) updated to read the scalar property first with navigation fallback for in-memory test compatibility. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ProcessReferenceAttribute sets the ReferenceValue navigation for same-page references (where the MVO may not be persisted yet and EF needs to handle insert ordering) and ReferenceValueId scalar FK for cross-page references (where the MVO already exists in the database). EF does not reliably infer the scalar FK from the navigation when using explicit Entry().State management (UpdateMetaverseObjectsAsync), leaving ReferenceValueId NULL in the database. Two fixes address this: 1. CreateMetaverseObjectsBulkAsync: removed the batchMvoIds restriction from the ReferenceValueId fixup, so references to MVOs from previous pages (already persisted) also get their FK set. 2. PersistPendingMetaverseObjectsAsync: added a tactical SQL UPDATE fixup (FixupMvoReferenceValueIdsAsync) that runs after MVO updates, setting ReferenceValueId where EF failed to infer it. This captures fixup data before SaveChangesAsync (which clears navigations) and matches by (MvoId, AttributeId) since attribute value IDs may not be assigned yet. TACTICAL: retire when MVO persistence moves to direct SQL. Also: in-memory SyncRepository fixup for ReferenceValueId from navigation (handles Guid.Empty edge case), and test assertions updated to check both FK and navigation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add-DirectoryGroupMember and Remove-DirectoryGroupMember passed full DNs (e.g. uid=quinn.baker96,ou=People,...) to Get-LDAPUser which expects a plain username. The LDAP filter became (uid=uid=...) and returned no results, causing member removal to silently fail during drift testing. Now detect if MemberName is already a full DN (contains = and ,) and use it directly, skipping the lookup. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The docker image prune in jim-reset only excluded Samba AD labels, causing OpenLDAP snapshot and base images to be deleted on every reset. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The integration test runner starts the full Docker stack (including Keycloak) but did not start the socat bridge (8181→8180) that makes Keycloak accessible in the browser via VS Code port forwarding. This meant port 8181 showed no running process during test runs, preventing developers from accessing the portal to monitor progress. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…rts (#72) The accesslog MDB database's default mapsize (~10MB) was silently exhausted during MediumLarge exports (5000+ objects), causing MDB_MAP_FULL errors and subsequent modify operations to not be logged. This made delta imports miss drift changes at scale. - Increase accesslog olcDbMaxSize to 1GB to prevent MDB_MAP_FULL - Set accesslog olcSizeLimit to unlimited (OpenLDAP enforces this as a hard cap even with paging controls for non-rootDN clients) - Add warning log when GetObjectByDn returns null during accesslog delta import, improving diagnostics for missed changes Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…72) LDAP object class names use camelCase (groupOfNames, inetOrgPerson, organizationalUnit) rather than PascalCase. The regex only matched [A-Z][a-z]+ segments, dropping the leading lowercase word entirely (e.g. "groupOfNames" → "Of Names" instead of "Group Of Names"). Updated regex to also capture a leading lowercase segment before the first capital letter, and title-case it in the output. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The bash subshell spawned by PowerShell exits immediately after launching the background socat process, which kills socat with it. Using nohup detaches socat from the parent shell so it persists for the duration of the test run. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When the integration test runner exits (e.g. SetupOnly mode), PowerShell kills all child processes in its process group. Neither nohup nor background (&) alone prevent this. Use setsid + disown via a temp script to create a new session, fully detaching socat so it survives after the PowerShell process exits. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The Scenario 8 population script places groups in OU=Entitlements, but the test validation was querying OU=Groups via the default SambaAD config, causing "No test groups found in Source directory" failures. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…RPEIs (#72) Two fixes for delta import behaviour on OpenLDAP: 1. When the accesslog is empty (e.g. after snapshot restore), the full import now generates a fallback timestamp as the watermark instead of storing null. This prevents the next delta import from falling back to a full import unnecessarily, avoiding re-importing all objects for no reason. 2. Connector-level warnings (like DeltaImportFallbackToFullImport) are now stored on Activity.WarningMessage instead of being created as phantom RPEIs with no CSO association. This eliminates the misleading extra row in the RPEI list, stops inflating error counts, and displays the warning as a banner on the activity detail page instead. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
PowerShell's Set-Content writes \r\n line endings, causing bash to fail with "$'\r': command not found". Use WriteAllText with explicit \n newlines instead. Also keeps the temp file approach (setsid + disown) which is required for socat to survive PowerShell exit. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…tting done (#72) - S8 marked done (all 6 tests passing at MediumLarge) - OpenLDAP snapshotting marked resolved - S3 marked deferred (out of scope) - Updated S8 notes with accesslog, watermark, and RPEI fixes - Remaining: Samba AD regression + scale testing Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The OpenLDAP parameterisation (24bfcd5) set LdapSearchPort/Scheme to 636/ldaps for Samba AD configs, matching JIM's connector settings. But these fields are used by Invoke-LDAPSearch which runs ldapsearch inside the container against localhost — where the self-signed TLS cert causes connection failures. Revert to 389/ldap which is what the original hardcoded Invoke-LDAPSearch used before parameterisation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…S8 (#72) The OpenLDAP parameterisation switched Get-DirectoryGroupMembers to use Get-LDAPGroupMembers (ldapsearch) for both directory types. For Samba AD this returns full DNs with CN=DisplayName, which don't match the sAMAccountName values from Get-DirectoryUserList. This broke Test-MemberInList matching, causing the test to think no users were in any group and then try to "add" users who were already members. Fix: use samba-tool group listmembers for the Samba AD path, which returns sAMAccountNames directly — matching the original behaviour before parameterisation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds a third option to the -DirectoryType parameter: "All", which runs the test suite for Samba AD first, then OpenLDAP. Available in both the CLI parameter and the interactive arrow-key menu. Also passes DirectoryType through in the -Scenario All recursive calls, which was previously missing (defaulting to SambaAD silently). Usage: ./Run-IntegrationTests.ps1 -Scenario All -Template Small -DirectoryType All Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- docs/INTEGRATION_TESTING.md: add DirectoryType parameter table, usage examples, and OpenLDAP column to scenario table - test/CLAUDE.md: add -DirectoryType OpenLDAP and -DirectoryType All examples to integration testing quick reference Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
OpenLDAP scenarios never use Samba AD containers. The unconditional Samba AD image check/build in Step 0 was causing false S6 failures when the image had been pruned between scenarios — the rebuild could take 600+ seconds and time out under disk/IO pressure. The Source/Target image check already had a DirectoryType guard but the Primary image check did not. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Scenario 1 tests HR-driven provisioning into a target directory. The pre-populated users (from Populate-SambaAD.ps1 / Populate-OpenLDAP.ps1) were completely unrelated identities that could never match the HR source data — different index ranges, no employee ID attribute set. They just added noise to the connector space without testing any matching logic. The target directory now starts clean (OUs only) for S1, which better reflects the scenario's purpose: introducing automated ILM to provision identities from an authoritative HR source into an empty directory. Other scenarios (S5, S8, S9) that need pre-populated data are unaffected. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… violations (#72) When creating MVOs in parallel across multiple database connections, reference attribute values (e.g. group members) could reference MVOs in a different partition's uncommitted transaction, causing FK constraint violations on ReferenceValueId. Split the parallel write into two phases: Phase 1 inserts all MVO rows (all partitions commit), Phase 2 inserts attribute values (all referenced MVOs now exist). This fixes Scenario 8 failures on Medium+ templates where the batch exceeds the parallelism threshold. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ailability The socat install in setup.sh (postCreateCommand) could silently fail due to network issues or apt cache state, leaving the Keycloak 8181 bridge broken. Moving it to the Dockerfile bakes it into the image layer reliably. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The docker image prune commands in Run-IntegrationTests.ps1 only protected snapshot-hash labels but not build-hash labels, causing the locally-built OpenLDAP base image to be deleted after every test run. This forced a full base image rebuild + snapshot recreation on each subsequent run. Samba AD was unaffected because its base image comes from an external registry. Added the missing build-hash filters to match jim-aliases.sh which already had the correct four-filter set. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
) OpenLDAP's RFC 2696 paging cookies are connection-scoped — a new search invalidates all outstanding cursors. Previously imports were serialised one combo at a time on a single connection. Now each container+objectType combo gets its own dedicated LdapConnection and runs in parallel, capped by a configurable "Import Concurrency" connector setting (default 4, max 8). - Extract CreateConnection factory from OpenImportConnection for reuse - Add GetFullImportObjectsParallel with SemaphoreSlim-capped concurrency - Add GetFullImportObjectsSequential as fallback for concurrency=1 - Add DrainAllPages helper that fully paginates a combo on one connection - Add Import Concurrency connector setting (General/Import category) - Add RFC 4533 LDAP Content Sync research note for future reference Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
-DirectoryType Alloption runs full regression against both directory types sequentiallyentryUUIDsupport, DN-aware exportLdapDirectoryTypeenum centralises directory-specific behaviourgroupOfNamesCloses #72
Test plan
-DirectoryType Allregression at Small — both directory types pass🤖 Generated with Claude Code