Unable to join Active Directory when FIPS is enabled on kernel

[kclinden]

Describe the bug

System fails to join active directory with error Error: LW_ERROR_ERRNO_ECONNREFUSED [code 0x00009d49]

This only happens when the Photon Kernel is configured in fips mode.

Reproduction steps

1. Install likewise-open
2. Configure Photon for FIPS and restart
3. Try to join Active Directory (also in fips mode)
4. Error
4a. If you run lsass directly then you get a more clear fips error.

...

Expected behavior

system joins domain successfully.

Additional context

lsass error

/opt/likewise/bin ]# /opt/likewise/sbin/lsassd --logfile /var/log/lsass.log --loglevel debug
md5_dgst.c(75): OpenSSL internal error, assertion failed: Low level API call to digest MD5 forbidden in FIPS mode!
Aborted (core dumped)

lsass verbose logs

20220408203816:DEBUG:0x7f1bd3fff700:[AD_JoinDomain() ../lsass/server/auth-providers/ad-open-provider/provider-main.c:2244] Joining domain HOME.LAB
20220408203816:VERBOSE:0x7f1bd3fff700:[AD_JoinDomain() ../lsass/server/auth-providers/ad-open-provider/provider-main.c:2264] Affinitized to DC 'dc01.home.lab' for join request to domain 'HOME.LAB'
20220408203816:DEBUG:0x7f1bd3fff700:[AD_PreJoinDomain() ../lsass/server/auth-providers/ad-open-provider/provider-main.c:2104] Clearing old join state
20220408203816:DEBUG:0x7f1bd3fff700:[RegTransactOpenKeyExW() ../lwreg/client/clientipc.c:439] Error at ../lwreg/client/clientipc.c:439 [status: LW_STATUS_OBJECT_NAME_NOT_FOUND = 0xC0000034 (-1073741772)]
20220408203816:DEBUG:0x7f1bd3fff700:[LwNtRegOpenKeyExA() ../lwreg/client/regntclient.c:931] Error at ../lwreg/client/regntclient.c:931 [status: LW_STATUS_OBJECT_NAME_NOT_FOUND = 0xC0000034 (-1073741772)]
20220408203816:DEBUG:0x7f1bd3fff700:[RegShellUtilGetValue() ../lwreg/shellutil/rsutils.c:1427] Error at ../lwreg/shellutil/rsutils.c:1427 [code: 40700]
20220408203816:DEBUG:0x7f1bd3fff700:[LwpsLegacyGetDefaultJoinedDomain() ../lsass/pstore/lsapstore-backend-legacy-internal.c:711] -> 0 (ERROR_SUCCESS) (EE = 685)
20220408203816:DEBUG:0x7f1bd3fff700:[RegTransactOpenKeyExW() ../lwreg/client/clientipc.c:439] Error at ../lwreg/client/clientipc.c:439 [status: LW_STATUS_OBJECT_NAME_NOT_FOUND = 0xC0000034 (-1073741772)]
20220408203816:DEBUG:0x7f1bd3fff700:[LwNtRegOpenKeyExA() ../lwreg/client/regntclient.c:931] Error at ../lwreg/client/regntclient.c:931 [status: LW_STATUS_OBJECT_NAME_NOT_FOUND = 0xC0000034 (-1073741772)]
20220408203816:DEBUG:0x7f1bd3fff700:[RegShellUtilGetValue() ../lwreg/shellutil/rsutils.c:1427] Error at ../lwreg/shellutil/rsutils.c:1427 [code: 40700]
20220408203816:DEBUG:0x7f1bd3fff700:[LwpsLegacyGetDefaultJoinedDomain() ../lsass/pstore/lsapstore-backend-legacy-internal.c:711] -> 0 (ERROR_SUCCESS) (EE = 685)
20220408203816:DEBUG:0x7f1bd3fff700:[RegTransactOpenKeyExW() ../lwreg/client/clientipc.c:439] Error at ../lwreg/client/clientipc.c:439 [status: LW_STATUS_OBJECT_NAME_NOT_FOUND = 0xC0000034 (-1073741772)]
20220408203816:DEBUG:0x7f1bd3fff700:[RegShellIsValidKey() ../lwreg/shellutil/rsutils.c:235] Error at ../lwreg/shellutil/rsutils.c:235 [code: 40700]
20220408203816:DEBUG:0x7f1bd3fff700:[RegShellUtilDeleteTree() ../lwreg/shellutil/rsutils.c:511] Error at ../lwreg/shellutil/rsutils.c:511 [code: 40700]
20220408203816:DEBUG:0x7f1bd3fff700:[RegTransactOpenKeyExW() ../lwreg/client/clientipc.c:439] Error at ../lwreg/client/clientipc.c:439 [status: LW_STATUS_OBJECT_NAME_NOT_FOUND = 0xC0000034 (-1073741772)]
20220408203816:DEBUG:0x7f1bd3fff700:[RegShellUtilGetKeyObjectCounts() ../lwreg/shellutil/rsutils.c:1175] Error at ../lwreg/shellutil/rsutils.c:1175 [code: 40700]
20220408203816:DEBUG:0x7f1bd3fff700:[LsaSetSMBCreds() ../lsass/server/auth-providers/ad-open-provider/join/lsakrb5smb.c:174] Switching default credentials path for new access token
20220408203816:DEBUG:0x7f1bd3fff700:[LwKrb5SetThreadDefaultCachePath() ../lwadvapi/threaded/lwkrb5.c:410] Switched gss krb5 credentials path from FILE:/tmp/krb5cc_0 to FILE:/tmp/tktNGeeqW
20220408203816:VERBOSE:0x7f1bf8f37700:[LsaSrvIpcCheckPermissions() ../lsass/server/api/ipc_state.c:79] Permission granted for (uid = 0, gid = 0, pid = 1008) to open LsaIpcServer
20220408203816:VERBOSE:0x7f1bf8f37700:[lwmsg_peer_log_accept() ../lwmsg/src/peer-task.c:271] (session:032f9b66556797c8-a4794c46271d654f) Accepted association 0x7f1be8003450
20220408203816:DEBUG:0x7f1bd37fe700:[NtlmServerAcquireCredentialsHandle() ../lsass/server/ntlm/acquirecreds.c:103] Error code: 40506 (symbol: LW_ERROR_NO_CRED)

[kclinden]

When setting up likewise I do the following:

root@photon3 [ /opt/likewise/bin ]# ./domainjoin-cli setname photon3
root@photon3 [ ~ ]# /opt/likewise/bin/lwregshell set_value '[HKEY_THIS_MACHINE\Services\lwio\Parameters\Drivers\rdr]' Smb2Enabled 1
root@photon3 [ /opt/likewise/bin ]# /opt/likewise/bin/lwsm restart lwio
Stopping service reverse dependency: vmafd
Stopping service reverse dependency: lsass
Stopping service reverse dependency: rdr
Stopping service: lwio
Starting service: lwio
Starting service reverse dependency: rdr
Starting service reverse dependency: lsass
Starting service reverse dependency: vmafd

root@photon3 [ /opt/likewise/bin ]# /opt/likewise/bin/domainjoin-cli --loglevel verbose --logfile somefile.log join home.lab klinden
Joining to AD Domain:   home.lab
With Computer DNS Name: photon3.home.lab

[kclinden]

partial coredump

Apr 08 20:38:19 photon3.home.lab systemd-coredump[1423]: Process 1371 (lsassd) of user 0 dumped core.
                                                         
                                                         Stack trace of thread 1416:
                                                         #0  0x00007f1bfa0e77ea raise (libc.so.6)
                                                         #1  0x00007f1bfa0e8881 abort (libc.so.6)
                                                         #2  0x00007f1bf9db4e2f OpenSSLDie (libcrypto.so.1.0.0)
                                                         #3  0x00007f1bf9dbb845 MD5_Init (libcrypto.so.1.0.0)
                                                         #4  0x00007f1bf76fe7a8 n/a (liblsass_auth_provider_ad_open.so)
                                                         #5  0x00007f1bf76fddc2 n/a (liblsass_auth_provider_ad_open.so)
                                                         #6  0x00007f1bf76f8fd3 n/a (liblsass_auth_provider_ad_open.so)
                                                         #7  0x00007f1bf76f7707 LsaJoinDomain (liblsass_auth_provider_ad_open.so)
                                                         #8  0x00007f1bf769066d n/a (liblsass_auth_provider_ad_open.so)
                                                         #9  0x00007f1bf7698c2b AD_ProviderIoControl (liblsass_auth_provider_ad_open.so)
                                                         #10 0x00007f1bfa5f8a9a LsaSrvProviderIoControl (liblsaserverapi.so.0)
                                                         #11 0x00007f1bfa5f1b4d n/a (liblsaserverapi.so.0)
                                                         #12 0x00007f1bfa5077d2 n/a (liblwmsg.so.0)
                                                         #13 0x00007f1bf9fd8c5f n/a (liblwbase.so.0)
                                                         #14 0x00007f1bf9fd79d3 n/a (liblwbase.so.0)
                                                         #15 0x00007f1bf9fd7b27 n/a (liblwbase.so.0)
                                                         #16 0x00007f1bfa271f87 n/a (libpthread.so.0)
                                                         #17 0x00007f1bfa1a560f __clone (libc.so.6)
                                                         
                                                         Stack trace of thread 1392:
                                                         #0  0x00007f1bfa277d7c pthread_cond_wait (libpthread.so.0)
                                                         #1  0x00007f1bfa2bb8b1 dcethread_cond_wait (libdcerpc.so.1)
                                                         #2  0x00007f1bfa2bb94d dcethread_cond_wait_throw (libdcerpc.so.1)
                                                         #3  0x00007f1bfa31e4a3 n/a (libdcerpc.so.1)
                                                         #4  0x00007f1bfa2bb0c5 n/a (libdcerpc.so.1)
                                                         #5  0x00007f1bfa271f87 n/a (libpthread.so.0)
                                                         #6  0x00007f1bfa1a560f __clone (libc.so.6)
                                                         
                                                         Stack trace of thread 1391:
                                                         #0  0x00007f1bfa277d7c pthread_cond_wait (libpthread.so.0)
                                                         #1  0x00007f1bfa2bb8b1 dcethread_cond_wait (libdcerpc.so.1)
                                                         #2  0x00007f1bfa2bb94d dcethread_cond_wait_throw (libdcerpc.so.1)
                                                         #3  0x00007f1bfa31e4a3 n/a (libdcerpc.so.1)
                                                         #4  0x00007f1bfa2bb0c5 n/a (libdcerpc.so.1)
                                                         #5  0x00007f1bfa271f87 n/a (libpthread.so.0)
                                                         #6  0x00007f1bfa1a560f __clone (libc.so.6)
                                                         
                                                         Stack trace of thread 1390:
                                                         #0  0x00007f1bfa277d7c pthread_cond_wait (libpthread.so.0)
                                                         #1  0x00007f1bfa2bb8b1 dcethread_cond_wait (libdcerpc.so.1)
                                                         #2  0x00007f1bfa2bb94d dcethread_cond_wait_throw (libdcerpc.so.1)
                                                         #3  0x00007f1bfa31e4a3 n/a (libdcerpc.so.1)
                                                         #4  0x00007f1bfa2bb0c5 n/a (libdcerpc.so.1)
                                                         #5  0x00007f1bfa271f87 n/a (libpthread.so.0)
                                                         #6  0x00007f1bfa1a560f __clone (libc.so.6)
                                                         
                                                         Stack trace of thread 1389:
                                                         #0  0x00007f1bfa277d7c pthread_cond_wait (libpthread.so.0)
                                                         #1  0x00007f1bfa2bb8b1 dcethread_cond_wait (libdcerpc.so.1)
                                                         #2  0x00007f1bfa2bb94d dcethread_cond_wait_throw (libdcerpc.so.1)
                                                         #3  0x00007f1bfa31e4a3 n/a (libdcerpc.so.1)
                                                         #4  0x00007f1bfa2bb0c5 n/a (libdcerpc.so.1)
                                                         #5  0x00007f1bfa271f87 n/a (libpthread.so.0)
                                                         #6  0x00007f1bfa1a560f __clone (libc.so.6)

[kclinden]

I am not totally sure, but I think it might be occurring here.
https://github.com/vmware/likewise-open/blob/2b535fa27889eeee24844f65ad3550f92c76f5e2/lsass/server/auth-providers/ad-open-provider/join/join.c#L1761

We should use AES instead of md5 or rc4.
This would be covered in section 3.2.2.4 AES Cipher Usage of the MS-SAMR.pdf