The "--kerberos=true" is not recognized

[Wodel]

Hi,

I've managed to get Sybil working on my POC, now I am trying to understand how to use it...

What was done so far :

• Compilation and creation of sybil rpms with support of slurm : make rpm WITH_SLURM=1 SLURM_VERSION=25.05
• Installation and configuration of sybild on the freeipa server.
• Creation of servicedelegationtarget and servicedelegationrule...
• Installation of sybil cli and the spank sybil plugin in : slurm-controller, login node, compute nodes.
• I have configured slurm to use the plugin using the example file : "required spank_sybil.so default=no min_tkt_lifetime="
• I've modified the slurmd systemd service file (on the compute nodes only).

Note : the login node is excluded from computation, and slurmd in not started, this node is not intended to be used for jobs.

As root on the login node I did : sybil kinit gandalf@EXAMPLE.LOCAL
then I did

ssh gandalf@login01

[gandalf@login01 ~]$ klist 
Ticket cache: KCM:1033000003:94279
Default principal: gandalf@EXAMPLE.LOCAL

Valid starting       Expires              Service principal
08/04/2025 10:54:44  08/04/2025 20:54:44  krbtgt/EXAMPLE.LOCAL@EXAMPLE.LOCAL
        renew until 08/11/2025 10:54:44
08/04/2025 10:56:30  08/04/2025 20:54:44  sybil/idm01.example.local@EXAMPLE.LOCAL
        renew until 08/11/2025 10:54:44


[gandalf@login01 ~]$ sybil store

[gandalf@login01 ~]$ sybil list
UID         START_TIME           END_TIME             RENEW_UNTIL          PRINCIPAL
1033000003  2025-08-04T10:56:43  2025-08-04T20:54:44  2025-08-11T10:54:44  gandalf@EXAMPLE.LOCAL

But when using srun I got :

[gandalf@login01 ~]$ srun --kerberos=true klist
srun: error: Invalid --kerberos argument: true

it is not recognizing --kerberos, I tried to do the same directly on one of the compute nodes, same error !!!!????

I have also some questions :

• "Allow a given host to perform impersonation against the Sybil service" : What hosts should I added, all of them or only the login node(s) only???
• KCM configuration, where should this be done? only on freeipa server (this is my understanding), or on all servers enrolled in freeipa including freeipa server itself?
• Since the "sybil kinit user@DOMAIN.LAN" has to be done as root, how can this be automated for a user when he logs-in without opening security holes?

Regards.

[3XX0]

• If you don't see the --kerberos option it means the SPANK plugin wasn't installed properly.
• For KCM it's up to you but it needs to be where the KDC and Sybil server is at the very least
• I'm not sure what you're trying to do with sybil kinit. This command is supposed to be used to impersonate users during protocol transition (e.g. CI/CD). This is not intended to provide your users a ticket on log-in

[Wodel]

Hi,

Thanks for your replies.

• I can see the plugin loaded properly by slurm, there is no error in the log file (I am using a special machine to build rpms and software for my POC). The documentation of Sybil, does not suggest to install/build against slurm-devel. I did mention only the version I am using make release WITH_SLURM=1 SLURM_VERSION=25.05. But anyways, I did rebuild Sybil after installing slurm-devel in my build machine but it didn't change anything.

[2025-08-05T12:38:32.183] [105.0] debug:  spank: opening plugin stack /etc/slurm/plugstack.conf.d/sybil.conf
[2025-08-05T12:38:32.184] [105.0] debug3: plugin_peek->_verify_syms: found Slurm plugin name:sybil type:spank version:0x190500
[2025-08-05T12:38:32.185] [105.0] debug3: plugin_load_from_file->_verify_syms: found Slurm plugin name:sybil type:spank version:0x190500
...
...
[2025-08-05T12:38:32.185] [105.0] debug:  spank: /etc/slurm/plugstack.conf.d/sybil.conf:1: Loaded plugin spank_sybil.so
....
....
[2025-08-05T12:38:32.238] [105.0] debug:  SPANK: appending plugin option "kerberos"
[2025-08-05T12:38:32.238] [105.0] debug2: spank: spank_sybil.so: init = 0
[2025-08-05T12:38:32.238] [105.0] debug2: spank: spank_sybil.so: init_post_opt = 0
[2025-08-05T12:38:32.238] [105.0] debug2: After call to spank_init()

I can see things like

Aug 03 10:57:01 idm01.example.local krb5kdc[1582](info): TGS_REQ (4 etypes {aes256-cts-hmac-sha384-192(20), aes128-cts-hmac-sha256-128(19), aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17)}) 10.240.9.4: ISSUE: authtime 1754215015, etypes {rep=aes256-cts-hmac-sha384-192(20), tkt=aes256-cts-hmac-sha384-192(20), ses=aes256-cts-hmac-sha384-192(20)}, host/login01.example.local@EXAMPLE.LOCAL for sybil/idm01.example.local@EXAMPLE.LOCAL
Aug 03 10:57:01 idm01.example.local krb5kdc[1582](info): closing down fd 11
Aug 03 10:58:51 idm01.example.local krb5kdc[1586](info): TGS_REQ (4 etypes {aes256-cts-hmac-sha384-192(20), aes128-cts-hmac-sha256-128(19), aes256-cts-hmac-sha1-96(18), aes128-cts-hmac-sha1-96(17)}) 10.240.9.4: ISSUE: authtime 1754215015, etypes {rep=aes256-cts-hmac-sha384-192(20), tkt=aes256-cts-hmac-sha384-192(20), ses=aes256-cts-hmac-sha384-192(20)}, host/login01.example.local@EXAMPLE.LOCAL for host/login01.example.local@EXAMPLE.LOCAL
Aug 03 10:58:51 idm01.example.local krb5kdc[1586](info): ... PROTOCOL-TRANSITION s4u-client=gandalf@EXAMPLE.LOCAL

In /var/log/krb5kdc.log in freeipa server.

• For KCM it was configured everywhere including freeipa server where sybild is running.
• What am I trying to do?
  - I have a kerberized NFS /home with ipa-automount.
  - When a user ssh/logs into the login node, its home is mounted, but this home is not mounted yet in compute nodes.
  - So when I launch an srun or sbatch they fail with permission denied.
  - Searching for a way to make this work, I found auks and sybil, sybil seemed to be newer with more features so I tried to use it.
  - Then I tried auks, and it worked, a simple auks -a with --auks=done, and srun and sbatch can be executed on compute nodes, and /home is being mounted automatically. I couldn't achieve the same thing with sybil.
  - So the bottom line, does sybil offer the same functionality?

Regards.

[3XX0]

• Oh my bad, it is registered, it's just that --kerberos true is not valid. It must be yes, no, auto or force. You can check srun --help. You can also configure the default in the spank config so you don't have to type it everytime on the CLI.

• For simply forwarding tickets through Slurm like you described, you don't need to setup delegation or invoke sybil kinit. The Spank plugin will do everything and KCM will store/renew tickets on the KDC. You can set SYBIL_LOG=sybil=debug to see it getting invoked