"Illegal instruction (core dumped)" when generating keys

[spireblockchain]

Experiencing error on:

DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=24.04
DISTRIB_CODENAME=noble
DISTRIB_DESCRIPTION="Ubuntu 24.04.2 LTS" 

Client: Docker Engine - Community
 Version:           28.1.1
 API version:       1.49
 Go version:        go1.23.8
 Git commit:        4eba377
 Built:             Fri Apr 18 09:52:14 2025
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          28.1.1
  API version:      1.49 (minimum version 1.24)
  Go version:       go1.23.8
  Git commit:       01f442b
  Built:            Fri Apr 18 09:52:14 2025
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.7.27
  GitCommit:        05044ec0a9a75232cad458027ca83437aae3f4da
 runc:
  Version:          1.2.5
  GitCommit:        v1.2.5-0-g59923ef
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Steps to reproduce

1. Run midnight-shell.sh to launch and enter container
   ./midnight-shell.sh

2. Invoke "./midnight-node wizards generate-keys" and use the default prompts
   ./midnight-node wizards generate-keys

Result

root@90d97d7ee4da:/# ./midnight-node wizards generate-keys
This 🧙 wizard will generate the following keys and save them to your node's keystore:
→  an ECDSA Cross-chain key
→  an ED25519 Grandpa key
→  an SR25519 Aura key
It will also generate a network key for your node if needed.

> node base path ./data

⚙️ Generating Cross-chain (ecdsa) key
running external command: /midnight-node key generate --scheme ecdsa --output-type json
💾 Inserting Cross-chain (ecdsa) key
running external command: /midnight-node key insert --base-path ./data --scheme ecdsa --key-type crch --suri 'wrist measure divert reveal illness wonder panda way upper submit document capable'
command output: Illegal instruction (core dumped)
Running executable failed with status exit status: 132
Error: Application(Failed to run command

Stack backtrace:
   0: anyhow::error::<impl anyhow::Error>::msg
   1: <partner_chains_cli::io::DefaultCmdRunContext as partner_chains_cli::io::IOContext>::run_command
   2: partner_chains_cli::generate_keys::store_keys
   3: partner_chains_cli::generate_keys::generate_or_load_key
   4: <partner_chains_cli::generate_keys::GenerateKeysCmd as partner_chains_cli::CmdRun>::run
   5: midnight_node::command::run
   6: midnight_node::main
   7: std::sys::backtrace::__rust_begin_short_backtrace
   8: std::rt::lang_start::{{closure}}
   9: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at ./rustc/05f9846f893b09a1be1fc8560e33fc3c815cfecb/library/core/src/ops/function.rs:284:13
  10: std::panicking::try::do_call
             at ./rustc/05f9846f893b09a1be1fc8560e33fc3c815cfecb/library/std/src/panicking.rs:587:40
  11: std::panicking::try
             at ./rustc/05f9846f893b09a1be1fc8560e33fc3c815cfecb/library/std/src/panicking.rs:550:19
  12: std::panic::catch_unwind
             at ./rustc/05f9846f893b09a1be1fc8560e33fc3c815cfecb/library/std/src/panic.rs:358:14
  13: std::rt::lang_start_internal::{{closure}}
             at ./rustc/05f9846f893b09a1be1fc8560e33fc3c815cfecb/library/std/src/rt.rs:168:24
  14: std::panicking::try::do_call
             at ./rustc/05f9846f893b09a1be1fc8560e33fc3c815cfecb/library/std/src/panicking.rs:587:40
  15: std::panicking::try
             at ./rustc/05f9846f893b09a1be1fc8560e33fc3c815cfecb/library/std/src/panicking.rs:550:19
  16: std::panic::catch_unwind
             at ./rustc/05f9846f893b09a1be1fc8560e33fc3c815cfecb/library/std/src/panic.rs:358:14
  17: std::rt::lang_start_internal
             at ./rustc/05f9846f893b09a1be1fc8560e33fc3c815cfecb/library/std/src/rt.rs:164:5
  18: main
  19: __libc_start_call_main
             at ./csu/../sysdeps/nptl/libc_start_call_main.h:58:16
  20: __libc_start_main_impl
             at ./csu/../csu/libc-start.c:360:3
  21: _start)

Additional tests

root@90d97d7ee4da:/# /midnight-node key generate --scheme ecdsa --output-type json
{
  "secretPhrase": "legend talent wing pause just tape uphold fever early best dial lecture",
  "networkId": "substrate",
  "secretSeed": "0xa62e606b75d81de8d1ee76d264d5d20502244b4040a5e40bf11e7561269e3aa0",
  "publicKey": "0x03d273a5d87804d55069a0a63e315a35304eaec8a0be5ddc8c766ed1ddd02efeaa",
  "ss58PublicKey": "KWDUFQmcRKPT6gY8anHWwoY7MiySQZvGTM9xvJu1PxLXFSEmk",
  "accountId": "0xc77694202a81242cc68927f62d46bb774bdc90a3b91cfb5f69d272a0503fe770",
  "ss58Address": "5GaEcR5fsoF89pWbW6KTCk1hsKXGCzGZXsLBLZSDQ67G4XtV"
}
root@90d97d7ee4da:/# /midnight-node key insert --base-path ./data --scheme ecdsa --key-type crch --suri 'wrist measure divert reveal illness wonder panda way upper submit document capable'
Illegal instruction (core dumped)

root@90d97d7ee4da:/# ls -al ./data
total 8
drwxr-xr-x 2 1000 1000 4096 May 22 16:40 .
drwxr-xr-x 1 root root 4096 May 22 16:39 ..
root@90d97d7ee4da:/# touch ./data/test-file
root@90d97d7ee4da:/# ls -al ./data
total 8
drwxr-xr-x 2 1000 1000 4096 May 22 16:40 .
drwxr-xr-x 1 root root 4096 May 22 16:39 ..
-rw-r--r-- 1 root root    0 May 22 16:40 test-file

[gilescope]

What specific CPU chip are you running? Just wondering if it's trying to use an instruction that your chip doesn't have... (if so we can be more conservative on instruction levels)

[spireblockchain]

Thanks. Can upgrade if needed, but this server runs other services/apps without issue.

HP DL320 G9 with:
56 x Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz (2 Sockets)

Running Proxmox with kernel version:
Linux 6.8.12-5-pve

VM instance shows:

cat /proc/cpuinfo  | grep 'name'| uniq
model name      : QEMU Virtual CPU version 2.5+

[haiphamhoang]

I have the same error. I run on debian 13.

root@1c95e081ccb9:/# ./midnight-node wizards generate-keys
This 🧙 wizard will generate the following keys and save them to your node's keystore:
→  an ECDSA Cross-chain key
→  an ED25519 Grandpa key
→  an SR25519 Aura key
It will also generate a network key for your node if needed.

🛠️ Loaded node base path from config (pc-resources-config.json): ./data

⚙️ Generating Cross-chain (ecdsa) key
running external command: /midnight-node key generate --scheme ecdsa --output-type json
💾 Inserting Cross-chain (ecdsa) key
running external command: /midnight-node key insert --base-path ./data --scheme ecdsa --key-type crch --suri 'swear approve vast battle cart govern museum amazing speed gain couple promote'
command output: Illegal instruction (core dumped)
Running executable failed with status exit status: 132
Error: Application(Failed to run command

Stack backtrace:
   0: anyhow::error::<impl anyhow::Error>::msg
   1: <partner_chains_cli::io::DefaultCmdRunContext as partner_chains_cli::io::IOContext>::run_command
   2: partner_chains_cli::generate_keys::store_keys
   3: partner_chains_cli::generate_keys::generate_or_load_key
   4: <partner_chains_cli::generate_keys::GenerateKeysCmd as partner_chains_cli::CmdRun>::run
   5: midnight_node::command::run
   6: midnight_node::main
   7: std::sys::backtrace::__rust_begin_short_backtrace
   8: std::rt::lang_start::{{closure}}
   9: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at ./rustc/05f9846f893b09a1be1fc8560e33fc3c815cfecb/library/core/src/ops/function.rs:284:13
  10: std::panicking::try::do_call
             at ./rustc/05f9846f893b09a1be1fc8560e33fc3c815cfecb/library/std/src/panicking.rs:587:40
  11: std::panicking::try
             at ./rustc/05f9846f893b09a1be1fc8560e33fc3c815cfecb/library/std/src/panicking.rs:550:19
  12: std::panic::catch_unwind
             at ./rustc/05f9846f893b09a1be1fc8560e33fc3c815cfecb/library/std/src/panic.rs:358:14
  13: std::rt::lang_start_internal::{{closure}}
             at ./rustc/05f9846f893b09a1be1fc8560e33fc3c815cfecb/library/std/src/rt.rs:168:24
  14: std::panicking::try::do_call
             at ./rustc/05f9846f893b09a1be1fc8560e33fc3c815cfecb/library/std/src/panicking.rs:587:40
  15: std::panicking::try
             at ./rustc/05f9846f893b09a1be1fc8560e33fc3c815cfecb/library/std/src/panicking.rs:550:19
  16: std::panic::catch_unwind
             at ./rustc/05f9846f893b09a1be1fc8560e33fc3c815cfecb/library/std/src/panic.rs:358:14
  17: std::rt::lang_start_internal
             at ./rustc/05f9846f893b09a1be1fc8560e33fc3c815cfecb/library/std/src/rt.rs:164:5
  18: main
  19: __libc_start_call_main
             at ./csu/../sysdeps/nptl/libc_start_call_main.h:58:16
  20: __libc_start_main_impl
             at ./csu/../csu/libc-start.c:360:3
  21: _start)

Cpu info:

root@1c95e081ccb9:/# lscpu
Architecture:                x86_64
  CPU op-mode(s):            32-bit, 64-bit
  Address sizes:             46 bits physical, 48 bits virtual
  Byte Order:                Little Endian
CPU(s):                      32
  On-line CPU(s) list:       0-31
Vendor ID:                   GenuineIntel
  Model name:                Intel(R) Xeon(R) CPU E5-2698 v3 @ 2.30GHz
    CPU family:              6
    Model:                   63
    Thread(s) per core:      2
    Core(s) per socket:      16