ngbuddy does not support the concurrent startup of multiple jails.

[zhourainsun]

I use AppJail. After the system restarts, AppJail will simultaneously launch multiple jails. I found that only one of the two jails was successfully started. Upon reviewing the logs, it was discovered that the problem lies in the "link[NUM]" section where the data retrieval is incorrect.

Below is the system log.

Rename datanode7 to datanode9 instead of datanode7.

Dec 22 02:59:57 hadoop-freebsd-test13 kernel: datanode7: link state changed to DOWN
Dec 22 03:00:30 hadoop-freebsd-test13 kernel: datanode9: link state changed to DOWN
Dec 22 03:01:39 hadoop-freebsd-test13 kernel: ngeth1: Ethernet address: 58:9c:fc:10:7a:fe
Dec 22 03:01:39 hadoop-freebsd-test13 kernel: ngeth1: link state changed to UP
Dec 22 03:01:39 hadoop-freebsd-test13 kernel: ngeth2: Ethernet address: 58:9c:fc:10:69:75
Dec 22 03:01:39 hadoop-freebsd-test13 kernel: ngeth1: changing name to 'datanode7'
Dec 22 03:01:39 hadoop-freebsd-test13 kernel: datanode7: changing name to 'datanode9'
Dec 22 03:01:39 hadoop-freebsd-test13 kernel: lo0: link state changed to UP

jail.conf is as follows

[root@hadoop-freebsd-test13 roothome]$ cat /usr/local/appjail/jails/hdfs-datanode9/conf/jail.conf
hdfs-datanode9 {
exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown jail";
mount.devfs;
persist;
vnet;
vnet.interface = "datanode9";
exec.prestart = "service ngbuddy jail datanode9 public";
exec.prestop = "service ngbuddy unjail datanode9 hdfs-datanode9";
exec.consolelog = "/var/log/appjail/jails/hdfs-datanode9/console/2025-12-22.log";
mount.fstab = "/usr/local/appjail/jails/hdfs-datanode9/conf/fstab";
host.hostname = "hdfs-datanode9.appjail";
path = "/usr/local/appjail/jails/hdfs-datanode9/jail";
}
[root@hadoop-freebsd-test13 roothome]$ cat /usr/local/appjail/jails/hdfs-datanode7/conf/jail.conf
hdfs-datanode7 {
exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown jail";
mount.devfs;
persist;
allow.raw_sockets;
devfs_ruleset = "4";
vnet;
vnet.interface = "datanode7";
exec.prestart = "service ngbuddy jail datanode7 public";
exec.prestop = "service ngbuddy unjail datanode7 hdfs-datanode7";
exec.consolelog = "/var/log/appjail/jails/hdfs-datanode7/console/2025-12-22.log";
mount.fstab = "/usr/local/appjail/jails/hdfs-datanode7/conf/fstab";
host.hostname = "hdfs-datanode7.appjail";
path = "/usr/local/appjail/jails/hdfs-datanode7/jail";

[bellhyve]

Thank you for this report. So we need atomicity in the object creation and tracking. I'll discuss this matter with some friends and colleagues in the FreeBSD world and figure out the best way to approach that aligns with the script's minimalism. A lockfile might be appropriate, or we could do something more sophisticated with link numbers, which don't need to be sequential.

Let me know if you have any ideas. I can't think of any other obvious workarounds since the interface creation can take a bit of time and thanks to your report I see a few ways they can bump into themselves. We also need to watch out for other netgraph object generators like vm-bhyve.

[zhourainsun]

Hello, I mainly tested the performance of ngbuddy when multiple jails are started concurrently. Since ngctl is atomic by itself, I first found the current maximum link number, and then let ngctl itself try to see which link number is occupied until it succeeds. The specific changes are as follows. The same approach was used for the "link number" in other places.

ngbuddy_create_eiface() {
ngbuddy_init
if_name="$1"
bridge="${2:-$(echo $ngbuddy_bridges | awk '{print $1}')}"
if ifconfig "$if_name" > /dev/null 2>&1; then
echo "$if_name" already exists.
return
fi
# Atomically claim a free link number on this bridge
i=$(ngctl list -l | awk '$1 ~ /^link/ { x = int(substr($1, 5)); if (x > y) y = x } END { print (y != "") ? y : 0 }')
while ! ngctl mkpeer "$bridge:" eiface "link${i}" ether >/dev/null 2>&1; do
i=$((i+1))
done
local ng_if="$(ngctl show -n "$bridge:link${i}" | awk '{print $2}')"
ngctl name "$ng_if:" "$if_name"
ifconfig "$ng_if" name "$if_name" up >/dev/null
ngbuddy_ifconfig_ether
}