Skip to content

Comparison table #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
miczyg1 merged 5 commits into from
Jul 3, 2025
Merged

Comparison table #13

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
13b54b3
openness_score/openness_score.py: Compare Dasharo vs proprietary score
PLangowski Jun 24, 2025
32db306
Add scripts/compare.sh
PLangowski Jun 25, 2025
8e8d1b3
README: Add score comparison instructions
PLangowski Jul 1, 2025
9f771e6
scripts/compare.sh: Add -a argument
PLangowski Jul 3, 2025
5d1767c
scripts/compare.sh: Do use shortcuts in board names
miczyg1 Jul 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 32 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,14 +24,18 @@ but also many more coreboot distributions like heads.
## Usage

```bash
usage: openness_score.py [-o OUTPUT] [-v] [-V] [file]
usage: openness_score.py [-c proprietary_file] [-p PLATFORM] [-o OUTPUT] [-v] [-m [-V] [file]

Calculate Dasharo Openness Score for firmware images

positional arguments:
file Firmware binary file to be parsed

options:
-c proprietary_file, --compare proprietary_file
Compare Dasharo and proprietary firmware scores and store result in a Markdown table. file should be the Dasharo binary and proprietary_file should be the proprietary firmware binary.
-p PLATFORM, --platform PLATFORM
Platform model to provide to the table when --compare is set.
-o OUTPUT, --output OUTPUT
Specifies the directory where to store the results
-a MICROARCH, --microarch MICROARCH
Expand Down Expand Up @@ -75,6 +79,33 @@ The utility will produce 3 files:
precise numbers and detailed classification of firmware image components
to closed-source, open-source, data and empty categories

Example with `--compare` flag:

```bash
./openness_score/openness_score.py msi_ms7d25_v1.1.4_ddr4.rom -c E7D25IMS.1M1 -p "MS-7D25"
```

Aside from the 3 files mentioned above, this will also produce `compare.md` - A
Markdown table containing the score comparison between the two binaries. If the
file already exists, the result will be appended as a single row.

The table contains the following metrics:

- `closed-source diff`
- `data size diff`
- `empty space diff`

Each metric is calculated using the formula:

```txt
(Dasharo <type> size - Proprietary <type> size) * 100 / Proprietary <type> size
```

`<type>` is replaced by `closed-source`, `data` or `empty space` accordingly.

You can use `scripts/compare.sh` to generate a comparison table for common
Dasharo platforms. For more information, see the script's help.

**The utility currently supports coreboot and pure UEFI images only.**

### Examples
Expand Down
96 changes: 82 additions & 14 deletions openness_score/openness_score.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,57 @@ def export_data(args, image):
image.export_charts(output_path)


def compare_scores(dasharo_image, proprietary_image):
"""Compares metrics between Dasharo and proprietary firmware.

:param dasharo_image: Dasharo image
:type dasharo_image: DasharoCorebootImage
:param proprietary_image: Proprietary firmware image
:type proprietary_image: DasharoCorebootImage

:return: Three metrics as percentages rounded to 1 decimal: Closed source difference, Data size difference, Empty space size difference
:rtype: float, float, float
"""
closed_source_diff = 100 * (dasharo_image.closed_code_size - proprietary_image.closed_code_size) / proprietary_image.closed_code_size
data_size_diff = 100 * (dasharo_image.data_size - proprietary_image.data_size) / proprietary_image.data_size
empty_diff = 100 * (dasharo_image.empty_size - proprietary_image.empty_size) / proprietary_image.empty_size
return round(closed_source_diff, 1), round(data_size_diff, 1), round(empty_diff, 1)


def export_compared_data(args, dasharo_image, proprietary_image):
"""Calls `openness_score.compare_scores` and saves the output to a Markdown file.

:param args: Program arguments
:type args: argparse.Namespace
:param dasharo_image: Dasharo image
:type dasharo_image: DasharoCorebootImage
:param proprietary_image: Proprietary firmware image
:type proprietary_image: DasharoCorebootImage
"""
output_path = Path.cwd()

if args.output is not None:
output_path = Path(args.output)
try:
output_path.mkdir(parents=True, exist_ok=True)
except PermissionError:
sys.exit('You do not have permission to write to %s' % args.output)

closed_source_diff, data_size_diff, empty_diff = compare_scores(dasharo_image, proprietary_image)
dasharo_name = Path(args.file).name
proprietary_name = Path(args.compare).name
output_file = output_path.joinpath('compare.md')
if Path(output_file).is_file():
with open(output_file, 'a') as md:
md.write("| %s | %s | %s | %s | %s | %s |\n" % (args.platform, dasharo_name, proprietary_name, closed_source_diff, data_size_diff, empty_diff))
else:
with open(output_file, 'w') as md:
md.write("# Openness score comparison table\n\n")
md.write("|Platform | Dasharo Firmware file | Proprietary Firmware file | closed-source diff [%] | data size diff [%] | empty space diff [%] |\n")
md.write("| --- | --- | --- | --- | --- | --- |\n")
md.write("| %s | %s | %s | %s | %s | %s |\n" % (args.platform, dasharo_name, proprietary_name, closed_source_diff, data_size_diff, empty_diff))


def OpennessScore():
"""Utility's entry point responsible for argument parsing and
creating firmware image class instances based on detected image format.
Expand All @@ -133,6 +184,10 @@ def OpennessScore():

parser.add_argument('file', help='Firmware binary file to be parsed',
nargs='?')
parser.add_argument('-c', '--compare', metavar='proprietary_file',
help='Compare Dasharo and proprietary firmware scores and store result in a Markdown table.' \
' file should be the Dasharo binary and proprietary_file should be the proprietary firmware binary.')
parser.add_argument('-p', '--platform', default='-', help='Platform model to provide to the table when --compare is set.')
parser.add_argument('-h', '--help', action='help', help=SUPPRESS)
parser.add_argument('-o', '--output', default='out/', help='\n'.join([
'Specifies the directory where to store the results']))
Expand All @@ -156,20 +211,33 @@ def OpennessScore():
parser.print_help(sys.stderr)
sys.exit(0)

fw_is_cbfs, fw_is_uefi = check_file(args.file)

if fw_is_cbfs:
print('\'%s\' detected as Dasharo image' % args.file)
print('\n\n\'%s\' Dasharo image statistics:' % args.file)
DasharoCbImg = DasharoCorebootImage(args.file, args.verbose, args.microarch)
print(DasharoCbImg)
export_data(args, DasharoCbImg)
elif fw_is_uefi:
print('\'%s\' detected as vendor image' % args.file)
print('\n\n\'%s\' vendor image statistics:' % args.file)
VendorImg = UEFIImage(args.file, args.verbose)
print(VendorImg)
export_data(args, VendorImg)
if args.compare:
fw_is_cbfs, _ = check_file(args.file)
if not fw_is_cbfs:
print("File must be Dasharo!")
sys.exit(1)
DasharoImage = DasharoCorebootImage(args.file, args.verbose)
fw_is_cbfs, fw_is_uefi = check_file(args.compare)
if fw_is_cbfs:
ProprietaryImage = DasharoCorebootImage(args.compare, args.verbose)
elif fw_is_uefi:
ProprietaryImage = UEFIImage(args.compare, args.verbose)
export_compared_data(args, DasharoImage, ProprietaryImage)
else:
fw_is_cbfs, fw_is_uefi = check_file(args.file)

if fw_is_cbfs:
print('\'%s\' detected as Dasharo image' % args.file)
print('\n\n\'%s\' Dasharo image statistics:' % args.file)
DasharoCbImg = DasharoCorebootImage(args.file, args.verbose, args.microarch)
print(DasharoCbImg)
export_data(args, DasharoCbImg)
elif fw_is_uefi:
print('\'%s\' detected as vendor image' % args.file)
print('\n\n\'%s\' vendor image statistics:' % args.file)
VendorImg = UEFIImage(args.file, args.verbose)
print(VendorImg)
export_data(args, VendorImg)


if __name__ == '__main__':
Expand Down
197 changes: 197 additions & 0 deletions scripts/compare.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,197 @@
#!/bin/bash

MINIO_CREDS="$1"
NC_CREDS="$2"
NO_DOWNLOAD="${3:-false}"

BINDIR="binaries"

usage() {
cat <<EOF
Compare Dasharo and proprietary firmware openness score

The script will download binaries to the 'binaries' directory and run openness_score.py with '--compare'.

Usage:
compare.sh MINIO_CREDS NOVACUSTOM_CREDS [no-download]

MINIO_CREDS - JSON with Dasharo MinIO credentials
NOVACUSTOM_CREDS - JSON with credentials to https://repo.palkeo.com/clevo-mirror
no-download - Do not download binaries (assume present in 'binaries' directory)

The JSONs should have the following structure:

{
"login": "<LOGIN>",
"password": "<PASSWORD>"
}

Replace <LOGIN> and <PASSWORD> with appropriate credentials.
EOF
}

download_protectli() {
mkdir -p "$BINDIR/protectli"
repo_path="/tmp/protectli-firmware-updater"
git clone https://github.com/protectli-root/protectli-firmware-updater.git "$repo_path"
cp "$repo_path/images/fw6_all_YKR6LV30.bin" "$repo_path/images/protectli_all_fw6_vault_kbl_v1.0.14.rom" \
"$repo_path/images/protectli_v1210_v0.9.3.rom" "$repo_path/images/v1210_JPL.2LAN.S4G.PCIE.6W.013.bin" \
"$repo_path/images/protectli_v1211_v0.9.3.rom" "$repo_path/images/v1211_JPL.2LAN.D8G.PCIE.6W.009.bin"\
"$repo_path/images/protectli_v1410_v0.9.3.rom" "$repo_path/images/v1410_JPL.4LAN.S8GB.PCIE.6W.007B.bin"\
"$repo_path/images/protectli_v1610_v0.9.3.rom" "$repo_path/images/v1610_JPL.6LAN.D16G.PCIE.007.bin" \
"$repo_path/images/protectli_vp2410_v1.1.1.rom" "$repo_path/images/vp2410_GLK4L280.bin" \
"$repo_path/images/protectli_vp2420_v1.2.1.rom" "$repo_path/images/vp2420_YELD4L13P.bin" \
"$repo_path/images/protectli_vp2430_v0.9.0.rom" "$repo_path/images/vp2430_PRALNDZ4L10.bin" \
"$repo_path/images/protectli_vp4600_v1.2.0.rom" "$repo_path/images/vp4630_v2_YW6L2318.bin" \
"$repo_path/images/protectli_vp6600_v0.9.2.rom" "$repo_path/images/vp6630_ADZ6L314.bin" \
"$BINDIR/protectli"
}

compare_protectli() {
./openness_score/openness_score.py "$BINDIR/protectli/protectli_all_fw6_vault_kbl_v1.0.14.rom" \
-c "$BINDIR/protectli/fw6_all_YKR6LV30.bin" -p "Protectli FW6" -a "sklkbl"
./openness_score/openness_score.py "$BINDIR/protectli/protectli_v1210_v0.9.3.rom" \
-c "$BINDIR/protectli/v1210_JPL.2LAN.S4G.PCIE.6W.013.bin" -p "Protectli V1210" -a "jsl"
./openness_score/openness_score.py "$BINDIR/protectli/protectli_v1211_v0.9.3.rom" \
-c "$BINDIR/protectli/v1211_JPL.2LAN.D8G.PCIE.6W.009.bin" -p "Protectli V1211" -a "jsl"
./openness_score/openness_score.py "$BINDIR/protectli/protectli_v1410_v0.9.3.rom" \
-c "$BINDIR/protectli/v1410_JPL.4LAN.S8GB.PCIE.6W.007B.bin" -p "Protectli V1410" -a "jsl"
./openness_score/openness_score.py "$BINDIR/protectli/protectli_v1610_v0.9.3.rom" \
-c "$BINDIR/protectli/v1610_JPL.6LAN.D16G.PCIE.007.bin" -p "Protectli V1610" -a "jsl"
./openness_score/openness_score.py "$BINDIR/protectli/protectli_vp2410_v1.1.1.rom" \
-c "$BINDIR/protectli/vp2410_GLK4L280.bin" -p "Protectli VP2410" -a "glk"
./openness_score/openness_score.py "$BINDIR/protectli/protectli_vp2420_v1.2.1.rom" \
-c "$BINDIR/protectli/vp2420_YELD4L13P.bin" -p "Protectli VP2420" -a "ehl"
./openness_score/openness_score.py "$BINDIR/protectli/protectli_vp2430_v0.9.0.rom" \
-c "$BINDIR/protectli/vp2430_PRALNDZ4L10.bin" -p "Protectli VP2430" -a "adl"
./openness_score/openness_score.py "$BINDIR/protectli/protectli_vp4600_v1.2.0.rom" \
-c "$BINDIR/protectli/vp4630_v2_YW6L2318.bin" -p "Protectli VP46XX" -a "cnl"
./openness_score/openness_score.py "$BINDIR/protectli/protectli_vp6600_v0.9.2.rom" \
-c "$BINDIR/protectli/vp6630_ADZ6L314.bin" -p "Protectli VP66XX" -a "adl"
}

download_msi() {
mkdir -p "$BINDIR/msi"

bios_url="https://download.msi.com/bos_exe/mb"
minio_bucket="dasharo-msi-uefi"

# Z690-A DDR4
wget -O "/tmp/7D25v1L.zip" "$bios_url/7D25v1L.zip"
unzip "/tmp/7D25v1L.zip" -d "/tmp"
cp "/tmp/7D25v1L/E7D25IMS.1L0" "$BINDIR/msi"

# Z690-A-WIFI
wget -O "/tmp/7D25vAL.zip" "$bios_url/7D25vAL.zip"
unzip "/tmp/7D25vAL.zip" -d "/tmp"
cp "/tmp/7D25vAL/E7D25IMS.AL0" "$BINDIR/msi"

# Z790-P DDR4
wget -O "/tmp/7E06v1F.zip" "$bios_url/7E06v1F.zip"
unzip "/tmp/7E06v1F.zip" -d "/tmp"
cp "/tmp/7E06v1F/E7E06IMS.1F0" "$BINDIR/msi"

# Z790-P WIFI
wget -O "/tmp/7E06vAH.zip" "$bios_url/7E06vAH.zip"
unzip "/tmp/7E06vAH.zip" -d "/tmp"
cp "/tmp/7E06vAH/E7E06IMS.AH0" "$BINDIR/msi"

mc get "openness-score/$minio_bucket/MS-7D25/v1.1.4/msi_ms7d25_v1.1.4_ddr4.rom" "$BINDIR/msi"
mc get "openness-score/$minio_bucket/MS-7D25/v1.1.4/msi_ms7d25_v1.1.4_ddr5.rom" "$BINDIR/msi"
mc get "openness-score/$minio_bucket/MS-7E06/v0.9.2/msi_ms7e06_v0.9.2_ddr4.rom" "$BINDIR/msi"
mc get "openness-score/$minio_bucket/MS-7E06/v0.9.2/msi_ms7e06_v0.9.2_ddr5.rom" "$BINDIR/msi"
}

compare_msi() {
./openness_score/openness_score.py "$BINDIR/msi/msi_ms7d25_v1.1.4_ddr4.rom" -c "$BINDIR/msi/E7D25IMS.1L0" -p "MSI MS-7D25 DDR4" -a "adl"
./openness_score/openness_score.py "$BINDIR/msi/msi_ms7d25_v1.1.4_ddr5.rom" -c "$BINDIR/msi/E7D25IMS.AL0" -p "MSI MS-7D25 DDR5" -a "adl"
./openness_score/openness_score.py "$BINDIR/msi/msi_ms7e06_v0.9.2_ddr4.rom" -c "$BINDIR/msi/E7E06IMS.1F0" -p "MSI MS-7E06 DDR4" -a "adl"
./openness_score/openness_score.py "$BINDIR/msi/msi_ms7e06_v0.9.2_ddr4.rom" -c "$BINDIR/msi/E7E06IMS.AH0" -p "MSI MS-7E06 DDR5" -a "adl"
}

download_novacustom() {
mkdir -p "$BINDIR/novacustom"
bios_url="https://repo.palkeo.com/clevo-mirror"
dasharo_url="https://dl.3mdeb.com/open-source-firmware/Dasharo"

wget_cmd="wget --user $NC_LOGIN --password $NC_PASSWORD"

# V54/56 MTL
$wget_cmd -O "/tmp/B10709.zip" "$bios_url/Vxx0TU/B10709.zip"
unzip "/tmp/B10709.zip" -d "/tmp"
cp "/tmp/B10709/V5xxTU(32M).09" "$BINDIR/novacustom"

# NV4x ADL
$wget_cmd -O "/tmp/B10703.zip" "$bios_url/NV4xPZ/B10703.zip"
unzip "/tmp/B10703.zip" -d "/tmp"
cp "/tmp/B10703/ALL/NV4xPZ(32M).03" "$BINDIR/novacustom"

# NV4x TGL
$wget_cmd -O "/tmp/B10705.zip" "$bios_url/NV4xMB_ME/B10705.zip"
unzip "/tmp/B10705.zip" -d "/tmp"
cp "/tmp/B10705/ALL/NV4XBX.05" "$BINDIR/novacustom"

# NS5x ADL
$wget_cmd -O "/tmp/B10709.zip" "$bios_url/NSxxPU/B10709.zip"
unzip "/tmp/B10709.zip" -d "/tmp"
cp "/tmp/B10709/ALL/NSx0PU(32M).09" "$BINDIR/novacustom"

# NS5x TGL
$wget_cmd -O "/tmp/B10716N.zip" "$bios_url/NS5xMU/B10716N.zip"
unzip "/tmp/B10716N.zip" -d "/tmp"
cp "/tmp/NS50_70MU.16N/ALL/68C1/NS50_70MU.16N" "$BINDIR/novacustom"

wget -O "$BINDIR/novacustom/novacustom_v54x_mtl_v0.9.0.rom" "$dasharo_url/novacustom_v54x_mtl/v0.9.0/novacustom_v54x_mtl_v0.9.0.rom"
wget -O "$BINDIR/novacustom/novacustom_v56x_mtl_v0.9.0.rom" "$dasharo_url/novacustom_v56x_mtl/v0.9.0/novacustom_v56x_mtl_v0.9.0.rom"
wget -O "$BINDIR/novacustom/novacustom_nv4x_adl_v1.7.2_full.rom" "$dasharo_url/novacustom_nv4x_adl/v1.7.2/novacustom_nv4x_adl_v1.7.2_full.rom"
wget -O "$BINDIR/novacustom/novacustom_nv4x_tgl_v1.5.2.rom" "$dasharo_url/novacustom_nv4x_tgl/v1.5.2/novacustom_nv4x_tgl_v1.5.2.rom"
wget -O "$BINDIR/novacustom/novacustom_ns5x_adl_v1.7.2.rom" "$dasharo_url/novacustom_ns5x_adl/v1.7.2/novacustom_ns5x_adl_v1.7.2.rom"
wget -O "$BINDIR/novacustom/novacustom_ns5x_tgl_v1.5.2.rom" "$dasharo_url/novacustom_ns5x_tgl/v1.5.2/novacustom_ns5x_tgl_v1.5.2.rom"
}

compare_novacustom() {
./openness_score/openness_score.py "$BINDIR/novacustom/novacustom_v54x_mtl_v0.9.0.rom" -c "$BINDIR/novacustom/V5xxTU(32M).09" -p "Novacustom V540TU" -a "mtl"
./openness_score/openness_score.py "$BINDIR/novacustom/novacustom_v56x_mtl_v0.9.0.rom" -c "$BINDIR/novacustom/V5xxTU(32M).09" -p "Novacustom V560TU" -a "mtl"
./openness_score/openness_score.py "$BINDIR/novacustom/novacustom_nv4x_adl_v1.7.2_full.rom" -c "$BINDIR/novacustom/NV4xPZ(32M).03" -p "Novacustom NV4xPZ" -a "adl"
./openness_score/openness_score.py "$BINDIR/novacustom/novacustom_nv4x_tgl_v1.5.2.rom" -c "$BINDIR/novacustom/NV4XBX.05" -p "Novacustom NV4xME_MB" -a "tgl"
./openness_score/openness_score.py "$BINDIR/novacustom/novacustom_ns5x_adl_v1.7.2.rom" -c "$BINDIR/novacustom/NSx0PU(32M).09" -p "Novacustom NSxxPU" -a "adl"
./openness_score/openness_score.py "$BINDIR/novacustom/novacustom_ns5x_tgl_v1.5.2.rom" -c "$BINDIR/novacustom/NS50_70MU.16N" -p "Novacustom NS5xMU" -a "tgl"
}

download_odroid() {
mkdir -p "$BINDIR/odroid"
bios_url="https://dn.odroid.com/ODROID-H4/bios"
minio_bucket="dasharo-odroid-h4-plus-uefi"
echo "Please download stock binaries from $bios_url to $BINDIR/odroid"
read -p "Press any key after downloading binaries"

mc get "openness-score/$minio_bucket/hardkernel_odroid_h4/v0.9.0/hardkernel_odroid_h4_v0.9.0.rom" "$BINDIR/odroid"
}

compare_odroid() {
./openness_score/openness_score.py "$BINDIR/odroid/hardkernel_odroid_h4_v0.9.0.rom" -c "$BINDIR/odroid/ADLN-H4_B1.07.bin" -p "Odroid H4" -a "adl"
}

if [ $# -lt 2 ]; then
usage
exit 1
fi

mkdir -p "$BINDIR"
MINIO_LOGIN=$(jq -r '.login' "$MINIO_CREDS")
MINIO_PASSWORD=$(jq -r '.password' "$MINIO_CREDS")
mc alias set openness-score "https://dl.dasharo.com" "$MINIO_LOGIN" "$MINIO_PASSWORD"

NC_LOGIN=$(jq -r '.login' "$NC_CREDS")
NC_PASSWORD=$(jq -r '.password' "$NC_CREDS")

if [ $NO_DOWNLOAD != "no-download" ]; then
download_msi
download_protectli
download_odroid
download_novacustom
fi
compare_msi
compare_protectli
compare_odroid
compare_novacustom
Loading