Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 46 additions & 7 deletions .github/workflows/zig.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,19 +7,55 @@ on:
branches: [ main, zig-port ]

jobs:
c-build:
c-build-gcc:
name: C Build (GCC)
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4

- name: Build C version
run: cd c && make clean && make
- name: Check GCC version
run: gcc --version

- name: Test C executable runs
run: cd c && echo -e "hi\npython\nexit" | timeout 5 ./chat || true
- name: Build with GCC (C17 and C23)
run: cd c && make clean && make gcc-builds

- name: Test GCC C17 executable
run: cd c && echo -e "hi\npython\nexit" | timeout 5 ./chat-gcc-c17 || true

- name: Test GCC C23 executable
run: cd c && echo -e "hi\npython\nexit" | timeout 5 ./chat-gcc-c23 || true

- name: Show binary sizes
run: ls -lh c/chat-gcc-*

c-build-clang:
name: C Build (Clang)
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4

- name: Install Clang
run: sudo apt-get update && sudo apt-get install -y clang

- name: Check Clang version
run: clang --version

- name: Build with Clang (C17 and C23)
run: cd c && make clean && make clang-builds

- name: Test Clang C17 executable
run: cd c && echo -e "hi\npython\nexit" | timeout 5 ./chat-clang-c17 || true

- name: Test Clang C23 executable
run: cd c && echo -e "hi\npython\nexit" | timeout 5 ./chat-clang-c23 || true

- name: Show binary sizes
run: ls -lh c/chat-clang-*

zig-build-and-test:
name: Zig Build & Test
runs-on: ubuntu-latest

steps:
Expand All @@ -30,10 +66,13 @@ jobs:
version: 0.15.2

- name: Build Zig chatbot
run: cd zig && mkdir -p zig-out/bin && zig build-exe src/main.zig -femit-bin=zig-out/bin/chat
run: cd zig && zig build

- name: Run Zig tests
run: cd zig && zig test src/chatbot.zig
run: cd zig && zig build test

- name: Test Zig executable runs
run: cd zig && echo -e "hi\npython\nexit" | timeout 5 ./zig-out/bin/chat || true

- name: Show binary size
run: ls -lh zig/zig-out/bin/chat
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -27,3 +27,7 @@ chat
# Zig build artifacts
.zig-cache/
zig-out/
c/chat-clang-c17
c/chat-clang-c23
c/chat-gcc-c17
c/chat-gcc-c23
100 changes: 73 additions & 27 deletions BENCHMARK.md
Original file line number Diff line number Diff line change
@@ -1,48 +1,94 @@
# Chatbot Build Benchmark Results
# Chatbot Build & Runtime Benchmark Results

Benchmark comparing C and Zig implementations of the chatbot.
Benchmark comparing C (GCC and Clang, C17 and C23 standards) and Zig implementations.

**System:** macOS arm64
**Date:** 2026-01-12

## Results
## Compiler Versions (Local)


| Compiler | Version |
| ----------- | -------------------------------------------- |
| GCC (macOS) | Apple Clang 17.0.0 (gcc is aliased to clang) |
| Clang | Homebrew Clang 21.1.8 |
| Zig | 0.15.2 |


## Runtime Results


| Version | Execution Time | Relative |
| --------- | -------------- | --------------- |
| GCC C17 | 228ms | 2.23x |
| GCC C23 | 102ms | 1.00x (fastest) |
| Clang C17 | 107ms | 1.04x |
| Clang C23 | 106ms | 1.03x |
| Zig | 628ms | 6.15x |


## Binary Sizes


| Version | Size |
| --------- | ---- |
| GCC C17 | 33K |
| GCC C23 | 33K |
| Clang C17 | 33K |
| Clang C23 | 33K |
| Zig | 1.3M |

| Metric | C | Zig | Difference |
|--------|---|-----|-----------|
| Build Time | 93ms | 710ms | Zig is 7.63x slower |
| Executable Size | 33K | 1.3M | Zig is 39x larger |

## Analysis

### Build Time
- **C (93ms)**: Fast compilation using GCC with minimal optimization
- **Zig (710ms)**: Longer compilation time due to Zig's more comprehensive compiler
### C Compilers

- **GCC C23** was fastest in this run (times vary between runs)
- **Clang** produces consistently fast binaries across C17/C23
- On macOS, `gcc` is actually Apple Clang; real GCC is tested on CI (Ubuntu)

### Zig

- Slower due to:
- GeneralPurposeAllocator overhead vs C's stack allocation
- Zig 0.15 buffered I/O system overhead
- Additional runtime safety checks
- Much larger binary (embeds stdlib, no libc dependency)

### Binary Size

The C version compiles significantly faster due to:
- Simpler compilation pipeline
- No build system overhead (direct gcc command)
- Minimal type checking and analysis
- All C versions: 33K (links against system libc)
- Zig: 1.3M (self-contained, no external dependencies)

### Executable Size
- **C (33K)**: Small, minimal runtime
- **Zig (1.3M)**: Larger due to Zig's standard library and runtime
## Build Configuration


| Compiler | Flags |
| --------- | ---------------------------------------- |
| GCC C17 | `gcc -std=c17 -Wall -Wextra -pedantic` |
| GCC C23 | `gcc -std=c23 -Wall -Wextra -pedantic` |
| Clang C17 | `clang -std=c17 -Wall -Wextra -pedantic` |
| Clang C23 | `clang -std=c23 -Wall -Wextra -pedantic` |
| Zig | `zig build` (debug mode) |

The Zig executable is larger because:
- Zig stdlib is embedded in the binary
- More comprehensive runtime features
- GeneralPurposeAllocator adds overhead

## Notes

- Both versions are unoptimized builds
- C build uses `-std=c11 -Wall -Wextra -pedantic`
- Zig build uses default optimization level
- Times are from cold builds (no cache)
- All versions produce identical output
- C23 falls back to `-std=c2x` on older compilers
- Times vary between runs; relative performance is more meaningful
- CI tests both GCC and Clang on Ubuntu

## Running the Benchmark
## Running the Benchmarks

```bash
# Full runtime benchmark (all compilers)
./run_benchmarks.sh

# Build-only benchmark
./benchmark.sh

# Show detected compilers
cd c && make info
```

This will clean, rebuild both versions, and compare build times and sizes.
57 changes: 54 additions & 3 deletions c/Makefile
Original file line number Diff line number Diff line change
@@ -1,5 +1,56 @@
all:
gcc -std=c11 -Wall -Wextra -pedantic src/chatbot.c -o chat
# Detect real GCC vs Apple's gcc alias
UNAME_S := $(shell uname -s)
ifeq ($(UNAME_S),Darwin)
# macOS: gcc is actually clang, use explicit paths if available
GCC := $(shell which gcc-14 gcc-13 gcc-12 2>/dev/null | head -1)
ifeq ($(GCC),)
GCC := gcc
GCC_NOTE := "(Apple Clang)"
endif
else
GCC := gcc
endif

CLANG := clang
CFLAGS_COMMON = -Wall -Wextra -pedantic
SRC = src/chatbot.c

# Default targets
all: gcc-builds clang-builds

# GCC builds
gcc-builds: chat-gcc-c17 chat-gcc-c23

chat-gcc-c17:
$(GCC) -std=c17 $(CFLAGS_COMMON) $(SRC) -o chat-gcc-c17

chat-gcc-c23:
$(GCC) -std=c23 $(CFLAGS_COMMON) $(SRC) -o chat-gcc-c23 2>/dev/null || \
$(GCC) -std=c2x $(CFLAGS_COMMON) $(SRC) -o chat-gcc-c23

# Clang builds
clang-builds: chat-clang-c17 chat-clang-c23

chat-clang-c17:
$(CLANG) -std=c17 $(CFLAGS_COMMON) $(SRC) -o chat-clang-c17

chat-clang-c23:
$(CLANG) -std=c23 $(CFLAGS_COMMON) $(SRC) -o chat-clang-c23 2>/dev/null || \
$(CLANG) -std=c2x $(CFLAGS_COMMON) $(SRC) -o chat-clang-c23

# Legacy targets for backwards compatibility
chat: chat-gcc-c17
cp chat-gcc-c17 chat

chat-c17: chat-gcc-c17
cp chat-gcc-c17 chat-c17

chat-c23: chat-gcc-c23
cp chat-gcc-c23 chat-c23

clean:
rm -rf *.o chat
rm -rf *.o chat chat-c17 chat-c23 chat-gcc-* chat-clang-*

info:
@echo "GCC: $(GCC) $(GCC_NOTE)"
@echo "Clang: $(CLANG)"
6 changes: 5 additions & 1 deletion c/src/chatbot.c
Original file line number Diff line number Diff line change
@@ -1,8 +1,12 @@
// Enable POSIX/GNU extensions (strdup, strncasecmp, etc.)
// _GNU_SOURCE is needed because -std=c17 disables extensions by default
#define _GNU_SOURCE

#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <ctype.h>
#include <limits.h>
#include <strings.h>
#include "chatbot.h"

// hash table implementation from here
Expand Down
Loading
Loading