From e781bc839a1e5b33153e6646e590f884a010f129 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Wed, 21 Jan 2026 19:43:34 +0000 Subject: [PATCH 1/3] Initial plan From a7fa0812258514da4f55aa2a4b74a82fcfcde8b6 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Wed, 21 Jan 2026 19:50:20 +0000 Subject: [PATCH 2/3] Add ROADMAP.md, CODE_STYLE.md and update README for accuracy Co-authored-by: omercsbn <88591959+omercsbn@users.noreply.github.com> --- CODE_STYLE.md | 550 ++++++++++++++++++++++++++++++++++++ README.md | 142 +++++----- ROADMAP.md | 764 ++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 1390 insertions(+), 66 deletions(-) create mode 100644 CODE_STYLE.md create mode 100644 ROADMAP.md diff --git a/CODE_STYLE.md b/CODE_STYLE.md new file mode 100644 index 0000000..be8c52a --- /dev/null +++ b/CODE_STYLE.md @@ -0,0 +1,550 @@ +# CLKernel Code Style Guide + +This document defines the coding standards and best practices for the CLKernel project. + +## Table of Contents + +1. [General Principles](#general-principles) +2. [C Language Standards](#c-language-standards) +3. [Formatting](#formatting) +4. [Naming Conventions](#naming-conventions) +5. [Comments and Documentation](#comments-and-documentation) +6. [Header Files](#header-files) +7. [Error Handling](#error-handling) +8. [Memory Management](#memory-management) +9. [Assembly Code](#assembly-code) +10. [Testing Guidelines](#testing-guidelines) + +--- + +## General Principles + +1. **Clarity over cleverness**: Write code that is easy to understand +2. **Minimal changes**: When modifying code, change only what's necessary +3. **Consistency**: Follow existing patterns in the codebase +4. **Safety first**: Always validate inputs and handle errors +5. **Document the "why"**: Code shows what, comments explain why + +--- + +## C Language Standards + +### Standard Version +- Use **C99** (ISO/IEC 9899:1999) as the base standard +- GCC extensions are permitted where they provide significant benefits: + - `__attribute__((packed))` for hardware structures + - `__attribute__((aligned(N)))` for alignment requirements + - Inline assembly with proper constraints + +### Required Compiler Flags +```makefile +CFLAGS = -std=gnu99 -ffreestanding -O2 -Wall -Wextra -Wpedantic \ + -nostdlib -nostartfiles -nodefaultlibs \ + -fno-builtin -fno-stack-protector \ + -mno-red-zone -mno-mmx -mno-sse -mno-sse2 +``` + +### Forbidden Constructs +- Variable-length arrays (VLAs) +- `goto` except for error cleanup patterns +- Floating-point operations in core kernel code +- Dynamic memory allocation in interrupt handlers + +--- + +## Formatting + +### Indentation +- Use **4 spaces** for indentation +- Never use tabs +- Configure your editor to convert tabs to spaces + +### Line Length +- Maximum **100 characters** per line +- Break long lines at logical points + +### Braces +- Use **Allman style** (braces on their own lines) for functions +- Use **K&R style** (opening brace on same line) for control structures + +```c +// Function definition - Allman style +void function_name(int parameter) +{ + // Function body +} + +// Control structures - K&R style +if (condition) { + // Code +} else { + // Code +} + +for (int i = 0; i < count; i++) { + // Code +} + +while (running) { + // Code +} +``` + +### Spacing + +#### Around Operators +```c +// Good +int result = a + b * c; +if (x == y && z != 0) { + value = (a < b) ? a : b; +} + +// Bad +int result=a+b*c; +if(x==y&&z!=0){ + value=(a +#include +#include +``` + +### Header Content +Headers should contain only: +- Type definitions +- Function declarations +- Macro definitions +- Extern variable declarations + +### Self-Contained Headers +Each header should be self-contained and include all dependencies: + +```c +// memory.h +#ifndef MEMORY_H +#define MEMORY_H + +#include +#include +#include + +// Now we can use uint32_t, bool, size_t +typedef struct { + uint32_t base; + size_t size; + bool available; +} memory_region_t; + +#endif // MEMORY_H +``` + +--- + +## Error Handling + +### Return Values +- Return `bool` for success/failure operations +- Return `NULL` for failed pointer operations +- Return negative values or error codes for detailed errors + +```c +// Simple success/failure +bool scheduler_add_actor(actor_t* actor); + +// Pointer return - NULL on failure +void* kmalloc(size_t size); + +// Error code return +typedef enum { + ERR_SUCCESS = 0, + ERR_NOMEM = -1, + ERR_INVALID = -2, + ERR_BUSY = -3 +} error_t; + +error_t file_open(const char* path, file_t** file); +``` + +### Error Checking +Always check return values: + +```c +// Good +void* buffer = kmalloc(1024); +if (!buffer) { + kprintf("[ERROR] Failed to allocate buffer\n"); + return false; +} + +// Bad +void* buffer = kmalloc(1024); +// Using buffer without checking... +``` + +### Assertions +Use `ASSERT` for conditions that should never be false: + +```c +void actor_send_message(actor_t* actor, message_t* msg) { + ASSERT(actor != NULL); + ASSERT(msg != NULL); + ASSERT(actor->state != ACTOR_STATE_TERMINATED); + + // Implementation +} +``` + +--- + +## Memory Management + +### Allocation Patterns +- Always check allocation results +- Free memory in reverse order of allocation +- Clear pointers after freeing + +```c +bool initialize_resources(void) { + resource_a = kmalloc(SIZE_A); + if (!resource_a) goto fail_a; + + resource_b = kmalloc(SIZE_B); + if (!resource_b) goto fail_b; + + resource_c = kmalloc(SIZE_C); + if (!resource_c) goto fail_c; + + return true; + +fail_c: + kfree(resource_b); +fail_b: + kfree(resource_a); +fail_a: + return false; +} +``` + +### Memory Barriers +Use appropriate barriers for multi-core safety: + +```c +// Full memory barrier +memory_barrier_full(); + +// Compiler barrier only +asm volatile("" ::: "memory"); +``` + +--- + +## Assembly Code + +### Inline Assembly +Use proper GCC extended inline assembly: + +```c +static inline uint32_t read_cr3(void) { + uint32_t value; + asm volatile("mov %%cr3, %0" : "=r"(value)); + return value; +} + +static inline void write_cr3(uint32_t value) { + asm volatile("mov %0, %%cr3" :: "r"(value) : "memory"); +} +``` + +### Assembly Files +- Use NASM syntax +- Comment thoroughly +- Use macros for repetitive code + +```nasm +; ============================================================================= +; Function: interrupt_handler +; Purpose: Common interrupt handler entry point +; ============================================================================= + +%macro ISR_NOERRCODE 1 +global isr%1 +isr%1: + cli ; Disable interrupts + push 0 ; Push dummy error code + push %1 ; Push interrupt number + jmp isr_common_stub ; Jump to common handler +%endmacro +``` + +--- + +## Testing Guidelines + +### Test Function Naming +```c +void test_memory_alloc_basic(void); +void test_memory_alloc_zero_size(void); +void test_memory_free_null(void); +``` + +### Test Structure +```c +void test_actor_creation(void) { + // Arrange + uint32_t expected_id = 1; + + // Act + uint32_t actor_id = actor_create(test_entry, NULL, PRIORITY_NORMAL, 0); + + // Assert + ASSERT_NE(actor_id, 0); + + // Cleanup + actor_terminate(actor_id); +} +``` + +### Test Coverage Goals +- All public functions should have tests +- Error paths should be tested +- Edge cases (NULL, zero, max values) should be tested + +--- + +## Version Control + +### Commit Messages +- Use imperative mood: "Add feature" not "Added feature" +- First line: 50 characters max, summary +- Body: Explain what and why (not how) + +``` +Add memory pool for actor message allocation + +Messages were being allocated from the main heap, causing +fragmentation. This commit adds a dedicated memory pool +for messages, improving allocation speed and reducing +fragmentation. +``` + +### Branch Naming +``` +feature/add-vfs-layer +bugfix/fix-page-fault-handling +refactor/cleanup-scheduler-code +``` + +--- + +*This style guide is a living document and may be updated as the project evolves.* diff --git a/README.md b/README.md index 1ac7746..7ae0a86 100644 --- a/README.md +++ b/README.md @@ -1,13 +1,13 @@ -# CLKernel Revolutionary OS +# CLKernel - Experimental OS Kernel -![CLKernel Logo](https://img.shields.io/badge/CLKernel-Revolutionary%20OS-blue?style=for-the-badge&logo=linux) -![Version](https://img.shields.io/badge/Version-1.0-green?style=for-the-badge) -![Status](https://img.shields.io/badge/Status-Production%20Ready-success?style=for-the-badge) +![CLKernel Logo](https://img.shields.io/badge/CLKernel-Experimental%20OS-blue?style=for-the-badge&logo=linux) +![Version](https://img.shields.io/badge/Version-0.1.0-yellow?style=for-the-badge) +![Status](https://img.shields.io/badge/Status-In%20Development-yellow?style=for-the-badge) ![Architecture](https://img.shields.io/badge/Architecture-x86%2032--bit-orange?style=for-the-badge) -## World's First Revolutionary Operating System Kernel +## Experimental Operating System Kernel with Async Actor Architecture -CLKernel represents a **groundbreaking advancement** in operating system design, featuring four revolutionary technologies that redefine what a modern kernel can achieve: +CLKernel is an **experimental research project** exploring modern operating system design concepts. It aims to implement four innovative technologies, though many features are still in development: ``` ▄████████ ▄█ ▄█ ▄█▄ ▄████████ ▄████████ ███▄▄▄▄ ▄████████ ▄█ @@ -20,44 +20,42 @@ CLKernel represents a **groundbreaking advancement** in operating system design, ████████▀ █████▄▄██ ███ ▀█▀ ██████████ ███ ███ ▀█ █▀ ██████████ █████▄▄██ ▀ ▀ ▀ ███ ███ ▀ - REVOLUTIONARY OPERATING SYSTEM v1.0 + EXPERIMENTAL OPERATING SYSTEM KERNEL v0.1.0 ``` --- -## Revolutionary Features - -### 1. AI Supervisor System -**World's first kernel with embedded machine learning capabilities** -- Real-time fault detection and prediction -- Intelligent system health monitoring -- Autonomous error recovery mechanisms -- Adaptive resource allocation optimization -- Machine learning-based performance tuning - -### 2. Hot-Swappable Module System -**True runtime modularity without system restart** -- Dynamic module loading/unloading at runtime -- **Timer Module**: Advanced scheduling and timing control -- **Logger Module**: Comprehensive system event logging -- **Diagnostic Module**: Real-time system analysis and profiling -- Zero-downtime subsystem replacement -- Module dependency management and versioning - -### 3. Sandboxing Engine -**Next-generation security with capability-based isolation** -- WASM-like process isolation technology -- Fine-grained capability-based permission system -- Secure execution environments for all processes -- Advanced memory protection and resource limiting -- Quantum-resistant security model - -### 4. Interactive Actor Shell -**Concurrent command processing with actor model** -- 45+ interactive system commands -- Actor-based parallel command execution -- Real-time system monitoring and control -- Advanced debugging and profiling tools +## Design Goals & Features + +> **Note**: Features marked with ⚠️ are partially implemented or in framework stage. +> Features marked with ❌ are planned but not yet implemented. + +### 1. AI Supervisor System ⚠️ +**Kernel-level behavioral monitoring with rule-based anomaly detection** +- ⚠️ Behavior pattern tracking and statistical analysis +- ⚠️ Anomaly detection (memory leaks, CPU spikes) +- ⚠️ Intervention framework (throttle, suspend, quarantine) +- ❌ Machine learning inference (framework only, no ML model yet) + +### 2. Hot-Swappable Module System ⚠️ +**Runtime modularity framework (partial implementation)** +- ⚠️ Module infrastructure and API design +- ❌ Dynamic module loading/unloading (stub implementation) +- ❌ Zero-downtime subsystem replacement +- ❌ Module dependency management + +### 3. Sandboxing Engine ❌ +**Planned capability-based isolation (not yet implemented)** +- ❌ WASM-like process isolation +- ❌ Capability-based permission system +- ❌ Memory protection and resource limiting + +### 4. Actor-Based Scheduler ⚠️ +**Cooperative multitasking with message passing** +- ✅ Actor creation and termination +- ⚠️ Asynchronous message passing +- ⚠️ Basic cooperative scheduling +- ❌ Interactive shell (not implemented) - Scriptable automation and workflow management ├─────────────────────────────────────────────────────────────┤ │ Natural Language CLI Layer │ @@ -165,30 +163,33 @@ CLKernel/ ## 🚀 Current Status -### ✅ Implemented +### ✅ Implemented (Basic/Working) - [x] MBR bootloader (512 bytes, switches to protected mode) - [x] Kernel entry point and C bridge - [x] VGA text mode display with printf support - [x] GDT setup for protected mode +- [x] IDT with interrupt/exception handlers +- [x] PIC initialization (8259) - [x] Build system with Makefile - [x] QEMU testing infrastructure -- [x] Modular project structure -### 🚧 In Progress -- [ ] IDT and interrupt handling -- [ ] Memory management (paging, heap) -- [ ] Async scheduler foundation -- [ ] Module system infrastructure - -### 📋 Planned Features -- [ ] Actor-based IPC system -- [ ] VFS with custom filesystem +### ⚠️ Partially Implemented +- [ ] Memory management (basic paging, bump allocator - needs improvement) +- [ ] Heap allocator (no proper free() support) +- [ ] Actor-based scheduler (framework exists, needs context switching) +- [ ] AI supervisor (rule-based anomaly detection, no ML inference) +- [ ] Module system (API defined, loading not implemented) +- [ ] Keyboard driver (reads scancodes, no translation) + +### ❌ Planned Features (Not Yet Started) +- [ ] Complete process model with user space +- [ ] System call interface +- [ ] Virtual filesystem (VFS) - [ ] Network stack -- [ ] AI supervisor integration - [ ] Live kernel patching - [ ] ARM64 support -- [ ] Natural language CLI -- [ ] Rust module integration +- [ ] Interactive shell +- [ ] Proper E820 memory detection ## 🧪 Testing @@ -298,13 +299,21 @@ Built for portability: This is an experimental research project. Contributions are welcome! -1. Focus on one subsystem at a time -2. Maintain the async-first design philosophy -3. Document architectural decisions -4. Test thoroughly in QEMU before submitting -5. Follow the existing code style +1. Read the [ROADMAP.md](./ROADMAP.md) to understand the project direction +2. Follow the [CODE_STYLE.md](./CODE_STYLE.md) guidelines +3. Focus on one subsystem at a time +4. Maintain the async-first design philosophy +5. Document architectural decisions +6. Test thoroughly in QEMU before submitting + +## 📖 Documentation + +- **[ROADMAP.md](./ROADMAP.md)** - Comprehensive development roadmap and gap analysis +- **[CODE_STYLE.md](./CODE_STYLE.md)** - Coding standards and best practices +- **[docs/ARCHITECTURE.md](./docs/ARCHITECTURE.md)** - System architecture details +- **[docs/MEMORY_SYSTEM.md](./docs/MEMORY_SYSTEM.md)** - Memory management documentation -## 📖 Learning Resources +## 📚 Learning Resources - **OS Development**: [OSDev Wiki](https://wiki.osdev.org/) - **x86 Assembly**: Intel Software Developer Manuals @@ -313,13 +322,14 @@ This is an experimental research project. Contributions are welcome! ## ⚠️ Current Limitations -- **Development Stage**: This is experimental research code -- **x86_64 Only**: ARM64 support is planned but not implemented +- **Development Stage**: This is experimental research code - not production ready +- **x86 32-bit Only**: ARM64 and x86-64 support are planned but not implemented - **Limited Hardware**: Currently supports basic VGA and keyboard only -- **No Filesystem**: VFS is stubbed out -- **No Networking**: Network stack is planned -- **AI Features Stubbed**: AI supervisor is not yet functional +- **No Filesystem**: VFS layer is not implemented +- **No Networking**: Network stack is not implemented +- **No User Space**: Processes cannot run in Ring 3 yet +- **AI Features Limited**: AI supervisor uses rule-based detection, no ML inference --- -**CLKernel v0.1.0** - Building the future of operating systems, one async actor at a time! 🚀 +**CLKernel v0.1.0** - An experimental OS kernel exploring async actor architecture 🚀 diff --git a/ROADMAP.md b/ROADMAP.md new file mode 100644 index 0000000..1085ee5 --- /dev/null +++ b/ROADMAP.md @@ -0,0 +1,764 @@ +# CLKernel Development Roadmap + +## 📋 Architecture Summary + +### Current Kernel Architecture + +CLKernel is designed as a hybrid kernel combining microkernel modularity with monolithic performance characteristics. The kernel targets x86 32-bit protected mode with plans for x86-64 and ARM64 support. + +``` +┌─────────────────────────────────────────────────────────────────────────────┐ +│ CLKernel Architecture │ +├─────────────────────────────────────────────────────────────────────────────┤ +│ │ +│ ┌─────────────────────────────────────────────────────────────────────┐ │ +│ │ User Space (Planned) │ │ +│ │ Applications │ Shell │ Services │ Modules │ │ +│ └─────────────────────────────────────────────────────────────────────┘ │ +│ ▼ │ +│ ┌─────────────────────────────────────────────────────────────────────┐ │ +│ │ System Call Interface (Planned) │ │ +│ └─────────────────────────────────────────────────────────────────────┘ │ +│ ▼ │ +│ ┌─────────────────────────────────────────────────────────────────────┐ │ +│ │ Kernel Services Layer │ │ +│ │ ┌───────────┐ ┌───────────┐ ┌───────────┐ ┌───────────────────────┐│ │ +│ │ │ VFS │ │ Network │ │ Actor │ │ AI Supervisor ││ │ +│ │ │ (Planned) │ │ (Planned) │ │ IPC ✓ │ │ ✓ ││ │ +│ │ └───────────┘ └───────────┘ └───────────┘ └───────────────────────┘│ │ +│ └─────────────────────────────────────────────────────────────────────┘ │ +│ ▼ │ +│ ┌─────────────────────────────────────────────────────────────────────┐ │ +│ │ Core Kernel Layer │ │ +│ │ ┌───────────┐ ┌───────────┐ ┌───────────┐ ┌───────────────────────┐│ │ +│ │ │ Scheduler │ │ Memory │ │ Module │ │ IDT/PIC ││ │ +│ │ │ ✓ │ │ ✓ │ │ System ✓ │ │ ✓ ││ │ +│ │ └───────────┘ └───────────┘ └───────────┘ └───────────────────────┘│ │ +│ │ ┌───────────┐ ┌───────────┐ ┌───────────┐ │ │ +│ │ │ Heap │ │ Paging │ │ GDT │ │ │ +│ │ │ ✓ │ │ ✓ │ │ ✓ │ │ │ +│ │ └───────────┘ └───────────┘ └───────────┘ │ │ +│ └─────────────────────────────────────────────────────────────────────┘ │ +│ ▼ │ +│ ┌─────────────────────────────────────────────────────────────────────┐ │ +│ │ Hardware Abstraction Layer │ │ +│ │ ┌───────────┐ ┌───────────┐ ┌───────────┐ ┌───────────────────────┐│ │ +│ │ │ VGA │ │ Keyboard │ │ Timer │ │ I/O Ports ││ │ +│ │ │ ✓ │ │ ✓ │ │ ✓ │ │ ✓ ││ │ +│ │ └───────────┘ └───────────┘ └───────────┘ └───────────────────────┘│ │ +│ └─────────────────────────────────────────────────────────────────────┘ │ +│ ▼ │ +│ ┌─────────────────────────────────────────────────────────────────────┐ │ +│ │ Bootloader (Assembly) │ │ +│ │ 512-byte MBR bootloader with protected mode switch │ │ +│ └─────────────────────────────────────────────────────────────────────┘ │ +│ │ +└─────────────────────────────────────────────────────────────────────────────┘ + +Legend: ✓ = Implemented (basic/partial) (Planned) = Not yet implemented +``` + +### Subsystem Details + +#### 1. Bootloader (`boot/boot.asm`) +- **Status**: ✅ Implemented +- **Features**: + - 512-byte MBR bootloader + - Real mode to protected mode switch + - GDT setup for protected mode + - Kernel loading from disk (sectors 2-9) + - Basic error handling + +#### 2. Kernel Core (`kernel/core/`) +- **Status**: ✅ Implemented (Basic) +- **Components**: + - `kernel_main.c` - Main entry point, initialization sequence + - `kernel_entry.asm` - Assembly bridge from bootloader + - `gdt.c` - Global Descriptor Table setup + - `idt.c` - Interrupt Descriptor Table with handlers + - `pic.c` - 8259 PIC initialization and IRQ management + +#### 3. VGA Display Driver (`kernel/core/vga.c`) +- **Status**: ✅ Implemented +- **Features**: + - VGA text mode (80x25) + - Color support + - Scrolling + - Cursor control + - printf-style formatting + +#### 4. Memory Management +- **Physical Memory** (`memory.c`): ✅ Basic implementation + - Memory region detection (simplified) + - Buddy allocator structure (partially implemented) + - Actor memory isolation (framework) +- **Virtual Memory** (`paging.c`): ✅ Basic implementation + - Page directory/table setup + - Identity mapping for kernel (first 4MB) + - Page mapping/unmapping +- **Heap Allocator** (`heap.c`): ✅ Basic implementation + - Simple bump allocator + - Slab allocator framework + - Actor-specific allocation tracking + +#### 5. Scheduler (`scheduler.c`) +- **Status**: ✅ Implemented (Basic) +- **Features**: + - Actor-based scheduling model + - Actor creation/termination + - Message passing (async) + - Cooperative multitasking framework + - AI supervision hooks + +#### 6. AI Supervisor (`ai_supervisor.c`) +- **Status**: ✅ Implemented (Framework) +- **Features**: + - Behavior pattern tracking + - Anomaly detection (memory leaks, CPU spikes) + - Intervention system (throttle, suspend, quarantine) + - Statistics collection +- **Limitations**: Actual ML inference not implemented + +#### 7. Module System (`modules.c`) +- **Status**: ⚠️ Stub Implementation +- **Framework exists but modules not loadable yet + +--- + +## 🔍 Gap Analysis + +### Critical Missing Components + +| Component | Status | Priority | Complexity | Description | +|-----------|--------|----------|------------|-------------| +| **Proper IDT Exception Handling** | ⚠️ Partial | P0 | Medium | ISR stubs exist but crash on most exceptions | +| **E820 Memory Detection** | ❌ Missing | P0 | Low | Currently assumes 32MB, needs BIOS memory map | +| **Complete Heap Implementation** | ⚠️ Partial | P0 | Medium | Current bump allocator doesn't support free() | +| **System Call Interface** | ❌ Missing | P1 | High | Required for user space | +| **Process/Thread Management** | ⚠️ Partial | P1 | High | Actors exist but no true processes | +| **Virtual Filesystem (VFS)** | ❌ Missing | P1 | Very High | Essential for file operations | +| **Keyboard Driver** | ⚠️ Partial | P2 | Low | Only reads scancodes, no translation | +| **Network Stack** | ❌ Missing | P2 | Very High | Required for networking | +| **User Space Support** | ❌ Missing | P1 | High | Ring 3 execution, TSS, user memory | + +### Detailed Gap Analysis + +#### 1. IDT & Interrupt Handling + +**Current State**: ISR assembly stubs exist, handlers implemented for exceptions 0-19 and IRQs 0-15. + +**Why Required**: Without proper exception handling, any CPU exception (divide by zero, page fault, etc.) will crash or hang the system. Interrupts are essential for: +- Timer-based preemptive scheduling +- Keyboard/device input +- Hardware event notification + +**Design Alternatives**: + +| Approach | Pros | Cons | +|----------|------|------| +| Monolithic handlers | Simple, fast | Less modular | +| Message-based (current) | Fits async model | Overhead for time-critical | +| Hybrid | Flexible | Complex implementation | + +**Pseudocode for Proper Exception Handler**: +```c +void exception_handler(interrupt_frame_t* frame) { + // 1. Disable further interrupts + cli(); + + // 2. Save complete CPU state + save_context(frame); + + // 3. Identify exception type + switch (frame->interrupt_number) { + case EXCEPTION_PAGE_FAULT: + // Try to handle page fault + if (handle_page_fault(get_cr2(), frame->error_code)) { + return; // Handled successfully + } + break; + case EXCEPTION_DIVIDE_ERROR: + // Signal current process + signal_process(current_process, SIGFPE); + return; + } + + // 4. If unrecoverable, panic + kernel_panic("Unhandled exception", frame); +} +``` + +#### 2. Memory Management (Paging, Heap) + +**Current State**: Basic paging with identity mapping; bump allocator for heap. + +**Why Required**: +- **Memory Protection**: Without proper paging, processes can access any memory +- **Virtual Memory**: Enables larger address spaces than physical RAM +- **Process Isolation**: Each process needs its own address space +- **Dynamic Allocation**: Heap required for kernel data structures + +**Design Alternatives**: + +| Technique | Use Case | Complexity | +|-----------|----------|------------| +| **Paging (4KB pages)** | Standard, good granularity | Medium | +| **Segmentation only** | Legacy x86, less isolation | Low | +| **Huge pages (2MB/1GB)** | Performance for large allocations | Medium | +| **Demand paging** | Memory efficiency | High | + +**Pseudocode for Page Fault Handler (Demand Paging)**: +```c +bool handle_page_fault(uint32_t fault_addr, uint32_t error_code) { + // 1. Check if address is in valid VMA + vma_t* vma = find_vma(current_process, fault_addr); + if (!vma) { + return false; // Invalid access - segfault + } + + // 2. Allocate physical page + page_t* page = alloc_page(); + if (!page) { + return false; // Out of memory + } + + // 3. Map the page + uint32_t flags = PAGE_PRESENT; + if (vma->flags & VMA_WRITE) flags |= PAGE_WRITABLE; + if (vma->flags & VMA_USER) flags |= PAGE_USER; + + map_page(fault_addr, page_to_phys(page), flags); + + // 4. Zero the page if anonymous + if (vma->type == VMA_ANONYMOUS) { + memset((void*)PAGE_ALIGN(fault_addr), 0, PAGE_SIZE); + } + + return true; +} +``` + +#### 3. Process Scheduler + +**Current State**: Actor-based cooperative scheduler with basic round-robin. + +**Why Required**: +- Fair CPU time distribution +- Priority-based execution +- Real-time task support +- Resource management + +**Design Alternatives**: + +| Scheduler Type | Pros | Cons | +|----------------|------|------| +| **Round-Robin** | Simple, fair | No priorities | +| **Priority-based** | Important tasks first | Starvation risk | +| **Completely Fair (CFS)** | Linux approach, balanced | Complex | +| **Cooperative** | Simpler, fits actors | Buggy actor can freeze | +| **Preemptive** | Robust | Context switch overhead | + +**Pseudocode for Priority Scheduler**: +```c +struct scheduler { + priority_queue_t ready_queue[MAX_PRIORITY]; + actor_t* current; + uint32_t time_slice; +}; + +actor_t* schedule_next(void) { + // Find highest priority non-empty queue + for (int p = PRIORITY_MAX; p >= PRIORITY_MIN; p--) { + if (!queue_empty(&scheduler.ready_queue[p])) { + actor_t* next = queue_dequeue(&scheduler.ready_queue[p]); + + // Calculate time slice based on priority + scheduler.time_slice = BASE_TIMESLICE * (p + 1); + + return next; + } + } + return idle_actor; // Run idle if nothing else +} + +void timer_interrupt_handler(void) { + scheduler.time_slice--; + + if (scheduler.time_slice == 0) { + // Preempt current actor + current->state = ACTOR_READY; + enqueue(&ready_queue[current->priority], current); + + // Switch to next + actor_t* next = schedule_next(); + context_switch(current, next); + } +} +``` + +#### 4. Actor-Based IPC System + +**Current State**: Message queue structure defined, send/receive implemented. + +**Why Required**: +- Inter-process/actor communication +- Event-driven architecture support +- Synchronization primitives + +**Design Alternatives**: + +| IPC Method | Latency | Throughput | Complexity | +|------------|---------|------------|------------| +| **Message Queues** | Medium | Medium | Low | +| **Shared Memory** | Very Low | Very High | Medium | +| **Pipes** | Low | High | Low | +| **Signals** | Very Low | Low | Low | +| **Actor Mailboxes** | Medium | Medium | Medium | + +**Pseudocode for Lock-Free Message Queue**: +```c +typedef struct message_queue { + message_t* head; + message_t* tail; + _Atomic uint32_t size; +} message_queue_t; + +bool message_enqueue(message_queue_t* queue, message_t* msg) { + msg->next = NULL; + + message_t* old_tail; + do { + old_tail = atomic_load(&queue->tail); + } while (!atomic_compare_exchange_weak(&queue->tail, &old_tail, msg)); + + if (old_tail) { + old_tail->next = msg; + } else { + atomic_store(&queue->head, msg); + } + + atomic_fetch_add(&queue->size, 1); + return true; +} + +message_t* message_dequeue(message_queue_t* queue) { + message_t* head; + do { + head = atomic_load(&queue->head); + if (!head) return NULL; + } while (!atomic_compare_exchange_weak(&queue->head, &head, head->next)); + + atomic_fetch_sub(&queue->size, 1); + return head; +} +``` + +#### 5. Virtual Filesystem (VFS) + +**Current State**: ❌ Not implemented + +**Why Required**: +- Unified interface to different filesystems +- Device abstraction +- File operations for processes +- Configuration storage + +**Design Alternatives**: + +| VFS Design | Complexity | Flexibility | +|------------|------------|-------------| +| **Unix-like VFS** | High | Very High | +| **Simple flat namespace** | Low | Low | +| **Plan 9 style (everything is a file)** | High | Very High | + +**Pseudocode for VFS Layer**: +```c +typedef struct vfs_operations { + int (*open)(struct inode*, struct file*); + int (*close)(struct file*); + ssize_t (*read)(struct file*, char*, size_t, off_t*); + ssize_t (*write)(struct file*, const char*, size_t, off_t*); + int (*readdir)(struct file*, struct dirent*); +} vfs_ops_t; + +typedef struct filesystem { + char name[32]; + vfs_ops_t* ops; + struct superblock* (*mount)(const char* device); +} filesystem_t; + +// Register filesystem +void register_filesystem(filesystem_t* fs) { + list_add(®istered_filesystems, fs); +} + +// Mount filesystem +int vfs_mount(const char* device, const char* mountpoint, const char* fstype) { + filesystem_t* fs = find_filesystem(fstype); + if (!fs) return -ENODEV; + + struct superblock* sb = fs->mount(device); + if (!sb) return -EIO; + + mount_entry_t* mnt = kmalloc(sizeof(mount_entry_t)); + mnt->mountpoint = strdup(mountpoint); + mnt->superblock = sb; + mnt->fs = fs; + + list_add(&mount_table, mnt); + return 0; +} +``` + +#### 6. Network Stack + +**Current State**: ❌ Not implemented + +**Why Required**: +- Internet connectivity +- Network services +- Modern OS functionality + +**Implementation Recommendation**: Start with lwIP (lightweight IP) or implement minimal UDP stack. + +**Pseudocode for Simple Packet Reception**: +```c +// Network interface structure +typedef struct net_interface { + uint8_t mac_address[6]; + uint32_t ip_address; + uint32_t netmask; + uint32_t gateway; + + int (*send_packet)(void* data, size_t len); + int (*receive_packet)(void* buffer, size_t max_len); +} net_if_t; + +// Ethernet frame handling +void handle_ethernet_frame(void* frame, size_t len) { + eth_header_t* eth = (eth_header_t*)frame; + + switch (ntohs(eth->ethertype)) { + case ETHERTYPE_ARP: + handle_arp(frame + sizeof(eth_header_t)); + break; + case ETHERTYPE_IP: + handle_ip(frame + sizeof(eth_header_t)); + break; + } +} +``` + +#### 7. Device Drivers + +**Current State**: VGA and keyboard (partial) implemented. + +**Needed Drivers**: +- Storage: ATA/AHCI, NVMe +- Input: Full keyboard with scancode translation +- Display: VESA/VBE for graphics mode +- Serial: For debugging +- RTC: Real-time clock + +#### 8. ARM64 Support + +**Current State**: ❌ Not implemented + +**Why Required**: +- Modern hardware support +- Mobile/embedded targets +- Architecture diversity + +**Implementation Steps**: +1. Create ARM64 bootloader (UEFI or bare-metal) +2. Implement ARM exception vectors +3. Port memory management (different page table format) +4. Port interrupt controller (GIC instead of PIC) +5. Abstract platform-specific code + +--- + +## 🤖 AI Supervisor Feasibility Analysis + +### Current Implementation Review + +The AI supervisor framework provides: +- Behavior pattern tracking with statistical analysis +- Anomaly detection algorithms (heuristic-based) +- Intervention mechanisms + +### Where AI Can Be Effectively Used + +| Application | Feasibility | Value | Implementation | +|-------------|-------------|-------|----------------| +| **Fault Prediction** | High | High | Pattern recognition on system metrics | +| **Scheduling Optimization** | Medium | High | Reinforcement learning for timeslice tuning | +| **Memory Leak Detection** | High | High | Statistical analysis of allocation patterns | +| **Resource Allocation** | Medium | Medium | Load balancing based on predictions | +| **Security Anomaly Detection** | Medium | Very High | Behavioral analysis for intrusion detection | + +### Recommended Frameworks for Kernel AI + +| Framework | Language | Size | Suitability | +|-----------|----------|------|-------------| +| **TensorFlow Lite Micro** | C++ | ~100KB | Good for microcontrollers | +| **CMSIS-NN** | C | ~50KB | ARM Cortex optimized | +| **uTensor** | C++ | ~30KB | Very lightweight | +| **Custom decision trees** | C | <10KB | Best for kernel space | +| **Rule-based systems** | C | <5KB | Simplest, most predictable | + +### Recommended Integration Strategy + +**Phase 1: Rule-Based System (Current)** +- Threshold-based anomaly detection +- Simple pattern matching +- No external dependencies + +**Phase 2: Statistical Models** +- Moving averages for trend detection +- Standard deviation for anomaly scoring +- K-means clustering (simple implementation) + +**Phase 3: Lightweight ML (Future)** +- Decision trees trained offline +- Run inference in kernel +- Export models as C arrays + +**Example: Decision Tree for Anomaly Classification** +```c +// Generated from trained model +typedef struct decision_node { + uint32_t feature_index; + float threshold; + int32_t left_child; // Index or -class_id if leaf + int32_t right_child; +} decision_node_t; + +// Compact model representation +static const decision_node_t anomaly_model[] = { + {0, 0.75f, 1, 2}, // Check memory_ratio + {1, 0.90f, -0, 3}, // Check cpu_ratio + {-1, 0, 0, 0}, // Anomaly class + {2, 100.0f, -0, -1}, // Check message_rate +}; + +int classify_anomaly(float* features) { + int node = 0; + while (node >= 0) { + decision_node_t* n = &anomaly_model[node]; + if (features[n->feature_index] <= n->threshold) { + node = n->left_child; + } else { + node = n->right_child; + } + } + return -node; // Return class ID +} +``` + +### AI Integration Considerations + +| Concern | Mitigation | +|---------|------------| +| **Latency** | Run AI inference in background, not in interrupt handlers | +| **Memory** | Pre-allocate fixed-size buffers, no dynamic allocation in AI code | +| **Determinism** | Use integer/fixed-point math, avoid floating point in critical paths | +| **Reliability** | AI should advise, not directly control critical functions | +| **Security** | Validate all AI outputs before acting | + +--- + +## 🎯 Prioritized Implementation Roadmap + +### Phase 1: Foundation Stabilization (Weeks 1-4) + +| Milestone | Tasks | Priority | Effort | +|-----------|-------|----------|--------| +| **M1.1** | Fix heap allocator (proper free list) | P0 | 3 days | +| **M1.2** | Implement E820 memory detection | P0 | 2 days | +| **M1.3** | Complete keyboard driver | P2 | 2 days | +| **M1.4** | Add comprehensive boot tests | P1 | 2 days | +| **M1.5** | Document current architecture | P2 | 1 day | + +### Phase 2: Core Kernel Features (Weeks 5-12) + +| Milestone | Tasks | Priority | Effort | +|-----------|-------|----------|--------| +| **M2.1** | Implement proper process model | P1 | 2 weeks | +| **M2.2** | Add preemptive scheduling | P1 | 1 week | +| **M2.3** | User space ring 3 support | P1 | 2 weeks | +| **M2.4** | System call interface | P1 | 1 week | +| **M2.5** | Complete actor IPC | P1 | 1 week | + +### Phase 3: Storage & Filesystem (Weeks 13-20) + +| Milestone | Tasks | Priority | Effort | +|-----------|-------|----------|--------| +| **M3.1** | ATA/IDE driver | P1 | 2 weeks | +| **M3.2** | VFS layer | P1 | 2 weeks | +| **M3.3** | RAM filesystem | P2 | 1 week | +| **M3.4** | FAT32 read support | P2 | 2 weeks | +| **M3.5** | Simple write support | P2 | 1 week | + +### Phase 4: Networking (Weeks 21-28) + +| Milestone | Tasks | Priority | Effort | +|-----------|-------|----------|--------| +| **M4.1** | NIC driver (e1000/virtio) | P2 | 2 weeks | +| **M4.2** | Ethernet layer | P2 | 1 week | +| **M4.3** | ARP implementation | P2 | 3 days | +| **M4.4** | IPv4 stack | P2 | 2 weeks | +| **M4.5** | UDP/TCP basics | P2 | 2 weeks | + +### Phase 5: Advanced Features (Weeks 29-40) + +| Milestone | Tasks | Priority | Effort | +|-----------|-------|----------|--------| +| **M5.1** | Enhanced AI supervisor | P3 | 3 weeks | +| **M5.2** | Hot module reloading | P3 | 2 weeks | +| **M5.3** | Shell with commands | P2 | 2 weeks | +| **M5.4** | Graphics mode support | P3 | 3 weeks | +| **M5.5** | ARM64 port | P3 | 8 weeks | + +--- + +## 🧪 Test Strategy + +### Unit Testing + +**Framework Recommendation**: Custom lightweight framework or µnit + +```c +// Example test structure +#define TEST(name) static void test_##name(void) +#define ASSERT_EQ(a, b) if ((a) != (b)) test_fail(__FILE__, __LINE__) + +TEST(heap_alloc_and_free) { + void* p1 = kmalloc(64); + ASSERT_NE(p1, NULL); + + void* p2 = kmalloc(128); + ASSERT_NE(p2, NULL); + ASSERT_NE(p1, p2); + + kfree(p1); + kfree(p2); +} +``` + +### Integration Testing with QEMU + +```bash +#!/bin/bash +# Run kernel in QEMU with timeout and capture output + +timeout 10 qemu-system-i386 \ + -drive file=build/clkernel.img,format=raw,if=floppy \ + -m 32M \ + -nographic \ + -serial stdio \ + 2>&1 | tee test_output.log + +# Check for expected output +grep -q "CLKernel initialization complete" test_output.log +``` + +### Suggested Test Categories + +1. **Boot Tests**: Kernel starts, subsystems initialize +2. **Memory Tests**: Allocation, deallocation, no leaks +3. **Interrupt Tests**: Timer fires, keyboard responds +4. **Scheduler Tests**: Actor creation, message passing +5. **Stress Tests**: Many actors, memory pressure + +--- + +## 🔒 Security Considerations + +### Current Vulnerabilities + +| Issue | Severity | Mitigation | +|-------|----------|------------| +| No ASLR | Medium | Randomize kernel/heap addresses | +| No stack canaries | High | Add `-fstack-protector` | +| No SMEP/SMAP | High | Enable CPU protections when available | +| Flat memory model | Critical | Implement proper paging with NX | + +### Recommended Security Features + +1. **Stack Protection** + - Enable compiler stack canaries + - Guard pages at stack boundaries + +2. **Memory Protection** + - NX (No-Execute) bit for data pages + - W^X policy (write XOR execute) + - ASLR for user space + +3. **Capability-Based Isolation** + - Actors have explicit capabilities + - No ambient authority + - Principle of least privilege + +4. **Sandboxing Engine** (Planned) + - WebAssembly-style isolation + - Memory bounds checking + - Syscall filtering + +--- + +## 📊 Performance Considerations + +### Actor Model vs Traditional IPC + +| Metric | Actor Model | Traditional Threads | +|--------|-------------|---------------------| +| Context switch | Fast (no TLB flush if shared) | Slower (full TLB flush) | +| Communication | Message copy overhead | Shared memory faster | +| Scalability | Excellent | Good | +| Complexity | Higher | Lower | +| Debugging | Harder | Easier | + +### Optimization Opportunities + +1. **Message Passing** + - Zero-copy for large messages (page remapping) + - Small message optimization (inline payload) + - Batched message delivery + +2. **Scheduling** + - Cache-aware actor placement + - Work stealing between CPU cores + - Priority inheritance for IPC + +3. **Memory** + - Page coloring for cache optimization + - Huge pages for frequently accessed regions + - Lazy allocation with demand paging + +--- + +## 📝 Code Style Guidelines + +See [CODE_STYLE.md](./CODE_STYLE.md) for detailed guidelines. + +### Quick Summary + +- **Language**: C99 with GCC extensions as needed +- **Indentation**: 4 spaces (no tabs) +- **Naming**: `snake_case` for functions/variables, `UPPER_CASE` for macros +- **Comments**: Doxygen style for public APIs +- **Headers**: Include guards with `#ifndef HEADER_H` + +--- + +## 📚 References + +- [OSDev Wiki](https://wiki.osdev.org/) - Comprehensive OS development resource +- [Intel Software Developer Manuals](https://www.intel.com/sdm) - x86 architecture reference +- [James Molloy's Kernel Tutorials](http://www.jamesmolloy.co.uk/tutorial_html/) - Classic x86 kernel tutorial +- [Writing a Simple Operating System from Scratch](https://www.cs.bham.ac.uk/~exr/lectures/opsys/10_11/lectures/os-dev.pdf) - Nick Blundell's guide + +--- + +*This roadmap is a living document and should be updated as development progresses.* From ffa734bc19d2ffd10b44c741634065a838f36463 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Wed, 21 Jan 2026 19:51:35 +0000 Subject: [PATCH 3/3] Fix code review findings in ROADMAP.md and CODE_STYLE.md Co-authored-by: omercsbn <88591959+omercsbn@users.noreply.github.com> --- CODE_STYLE.md | 6 +++++- ROADMAP.md | 2 +- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/CODE_STYLE.md b/CODE_STYLE.md index be8c52a..edfde01 100644 --- a/CODE_STYLE.md +++ b/CODE_STYLE.md @@ -124,7 +124,11 @@ int* array; void* ptr; // When declaring multiple pointers (avoid if possible) -int *a, *b, *c; // Each needs its own asterisk +int* a; +int* b; +int* c; +// Or if on one line (each needs its own asterisk): +// int *a, *b, *c; ``` ### Blank Lines diff --git a/ROADMAP.md b/ROADMAP.md index 1085ee5..5058534 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -121,7 +121,7 @@ Legend: ✓ = Implemented (basic/partial) (Planned) = Not yet implemented #### 7. Module System (`modules.c`) - **Status**: ⚠️ Stub Implementation -- **Framework exists but modules not loadable yet +- **Note**: Framework exists but modules not loadable yet. ---