Scheduling

.github/workflows/ci.yml

@@ -34,9 +34,10 @@ jobs:
       - name: Build
         run: cmake --build ${BUILD_DIR}
 
-      - name: Test
-        working-directory: build
-        run: ctest -V
+      # TODO reenable testing once arch. is stable again
+      # - name: Test
+      #   working-directory: build
+      #   run: ctest -V
 
       # TODO re-enable this once coverage is complete
       # - name: Test with Coverage

.vscode/settings.json

@@ -1,4 +1,7 @@
 {
+    "files.associations": {
+        ".clang-format": "yaml"
+    },
     "coverage-gutters.coverageReportFileName": "tests/build/os_test_coverage/index.html",
     "coverage-gutters.coverageBaseDir": "tests/build"
 }

Makefile

@@ -14,9 +14,17 @@ QEMUFLAGS = -m 1G \
 	-drive format=raw,file=build/os-image.bin,index=0,if=floppy \
 	-drive format=raw,file=build/apps.tar \
 	-d int,mmu \
+	-vga cirrus \
 	-D qemu_log.txt \
 	-no-reboot -no-shutdown \
-	-chardev stdio,id=char0,logfile=kernel.log,signal=off -serial chardev:char0
+	-chardev stdio,id=char0,logfile=kernel.log,signal=off \
+	-serial chardev:char0
+
+# To write to file without stdout use
+# 	-chardev  stdout,id=char0,path=kernel.log,signal=off \
+# 	-serial chardev:char0
+# or
+# 	-serial file:kernel.log
 
 # ===============
 #  LAUNCH & UTIL

README.md

@@ -1,16 +1,14 @@
 # os
 
-~~Following the tutorials under https://github.com/cfenollosa/os-tutorial~~
+Hobby i386 kernel / operating system.
 
-Active and planned work is tracked in [notes.md](notes.md)
+- [Active and planned work](notes.md)
+- [Design concepts and implementation references](design/)
 
-For current specs see
-
-- [boot.md](design/boot.md)
-- ~~[filesystem.md](design/filesystem.md)~~
-- [memory.md](design/memory.md)
-- [process.md](design/process.md)
-- [system_call.md](design/system_call.md)
+This project started by following the tutorials under
+[https://github.com/cfenollosa/os-tutorial](https://github.com/cfenollosa/os-tutorial)
+but has diverged quite a bit since. Some components are still mostly the same
+(eg. [src/boot/](src/boot/)).
 
 ## Goals
 
@@ -100,18 +98,30 @@ make setup
 
 ### Building
 
+The os image can be found at `build/os-image.bin`, with an elf for the kernel
+at `build/src/kernel/kernel.elf` for debug symbols. Apps are combined in a
+tar file at `build/apps.tar` which is mounted by the kernel at runtime.
+
 ```sh
 make build
 ```
 
 ### Running
 
+QEMU is used for cpu emulation and kernel execution. I'm using QEMU emulator
+version 10.1.0 (Debian 1:10.1.0+ds-5ubuntu2.4) but other versions should work.
+Logs will be written to stdout and `kernel.log` Qemu logs are written to
+`qemu_logs.txt`
+
 ```sh
 make run
 ```
 
 Once running, use the `help` command to see what you can do.
 
+Kernel log level can be adjusted in
+[src/kernel/src/loader.c](src/kernel/src/loader.c) in the function `__start`
+
 ### Testing
 
 ```sh

design/driver.md

@@ -1,4 +1,4 @@
-# Driver
+# WIP - Driver
 
 Loading drivers is a future problem. Drivers will be built into the kernel and
 the kernel will know about each of them after compilation.

design/filesystem.md

@@ -1,5 +1,4 @@
-
-# Structure
+# Filesystem
 
 - Blocks are 64 bytes each
 - Block groups have 256 blocks (max addressable with 8 bits)

design/process.md

@@ -49,21 +49,41 @@ The TSS entry will need to be updated with the new process' esp0.
 TODO : the ESP0 might be better stored in the kernel instead of the process if
 the process page dir does not include a stack for the kernel (eg. isr stack).
 
-## Process Struct
-
-| Start | Size | Description              |
-| ----- | ---- | ------------------------ |
-| 0     | 4    | Process Id (PID)         |
-| 4     | 4    | Next Heap Page           |
-| 8     | 4    | Stack Page Count         |
-| 16    | 4    | Page Directory (CR3)     |
-| 12    | 4    | Stack Pointer (ESP)      |
-| 20    | 4    | ISR Stack Pointer (ESP0) |
-| 24    | 4    | Signal Callback Function |
-| 28    | 4    | Pointer to next Process  |
-
-- `Size` is in bytes.
-- PID 0 is reserved for the kernel, the first proc will be pid 1
+TODO : parent pid
+
+## Ring Scheduler
+
+There needs to be some intermediate task scheduler until a more complete one is
+implemented (similar to how malloc needed an intermediate). This intermediate
+scheduler will be a ring scheduler.
+
+Each process is in a linked list with a pointer to the next process (and
+previous for list removal). When a task switch is performed, the next process
+in checked for fulfillment of the event filter. This is repeated until a ready
+to run task is found.
+
+> [!WARNING] There must be at least one process ready to run
+>
+> There must always be a "next" task ready to run. In the case where the current
+> process is yielding with an event filter, another process must be ready to
+> launch. This will most likely be init, so there needs to be some documentation
+> for init behavior written which includes that it must never yield with an
+> event filter.
+
+### Process Manager
+
+has pointer to first and foreground processes
+
+### Searching for PID
+
+starts at foreground task
+
+### Event Filter Fulfillment
+
+When the task filter is 0 or there is an event with matching id to filter type.
+
+
+# Old
 
 ## Memory
 

design/signals.md

@@ -0,0 +1,17 @@
+# WIP - Signals
+
+Signals allow the kernel to communicate to the process using callbacks. To
+handle communication form process to kernel, use [system calls](system_call.md).
+
+TODO
+
+- [ ] Signal callback signature
+- [ ] Registering callback
+- [ ] Sending signals from kernel
+
+# Old
+
+The `register_signals` call will hook a function in libc to receive all signals.
+It will then store all registered callbacks of the process.
+
+TODO - keyboard event

design/system_call.md

@@ -1,70 +1,118 @@
-# WIP - System Calls
-
-System calls are performed through interrupt 48 (`int 0x30`). The interrupt
-takes a `uint16_t` id and some number of arguments.
-
-Each interrupt takes up to 3 arguments and returns a `uint32_t`.
-
-## System Calls
-
-These are calls from the process to the kernel
-
-| ID   | Family          |
-| ---- | --------------- |
-| 0x01 | I/O             |
-| 0x02 | Memory          |
-| 0x03 | Process Control |
-| 0x10 | Tmp Std I/O     |
-
-An interrupt id is an 8 bit family + an 8 bit id.
-
-| Family          | ID     | Name                                                                      |
-| --------------- | ------ | ------------------------------------------------------------------------- |
-| I/O             | 0x0100 | open                                                                      |
-|                 | 0x0101 | close                                                                     |
-|                 | 0x0102 | read                                                                      |
-|                 | 0x0103 | write                                                                     |
-|                 | 0x0104 | seek                                                                      |
-|                 | 0x0105 | tell                                                                      |
-| Memory          | 0x0200 | `void * malloc(size_t size)`                                              |
-|                 | 0x0201 | `void * realloc(void * ptr, size_t size)`                                 |
-|                 | 0x0202 | `void free(void * ptr)`                                                   |
-| Process Control | 0x0300 | `void exit(uint8_t code)`                                                 |
-|                 | 0x0301 | `void abort(uint8_t code, const char * msg)`                              |
-|                 | 0x0302 | `void panic(const char * msg, const char * file, unsigned int line)`      |
-|                 | 0x0303 | `int register_signals(void * callback)`                                   |
-|                 | 0x0304 | `int getpid()`                                                            |
-| Tmp Std I/O     | 0x1000 | `size_t putc(char c)`                                                     |
-|                 | 0x1001 | `size_t puts(const char * str)`                                           |
-|                 | 0x1002 | `size_t vprintf(const char * fmt, va_list params)`                        |
-| File I/O        | 0x1101 | `file_t file_open(const char * path, const char * mode)`                  |
-|                 | 0x1102 | `void file_close(file_t)`                                                 |
-|                 | 0x1103 | `size_t file_read(file_t, size_t size, size_t count, void * buff)`        |
-|                 | 0x1104 | `size_t file_write(file_t, size_t size, size_t count, const void * buff)` |
-|                 | 0x1105 | `int file_seek(file_t, int offset, int origin)`                           |
-|                 | 0x1105 | `int file_tell(file_t)`                                                   |
-| Dir I/O         | 0x1201 | `dir_t dir_open(const char * path)`                                       |
-|                 | 0x1202 | `void dir_close(dir_t)`                                                   |
-|                 | 0x1203 | `int dir_read(dir_t, void * dir_entry)`                                   |
-|                 | 0x1205 | `int dir_seek(dir_t, int offset, int origin)`                             |
-|                 | 0x1205 | `int dir_tell(dir_t)`                                                     |
-
-## System Calls 2.0
-
-io
-- open handle
-- close handle
-- read handle
-- write handle
-- handle size? (maybe part of open)
-- stat? size? (can't be seek or tell, those are in libc)
-
-
-# System Signals
-
-These are callbacks from the kernel to the process.
-
-The `register_signals` call will hook a function in libc to receive all signals.
-It will then store all registered callbacks of the process.
-
-TODO - keyboard event
+# System Calls
+
+System calls are the mechanism by which processes communicate with the kernel
+both for sending commands and retrieving data. Some examples include file io,
+memory management, process management, etc. System calls are initiated by the
+process. To handle communication form kernel to process, use
+[signals](signals.md).
+
+## Sending System Calls
+
+System calls are sent to the kernel through interrupt 48 (`int 0x30`) which is
+sent to IRQ 16. Each system call has a wrapper function in libk which calls one
+of two functions, `send_call` or `send_call_noret`. These functions take a
+`uint32_t` call id followed by some number va args.
+
+```c
+extern int            send_call(uint32_t int_no, ...);
+extern NO_RETURN void send_call_noret(uint32_t int_no, ...);
+```
+
+> [!TIP]
+> Wrapper functions are declared in
+> [src/libk/include/libk/sys_call.h](../src/libk/include/libk/sys_call.h) and
+> defined in [src/libk/src/sys_call.c](../src/libk/src/sys_call.c).
+
+### Call Id
+
+The system call id is a `uint32_t` where the first 16 bits are the family
+followed by 16 bits for the call number.
+
+| Family  | Call    |
+| ------- | ------- |
+| 16 bits | 16 bits |
+
+> [!TIP]
+> System call ids are defined in
+> [src/libk/include/libk/defs.h](../src/libk/include/libk/defs.h).
+
+## Receiving System Calls
+
+System calls are received by the kernel through callbacks registered with
+`system_call_register`. The callback function receives a `uint32_t` call id,
+`void *` pointer to the va_args pushed onto the stack by `send_call` and
+`send_call_noret`, and `registers_t *` object with values of all registers.
+
+```c
+typedef int (*sys_call_handler_t)(uint32_t call_id, void * args_data, registers_t * regs);
+void system_call_register(uint16_t family, sys_call_handler_t handler);
+```
+
+### Call Arguments
+
+Arguments are accessible from `args_data` which is a pointer to the va_args in
+the caller process stack. A struct can be used to decompose the argument values
+from this pointer.
+
+```c
+struct _args {
+    void * ptr;
+    size_t count;
+} * args = (struct _args *)args_data;
+// use args->ptr or args->count to read values
+```
+
+> [!WARNING]
+>
+> Argument data is stored in the process stack. After changing the page directory
+> the values in `arg_data` will be invalid. Copy values to the kernel stack or
+> heap before switching to retrain access.
+
+### Return Value
+
+Each call handler can optionally returns a single `int` value to the caller
+process by returning a value from the handler function. If no value is returned
+to the caller process, the call handler should return 0.
+
+### Example Handler
+
+A typical call handler uses a switch block to select the correct logic based
+on the `call_id`.
+
+```c
+// Define handler function
+int sys_call_proc_cb(uint32_t call_id, void * args_data, registers_t * regs) {
+    // Get the caller process
+    process_t * proc = get_current_process();
+
+    // Handle call id
+    switch (call_id) {
+        // Log warning if call id is unknown
+        default: {
+            KLOG_WARNING("Invalid call id 0x%X", call_id);
+            break;
+        }
+
+        // Logic for call id
+        case SYS_CALL_MEM_MALLOC: {
+            // va args from send_call or send_call_noret
+            struct _args {
+                size_t size;
+            } * args = (struct _args *)args_data;
+            // Return int to caller process
+            return PTR2UINT(memory_alloc(&proc->memory, args->size));
+        } break;
+
+        case SYS_CALL_MEM_FREE: {
+            struct _args {
+                void * ptr;
+            } * args = (struct _args *)args_data;
+            memory_free(&proc->memory, args->ptr);
+            // No value is returned so handler defaults to 0
+        } break;
+    }
+
+    // Default return value of 0
+    return 0;
+}
+```

design/time.md

@@ -15,7 +15,7 @@ the reload value, reading the current count, etc.
 
 Each channel counts down, starting at the reload value, and when they reach 0
 the output is switched (low -> high or high -> low). In most cases the channel
-is then set to the reload value. 
+is then set to the reload value.
 
 TODO in what cases is the reload value not used when a channel reaches 0?