Fix WAL and add it to cache

[Polyphemus980]

Need to add config to server to decide whether wal should be used or not and then modify the initialization. Option based as i cant be bothered to add cfg and I dont know if that even makes sense if we would want to do some tests with wal and some without for comparison in the same cargo test

[Copilot]

Pull request overview

This PR integrates Write-Ahead Logging (WAL) into the storage layer to enable crash recovery and durability. The implementation migrates from std::sync::mpsc to crossbeam channels for better performance, refactors WAL record operations to use FilePageRef instead of PageId for improved context, and adds serialization support for storage types to enable WAL persistence.

Key changes:

• Implemented WAL integration with periodic and threshold-based automatic flushing
• Migrated all channel-based communication from std::sync::mpsc to crossbeam::channel
• Refactored WAL operations to use FilePageRef (containing both file and page information) instead of just PageId

Reviewed changes

Copilot reviewed 10 out of 11 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
storage/src/write_ahead_log.rs	Added automatic flush mechanisms (periodic and threshold-based), refactored return types from StartHandle to WalHandle and BackgroundWorkerHandle, changed from PageId to FilePageRef for operations, migrated to crossbeam channels
storage/src/cache.rs	Integrated WalClient and WalHandle into Cache initialization (though not yet actively used for logging)
storage/src/files_manager.rs	Added DbSerializable implementations for FileType and FileKey to support WAL persistence, migrated to crossbeam channels
storage/src/background_worker.rs	Migrated shutdown mechanism from std::sync::mpsc to crossbeam::channel
executor/src/lib.rs	Added WAL initialization in with_background_workers, added WAL error variant, updated worker count
executor/src/consts.rs	Added WAL configuration constants for flush interval and max unflushed records
engine/src/heap_file.rs	Updated test to pass None for WAL client parameter
engine/src/b_tree.rs	Updated test to pass None for WAL client parameter
storage/Cargo.toml	Added crossbeam dependency
server/Cargo.toml	Reordered dependencies (moved storage before protocol)
Cargo.lock	Updated lockfile with crossbeam and removed serde from server

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.