ETS Release 2.0.0: Fix critical persistence bugs and improve reliability #33
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…rehensive documentation
## Why This Change Was Made
- Persistence operations (save_to_file/load_from_file) were COMPLETELY BROKEN due to binary/charlist mismatch in FFI
- Error handling violated coding standards by throwing away errors with Error(_) patterns and panicking
- Documentation examples taught users BAD patterns (Error(_) everywhere)
- Test coverage gaps meant bugs went undetected (persistence had ZERO unit tests)
- Functions claimed to never fail (keys() -> List) but could panic on decode errors
This was discovered when trying to verify documentation snippets actually worked.
The type error comment in the snippet couldn't be verified because snippets weren't compiled.
Following the chain led to: untested snippets → broken examples → missing tests → FFI bugs.
## What Was Changed
### Critical Bug Fixes (FFI)
- save_to_file: Convert Gleam String (binary) to Erlang charlist for ets:tab2file
- load_from_file: Same charlist conversion for ets:file2tab
- update_element: Return {ok, nil} instead of ok atom
- ets_tab2file/ets_file2tab: Return {ok, nil} instead of ok atom
- ets_first: Return empty_table instead of empty atom
### Breaking API Changes (Proper Error Handling)
- keys() now returns Result(List(k), EtsError) - can fail on decode errors
- values() now returns Result(List(v), EtsError) - can fail on decode errors
- to_list() now returns Result(List(#(k,v)), EtsError) - can fail on decode errors
- size() now returns Result(Int, EtsError) - depends on keys() which can fail
Removed all panic calls - libraries shouldn't crash user processes. Added proper error
propagation with helpful messages (e.g., 'Key disappeared during iteration').
### Documentation & Testing
- Added world-class hexdocs for all 100+ public functions and types
- Added 4 new test suites: persistence_test, encoders_test, helpers_test, advanced_operations_test
- Increased test coverage from 62 to 77 tests (100% of public API)
- Created 9 verified, tested code snippets (moved to test/snippets/)
- Completely rewrote README with actual tested code (no more Error(_) anti-patterns)
- Fixed all Error(_) and Error(_error) violations in source code
### New Examples
- Custom type encoding with JSON (critical for real apps)
- Preventing duplicates with insert_new() (distributed locks, user registration)
## Note to Future Engineer
The charlist bug is a classic Gleam/Erlang interop gotcha - Gleam Strings are binaries,
but many Erlang file operations expect charlists. Always check Erlang docs for string types.
The panic-on-decode-error bug was insidious because it only happens when:
1. You change your encoder/decoder (migration)
2. Data gets corrupted
3. You have a bug in custom encoder
Without Result types, users had no way to handle this gracefully. Their app would just crash
when iterating keys. Now they can catch decode errors and recover (delete corrupt entries,
log issues, etc.).
Oh, and the persistence operations had ZERO tests before this. They were completely broken
and nobody knew because... nobody tested them. Remember: if it's not tested, it's broken.
You just don't know it yet. 💀
BREAKING CHANGE: keys(), values(), to_list(), and size() now return Result types instead of bare values
…rror-handling Fix broken persistence and improve reliability in dream_ets
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Release: dream_ets v2.0.0
This release fixes critical bugs in dream_ets persistence operations and improves error handling across the module.
What's in This Release
Critical Bug Fixes
Persistence Operations Fixed:
save_to_file()andload_from_file()were completely non-functional due to FFI bugsokinstead of{ok, nil})Error Handling Improved:
Error(_)patterns that threw away error informationResultto handle decode errors gracefullyBreaking Changes
operations.keys():List(k)→Result(List(k), EtsError)operations.values():List(v)→Result(List(v), EtsError)operations.to_list():List(#(k,v))→Result(List(#(k,v)), EtsError)operations.size():Int→Result(Int, EtsError)Migration:
Rationale: These functions iterate and decode table data. If decoding fails (corrupt data, encoder changes), the old code would panic. Now users can handle errors gracefully.
Documentation & Testing
Test Coverage:
Documentation:
Impact Assessment
Breaking changes do NOT affect:
Only affects users of:
keys(),values(),to_list(),size()functionsTesting
All tests passing:
Total: 330 tests, 0 failures
Files Changed
57 files changed (+4000, -1417)
Release Checklist
Ready to merge and release to Hex.pm.