Merged
Conversation
…for all tidesdb tables in information_schema.tables. the info() handler was using total_data_size from tidesdb_get_stats() which only counts sstable klog+vlog on disk and ignores the active memtable. for tables that have not flushed yet (or when the library does not track per-sstable sizes) this was always zero. the fix adds memtable_size to the data size for both the main column family and each secondary index column family. when both sstable and memtable sizes are still zero but total_keys is positive, it falls back to an estimate using total_keys * (avg_key_size + avg_value_size). the same logic is applied to index_file_length by summing across all secondary index column families.
…l for all tidesdb tables in information_schema.tables. create_time is now seeded from the .frm file mtime at first open and stored in tidesdb_share. update_time is bumped atomically once per write statement in external_lock(F_UNLCK) when stmt_txn_dirty is true. both are reported under HA_STATUS_TIME in info(), which is the flag information_schema uses. the tidesdb_info_schema mtr test now covers data_length, index_length, table_rows, create_time, and update_time.
…back signatures (commit, rollback, close_connection) dropped the handlerton* first parameter and st_typelib gained a hidden_values member. the version guard was >= 120000 which missed 11.8. lowered to >= 110800. #55 corrected; compilation error on mariadb 12.3. TABLE_SHARE::option_struct was removed. added TDB_TABLE_OPTIONS(tbl) compat macro that resolves to tbl->option_struct on 12.3+ and tbl->s->option_struct on earlier versions. all table option accesses now go through this macro.
…serts were silently accepted because tidesdb_txn_put overwrites existing keys without error. write_row now checks for existing pk via tidesdb_txn_get before inserting, and checks unique secondary indexes via iterator prefix scan. returns HA_ERR_FOUND_DUPP_KEY with lookup_errkey set so mariadb produces the standard ER_DUP_ENTRY error. the check is skipped for REPLACE INTO and INSERT ON DUPLICATE KEY UPDATE via the write_can_replace_ flag set from HA_EXTRA_WRITE_CAN_REPLACE and HA_EXTRA_INSERT_WITH_UPDATE in extra(). the tidesdb_stress test was updated to verify both duplicate rejection and REPLACE INTO behavior. #53 updated read me mariadb support
…licates exist. inplace_alter_table now tracks seen index-column prefixes per unique index via std::set during the population scan. if a duplicate prefix is found the ALTER is aborted with ER_DUP_ENTRY and the partially built index CF is rolled back. the tidesdb_online_ddl test now verifies that ADD UNIQUE rejects duplicates and preserves all existing rows
guycipher
changed the title
…_struct was removed. added TDB_TABLE_OPTIONS(tbl) compat macro that resolves to tbl->option_struct on 12.3+ and tbl->s->option_struct on earlier versions. all table option accesses now go through this macro. #61 corrected; converting InnoDB tables with foreign keys to TidesDB failed with ER_ROW_IS_REFERENCED ('Cannot delete or update a parent row'). the MariaDB SQL layer called InnoDB's can_switch_engines() which unconditionally returns false when foreign keys exist, and the check did not respect SET FOREIGN_KEY_CHECKS=0 unlike other FK checks in ALTER TABLE. fixed by adding the OPTION_NO_FOREIGN_KEY_CHECKS guard to the can_switch_engines check in sql/sql_table.cc. users can now SET FOREIGN_KEY_CHECKS=0 before ALTER TABLE t ENGINE=TidesDB. the foreign keys are silently dropped since TidesDB does not support them. a new tidesdb_fk_convert mtr test verifies self-referencing and parent-child FK conversions with data preservation. #70 corrected; ALTER TABLE crashes MariaDB with large tables. the copy-based ALTER TABLE scan reads every row from the source table. with REPEATABLE_READ isolation (e.g. when autocommit=0), the TidesDB library added each key to the transaction's read-set for conflict detection, causing unbounded memory growth and heap corruption in tidesdb_txn_add_to_read_set during realloc. fixed by forcing READ_COMMITTED isolation for all DDL operations (ALTER TABLE, CREATE/DROP INDEX, TRUNCATE, OPTIMIZE, CREATE TABLE, DROP TABLE) in both external_lock and ensure_stmt_txn. at READ_COMMITTED the library skips read-set tracking entirely. DDL never needs OCC conflict checks so this is always safe. a new tidesdb_alter_crash mtr test verifies ALTER TABLE with ~100K rows under autocommit=0 completes without crash. #64 verified mtr extended; START TRANSACTION WITH CONSISTENT SNAPSHOT now works correctly. the start_consistent_snapshot handlerton callback was already implemented -- it eagerly creates a TidesDB transaction with REPEATABLE_READ isolation so the snapshot sequence number is captured immediately, not lazily at first data access. rows committed by other connections after the snapshot are correctly filtered by the library's sequence-based visibility check (seq <= snapshot_seq). a new tidesdb_consistent_snapshot mtr test verifies snapshot isolation across connections, multiple inserts after snapshot, and that plain BEGIN (without CONSISTENT SNAPSHOT) correctly sees new data. #57 corrected mtr extended; data survived DROP TABLE and CREATE OR REPLACE TABLE because the handlerton drop_table callback returned -1 ('not handled'). mariadb 12.x only calls hton->drop_table during DROP TABLE -- the handler::delete_table method is never reached. since the callback returned -1, the TidesDB column families and their on-disk directories were never removed. the next CREATE TABLE with the same name reused the existing CF with all its old SSTables. fixed by implementing tidesdb_hton_drop_table which calls tidesdb_drop_column_family for the main data CF and all secondary index CFs, then force-removes directories as a safety net. both hton->drop_table and handler::delete_table now share the same tidesdb_drop_table_impl. a new tidesdb_drop_create mtr test verifies DROP + CREATE, CREATE OR REPLACE, secondary index cleanup, and TRUNCATE TABLE. #56 added per-session TTL override via MYSQL_THDVAR_ULONGLONG. users can now set tidesdb_ttl at session scope to apply TTL to inserts/updates on any table, even tables without a table-level TTL option. priority: per-row TTL_COL > session tidesdb_ttl > table TTL option > no expiration. supports SET SESSION tidesdb_ttl=N as well as SET STATEMENT tidesdb_ttl=N FOR INSERT which scopes TTL to a single statement. the tidesdb_ttl mtr test now covers session override, SET STATEMENT syntax, and priority ordering vs per-row TTL_COL. #53 updated read me mariadb support
…convert test skip, and library cache staleness the consistent snapshot test failed because after commit frees the txn and the allocator reuses the same address for the next txn, the pointer comparison in rnd_init falsely passes and the old iterator with a stale snapshot is reused. added a monotonic txn_generation counter to tidesdb_trx_t that increments each time get_or_create_trx creates a new txn; rnd_init, index_init, and ensure_scan_iter now compare both the pointer and the generation before reusing a cached iterator. the fk convert test now detects when the server-side sql_table.cc patch is absent (upstream mariadb) and skips gracefully instead of failing. the alter crash was caused by a stale cached tidesdb library in ci that lacked the read_committed guard in tidesdb_txn_add_to_read_set; removed tidesdb-lib/build from the cache paths and added rm -rf tidesdb-lib before each clone step so the install always builds fresh from latest master.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…for all tidesdb tables in
information_schema.tables. the info() handler was using total_data_size from tidesdb_get_stats() which only counts sstable klog+vlog on disk and ignores the active memtable. for tables that have not flushed yet (or when the library does not track per-sstable sizes) this was always zero. the fix adds memtable_size to the data size for both the main column family and each secondary index column family. when both sstable and memtable sizes are still zero but total_keys is positive, it falls back to an estimate using total_keys * (avg_key_size + avg_value_size). the same logic is applied to index_file_length by summing across all secondary index column families.