Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
e362b4b
Inverted Text Index Implementation
tomijaga Jul 19, 2025
ce1569c
BasicTokenizer: Initial Implmentation
tomijaga Aug 15, 2025
fc43697
Merge branch 'main' into inverted-text-index
tomijaga Aug 16, 2025
009cfe4
pending update
tomijaga Aug 23, 2025
3beb2b8
Merge branch 'main' into versioning
tomijaga Aug 28, 2025
db2432d
generate random timestamp based unique ids
tomijaga Aug 28, 2025
aa37b4c
TextIndex: Initial Implementation
tomijaga Aug 16, 2025
382b359
Support version pinning for all dependencies
tomijaga Sep 13, 2025
d928a05
Support Type migrations from version 0.2.0 and upwards
tomijaga Sep 13, 2025
38b4afb
Merge branch 'dev-minor-change' into versioning
tomijaga Sep 13, 2025
429732a
Export stats() from each module: ZenDBInstance, Database, Collection …
tomijaga Sep 13, 2025
17fb914
update internal functions to snake_case
tomijaga Sep 13, 2025
88e2a9f
Add a 4 byte instance id to each 8 bytes document id and represent th…
tomijaga Sep 13, 2025
c59e1e6
update benchmarks to support 12 byte document id
tomijaga Sep 17, 2025
27691ca
Support both Text and Composite Indexes
tomijaga Oct 3, 2025
912a939
Update serde version to 3.3.3
tomijaga Oct 8, 2025
e518f89
Add insertDocs() for multi-document inserts
tomijaga Oct 13, 2025
43bf943
Merge branch 'dev-minor-change' into inverted-text-index
tomijaga Oct 13, 2025
f4f5521
Populate Indexes In Batches
tomijaga Oct 27, 2025
7aec64c
Add function to delete collection
tomijaga Oct 30, 2025
da42453
update benchmarks, type migrations and add flattenQuery() fn
tomijaga Oct 31, 2025
46cba97
Adding Support for ZenDB RemoteInstance via a dedicated DB canister
tomijaga Nov 2, 2025
58de2ca
Collection return type updates to return the instructions used by the…
tomijaga Nov 7, 2025
1dffd5e
pending index fields update
tomijaga Nov 10, 2025
909fe60
Collection return type updates to return the instructions used by the…
tomijaga Nov 7, 2025
013e68b
Enable Filtering and Sorting on indexed keys to avoid decoding the en…
tomijaga Nov 10, 2025
afeb526
Add global Document cache to ZenDB instance
tomijaga Nov 10, 2025
f0436eb
Collection return type updates to return the instructions used by the…
tomijaga Nov 7, 2025
3f53fd4
Code update to use the Candid.TypedSerializer module that's more effi…
tomijaga Jul 28, 2025
6652a3a
Add global Document cache to ZenDB instance
tomijaga Nov 10, 2025
368b673
Merge branch 'merge-serde' into dev-minor-change
tomijaga Nov 11, 2025
21c982d
Update the DocumentStore module so its encoding utils are generated w…
tomijaga Nov 11, 2025
2e66d24
Add support for Cursor Pagination
tomijaga Nov 13, 2025
3bb9863
Add kmerge for sorted and iterators and Query.CrossProduct() to expan…
tomijaga Nov 20, 2025
488eba3
Replace BitMap module with SparseBitMap64 module to support larger ke…
tomijaga Nov 22, 2025
4f6d8e8
Update pinned dependency versions
tomijaga Nov 23, 2025
e84ad1b
Add major version v1.0.0 to type migrations
tomijaga Nov 24, 2025
b50577d
Add previous state to Type Migrations
tomijaga Nov 24, 2025
bd4db6e
Refactor codebase and add functions to list ZenDB resources (indexes,…
tomijaga Nov 28, 2025
a550771
Rename Query Sort() fn to SortBy()
tomijaga Nov 28, 2025
5e83a81
use the moc and wasmtime version in the mops.toml file
tomijaga Nov 28, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
197 changes: 97 additions & 100 deletions .github/workflows/benchmarks.yml
Original file line number Diff line number Diff line change
@@ -1,117 +1,114 @@
name: Benchmarks

on:
push:
branches:
- main
pull_request:
branches:
- "*"
push:
branches:
- main
pull_request:
branches:
- "*"

jobs:
benchmark:
runs-on: ubuntu-latest
name: Run Benchmarks and Comment
benchmark:
runs-on: ubuntu-latest
name: Run Benchmarks and Comment

steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
- name: Cache Node modules
uses: actions/cache@v4
with:
path: ~/.npm
key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-node-
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
- name: Cache Node modules
uses: actions/cache@v4
with:
path: ~/.npm
key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-node-

- name: Cache mops packages
uses: actions/cache@v4
with:
key: mops-packages-${{ hashFiles('mops.toml') }}
path: |
~/.cache/mops
- name: Cache mops packages
uses: actions/cache@v4
with:
key: mops-packages-${{ hashFiles('mops.toml') }}
path: |
~/.cache/mops

- name: Install dfx
uses: dfinity/setup-dfx@main
- name: Confirm successful installation
run: dfx --version
- name: Install dfx
uses: dfinity/setup-dfx@main
- name: Confirm successful installation
run: dfx --version

- name: Install dfx cache
run: dfx cache install
- name: Install dfx cache
run: dfx cache install

- name: Install mops & mocv
run: |
npm --yes -g i ic-mops
mops i
mops toolchain init
mops toolchain use moc latest
- name: Install mops & mocv
run: |
npm --yes -g i ic-mops
mops i
mops toolchain init

# set moc path for dfx to use
echo "DFX_MOC_PATH=$(mops toolchain bin moc)" >> $GITHUB_ENV
# set moc path for dfx to use
echo "DFX_MOC_PATH=$(mops toolchain bin moc)" >> $GITHUB_ENV

- name: Create benchmark-results branch if it doesn't exist
uses: peterjgrainger/action-create-branch@v2.2.0
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
branch: "benchmark-results"
sha: "${{ github.event.pull_request.head.sha }}"
continue-on-error: true
- name: Create benchmark-results branch if it doesn't exist
uses: peterjgrainger/action-create-branch@v2.2.0
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
branch: "benchmark-results"
sha: "${{ github.event.pull_request.head.sha }}"
continue-on-error: true

- name: Checkout out the branch with benchmark results
if: github.event_name == 'pull_request'
uses: actions/checkout@v4
with:
ref: benchmark-results
path: .benchmark-results-branch/
- name: Move Saved Benchmarks
if: github.event_name == 'pull_request'
run: mv .benchmark-results-branch/.bench .bench 2>/dev/null || mkdir -p .bench
- name: Checkout out the branch with benchmark results
uses: actions/checkout@v4
with:
ref: benchmark-results
path: .benchmark-results-branch/
- name: Move Saved Benchmarks
run: mv .benchmark-results-branch/.bench .bench 2>/dev/null || mkdir -p .bench

- name: Benchmarks
if: github.event_name == 'pull_request'
id: benchmarks
run: |
mops bench --gc incremental --compare 2>&1 | tee benchmark_results.txt
echo "result<<EOF" >> $GITHUB_OUTPUT
cat benchmark_results.txt >> $GITHUB_OUTPUT
echo "EOF" >> $GITHUB_OUTPUT
- name: Benchmarks
if: github.event_name == 'pull_request'
id: benchmarks
run: |
mops bench --gc incremental --compare 2>&1 | tee benchmark_results.txt
echo "result<<EOF" >> $GITHUB_OUTPUT
cat benchmark_results.txt >> $GITHUB_OUTPUT
echo "EOF" >> $GITHUB_OUTPUT

- name: Find Benchmark Comment
if: github.event_name == 'pull_request'
uses: peter-evans/find-comment@v3
id: fc
with:
issue-number: ${{ github.event.pull_request.number }}
comment-author: "github-actions[bot]"
body-includes: Benchmark Results
- name: Find Benchmark Comment
if: github.event_name == 'pull_request'
uses: peter-evans/find-comment@v3
id: fc
with:
issue-number: ${{ github.event.pull_request.number }}
comment-author: "github-actions[bot]"
body-includes: Benchmark Results

- name: Create or update comment
if: github.event_name == 'pull_request'
uses: peter-evans/create-or-update-comment@v4
with:
comment-id: ${{ steps.fc.outputs.comment-id }}
issue-number: ${{ github.event.pull_request.number }}
body: ${{ steps.benchmarks.outputs.result }}
edit-mode: replace
- name: Create or update comment
if: github.event_name == 'pull_request'
uses: peter-evans/create-or-update-comment@v4
with:
comment-id: ${{ steps.fc.outputs.comment-id }}
issue-number: ${{ github.event.pull_request.number }}
body: ${{ steps.benchmarks.outputs.result }}
edit-mode: replace

- name: Generate Benchmark Results
if: github.event_name != 'pull_request'
run: |
set -o pipefail
mops bench --gc incremental --save --compare 2>&1 | tee bench-results.md
- name: Move Generated Results
if: github.event_name != 'pull_request'
run: |
mkdir .bench-br
mv .bench .bench-br/.bench
mv bench-results.md .bench-br/README.md 2>/dev/null || mkdir -p .bench
- name: Upload benchmark results
if: github.event_name != 'pull_request'
uses: JamesIves/github-pages-deploy-action@v4
with:
token: ${{ secrets.GITHUB_TOKEN }}
branch: benchmark-results
folder: .bench-br
- name: Generate Benchmark Results
if: github.event_name != 'pull_request'
run: |
set -o pipefail
mops bench --gc incremental --save --compare 2>&1 | tee bench-results.md
- name: Move Generated Results
if: github.event_name != 'pull_request'
run: |
mkdir .bench-br
mv .bench .bench-br/.bench
mv bench-results.md .bench-br/README.md 2>/dev/null || mkdir -p .bench
- name: Upload benchmark results
if: github.event_name != 'pull_request'
uses: JamesIves/github-pages-deploy-action@v4
with:
token: ${{ secrets.GITHUB_TOKEN }}
branch: benchmark-results
folder: .bench-br
92 changes: 45 additions & 47 deletions .github/workflows/makefile.yml
Original file line number Diff line number Diff line change
@@ -1,52 +1,50 @@
name: Makefile CI

on:
push:
branches:
- main
pull_request:
branches:
- "*"
push:
branches:
- main
pull_request:
branches:
- "*"

jobs:
build:
runs-on: ubuntu-latest

name: Build and test
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
- name: Cache Node modules
uses: actions/cache@v4
with:
path: ~/.npm
key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-node-

- name: Install dfx
uses: dfinity/setup-dfx@main
- name: Confirm successful installation
run: dfx --version

- name: Install dfx cache
run: dfx cache install

- name: Install mops & mocv
run: |
npm --yes -g i ic-mops
mops i
mops toolchain init
mops toolchain use moc latest
mops toolchain use wasmtime 14.0.4

# set moc path for dfx to use
echo "DFX_MOC_PATH=$(mops toolchain bin moc)" >> $GITHUB_ENV

- name: Detect warnings
run: make check

- name: Run Tests
run: make test
build:
runs-on: ubuntu-latest

name: Build and test
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
- name: Cache Node modules
uses: actions/cache@v4
with:
path: ~/.npm
key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-node-

- name: Install dfx
uses: dfinity/setup-dfx@main
- name: Confirm successful installation
run: dfx --version

- name: Install dfx cache
run: dfx cache install

- name: Install mops & mocv
run: |
npm --yes -g i ic-mops
mops i
mops toolchain init

# set moc path for dfx to use
echo "DFX_MOC_PATH=$(mops toolchain bin moc)" >> $GITHUB_ENV

- name: Detect warnings
run: make check

- name: Run Tests
run: make test
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@
.idea/
**/*~

# Copilot workspace files
.copilot/

# Mac OSX temporary files
.DS_Store
**/.DS_Store
Expand Down
4 changes: 4 additions & 0 deletions .prettierrc.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"tabWidth": 4,
"useTabs": false
}
Loading
Loading