refactor(web): add unique identifier to transform-tokenization subsets 🚂#15094
Merged
jahorton merged 1 commit intoepic/autocorrectfrom Jan 28, 2026
Merged
Conversation
User Test ResultsTest specification and instructions User tests are not required Test Artifacts
|
keymanapp-test-bot
bot
changed the title
jahorton
commented
Nov 4, 2025
web/src/engine/predictive-text/worker-thread/src/main/correction/tokenization-subsets.ts
Outdated
Show resolved
Hide resolved
jahorton
force-pushed
the
refactor/web/transform-tokenization-subset-ids
branch
2 times, most recently
from
d13ebc8 to
1d113f5
Compare
jahorton
force-pushed
the
refactor/web/complex-search-space-reuse
branch
from
ca906dd to
316abe3
Compare
jahorton
force-pushed
the
refactor/web/transform-tokenization-subset-ids
branch
from
1d113f5 to
e97c968
Compare
jahorton
changed the base branch from
refactor/web/complex-search-space-reuse
to
refactor/web/relocate-source-range-key
jahorton
force-pushed
the
refactor/web/transform-tokenization-subset-ids
branch
from
e97c968 to
0792377
Compare
jahorton
force-pushed
the
refactor/web/relocate-source-range-key
branch
from
7831a6d to
346f737
Compare
jahorton
changed the base branch from
refactor/web/relocate-source-range-key
to
change/web/rename-inputsource-as-pathinputprops
jahorton
force-pushed
the
refactor/web/transform-tokenization-subset-ids
branch
from
fb4b036 to
15b942e
Compare
jahorton
force-pushed
the
refactor/web/transform-tokenization-subset-ids
branch
from
15b942e to
1e13233
Compare
jahorton
force-pushed
the
change/web/rename-inputsource-as-pathinputprops
branch
from
e3b5354 to
27fc7da
Compare
jahorton
changed the base branch from
change/web/rename-inputsource-as-pathinputprops
to
refactor/web/relocate-source-range-key
jahorton
force-pushed
the
refactor/web/transform-tokenization-subset-ids
branch
from
1e13233 to
56676d8
Compare
jahorton
force-pushed
the
refactor/web/relocate-source-range-key
branch
from
a6d4180 to
3a19b06
Compare
jahorton
force-pushed
the
refactor/web/transform-tokenization-subset-ids
branch
from
56676d8 to
fd4a0d3
Compare
jahorton
force-pushed
the
refactor/web/relocate-source-range-key
branch
from
3a19b06 to
01371c0
Compare
jahorton
force-pushed
the
refactor/web/transform-tokenization-subset-ids
branch
from
fd4a0d3 to
23fdcdc
Compare
jahorton
force-pushed
the
refactor/web/relocate-source-range-key
branch
from
01371c0 to
a679ebe
Compare
jahorton
force-pushed
the
refactor/web/transform-tokenization-subset-ids
branch
from
23fdcdc to
8d4aa9e
Compare
jahorton
force-pushed
the
refactor/web/relocate-source-range-key
branch
from
cb6ffee to
cf39508
Compare
This addition will allow us to clearly and cleanly indicate transforms that are two (or more) halves of the same original whole. It is notably more selective than just the original transition ID and is better suited for indicating split-transform cases. Build-bot: skip build:web Test-bot: skip
jahorton
force-pushed
the
refactor/web/transform-tokenization-subset-ids
branch
from
d36203a to
fd45aa4
Compare
mcdurdin
changed the title
ermshiperete
approved these changes
Jan 28, 2026
Base automatically changed from
refactor/web/relocate-source-range-key
to
epic/autocorrect
January 28, 2026 23:20
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This addition - of a unique identifier to distinct transition 'edge' sets leading from one tokenization (see #15095, the followup) - will allow us to clearly and cleanly indicate transforms that are two (or more) halves of the same original whole. It is notably more selective than just the original transition ID and is better suited for indicating split-transform cases. These changes will help support #15023 and its improved abstractions for token-splitting.
Build-bot: skip build:web
Test-bot: skip