Skip to content

fix: ensure cell selections are always rectangular#330

Open
liwenka1 wants to merge 4 commits intoProseMirror:masterfrom
liwenka1:fix/rectangular-cell-selection
Open

fix: ensure cell selections are always rectangular#330
liwenka1 wants to merge 4 commits intoProseMirror:masterfrom
liwenka1:fix/rectangular-cell-selection

Conversation

@liwenka1
Copy link
Contributor

Problem

When selecting table cells with colspan or rowspan attributes, the current implementation allows non-rectangular selections (T-shaped, L-shaped, etc.). This behavior is inconsistent with mature table editors and can lead to unexpected results.

Example Issue

When selecting from cell A to cell C in the following table:

| A | B (rowspan=2) | C |
| D | B             | E |

Before Fix: The selection would be T-shaped, only including cells A, B, and C.

After Fix: The selection automatically expands to include all cells (A, B, C, D, E), forming a complete rectangle.

Before Fix

20251225174815_rec_

The selection creates a non-rectangular shape when crossing cells with rowspan/colspan.

After Fix

20251225174852_rec_

The selection automatically expands to ensure a rectangular shape, including all affected cells.

Industry Standards

This fix aligns with the behavior of professional table editors:

Feishu (Lark) Table Selection

20251225174706_rec_

Feishu enforces rectangular selections in tables.

Microsoft Office Table Selection

img_v3_02ta_1c853afd-ed94-4df9-9dcd-c2283e9422hu

Microsoft Word/Excel also maintains rectangular cell selections.

Implementation Details

Core Changes

Modified TableMap.rectBetween() in src/tablemap.ts:

  • Added automatic rectangle expansion logic
  • Iteratively checks all cells within the current rectangle
  • If any cell spans beyond the rectangle boundaries (via colspan or rowspan), the rectangle expands to include it
  • Continues until the rectangle is stable (no more expansion needed)

Algorithm

1. Calculate initial rectangle between anchor and head cells
2. While rectangle is expanding:
   a. For each cell in current rectangle:
      - Get the cell's actual boundaries (considering colspan/rowspan)
      - If cell extends beyond rectangle, expand rectangle to include it
   b. If any expansion occurred, repeat step 2
3. Return the final rectangular selection

Test Coverage

  • ✅ Added test/cellselection-rect.test.ts with specific tests for rectangular constraints
  • ✅ Updated test/tablemap.test.ts to reflect new rectBetween behavior
  • ✅ Updated test/cellselection.test.ts - selections now expand instead of cutting off cells
  • ✅ Updated test/commands.test.ts - commands now operate on expanded rectangular selections

All 181 tests pass ✅

Benefits

  1. Consistent UX: Matches user expectations from other table editors (Feishu, Office, Google Docs)
  2. Predictable Behavior: No more surprising T-shaped or L-shaped selections
  3. Correct Operations: Table operations (merge, delete, copy) now work on complete rectangular regions
  4. Better Accessibility: Users can select complete table regions without worrying about internal cell structure

Breaking Changes

⚠️ Behavior Change: Cell selections that previously created non-rectangular shapes will now automatically expand to rectangles. This could affect:

  • Custom code that relies on non-rectangular selections
  • Tests that expect partial cell selections across spanning cells

However, this change makes the behavior more intuitive and consistent with industry standards.

Migration Guide

If you have code that depends on the old behavior:

Before:

// Selection from A to C would only include A, B, C
const selection = new CellSelection($anchorCell, $headCell);

After:

// Selection from A to C now includes A, B, C, D, E (complete rectangle)
const selection = new CellSelection($anchorCell, $headCell);
// The selection is automatically expanded to be rectangular

No API changes are required - the expansion happens automatically in the CellSelection constructor.

Related Issues

Fixes issue where table selections could create non-rectangular shapes when cells have colspan or rowspan attributes.


Checklist:

  • All tests pass
  • Code follows project style guidelines (Prettier + ESLint)
  • New tests added for the feature
  • Behavior aligns with industry standards (Feishu, Office)

- Modified TableMap.rectBetween to automatically expand selection rectangles
- Selections now include all cells that span across the boundary
- Prevents T-shaped or L-shaped selections when cells have colspan/rowspan
- Added tests to verify rectangular selection constraint
- Updated existing tests to reflect the new behavior

This change aligns with the behavior of mature table editors like Feishu
and Microsoft Office, where table selections are always rectangular.
@pkg-pr-new
Copy link

pkg-pr-new bot commented Dec 25, 2025

Open in StackBlitz

npm i https://pkg.pr.new/ProseMirror/prosemirror-tables@330

commit: f5b03ce

@github-actions
Copy link
Contributor

github-actions bot commented Dec 25, 2025

Coverage Report

Status Category Percentage Covered / Total
🔵 Lines 59.52% 987 / 1658
🔵 Statements 59.39% 1144 / 1926
🔵 Functions 53.03% 105 / 198
🔵 Branches 52.98% 648 / 1223
File Coverage
File Stmts Branches Functions Lines Uncovered Lines
Changed Files
src/cellselection.ts 68.29% 61.31% 56% 70.21% 96, 159, 180-182, 212-231, 282, 292-295, 340-342, 351-353, 359-370, 387, 400-475, 499-502
src/index.ts 63.63% 35.71% 80% 66.66% 112, 114-129
src/input.ts 23.48% 23.85% 35.71% 25.45% 54, 64, 67-71, 73, 75, 86-89, 97-254, 260, 265, 276-302
src/tablemap.ts 85.61% 86.66% 87.5% 83.89% 62-72, 140, 150, 212-213, 238-239, 246-247, 250-251, 267-268, 271-272, 275-276, 279-280, 293-294, 297-298, 301-302, 305-306, 367, 484
Unchanged Files
src/columnresizing.ts 3.42% 0.77% 4% 3.82% 63-385, 398-430
src/commands.ts 78.4% 70.56% 74.35% 80.66% 109, 175, 179, 187, 258, 275, 339, 343, 351, 388, 437, 501, 509, 570, 596, 690, 783-988
src/copypaste.ts 98.36% 91.48% 100% 98.7% 39, 333, 345
src/fixtables.ts 74.44% 70.21% 75% 77.61% 34-49, 73, 97, 109, 116
src/schema.ts 46.15% 23.68% 41.66% 47.91% 15-37, 42, 43, 45, 48, 122-130, 165, 182, 194-197
src/tableview.ts 0% 0% 0% 0% 16-100
src/util.ts 74.28% 79.16% 73.33% 73.68% 38, 47, 80-102, 117, 135-142
src/utils/convert.ts 97.95% 91.66% 100% 97.77% 92
src/utils/get-cells.ts 0% 0% 0% 0% 17-72
src/utils/move-column.ts 0% 0% 0% 0% 35-88
src/utils/move-row-in-array-of-rows.ts 100% 100% 100% 100%
src/utils/move-row.ts 0% 0% 0% 0% 34-80
src/utils/query.ts 0% 0% 0% 0% 13-117
src/utils/selection-range.ts 0% 0% 0% 0% 28-190
src/utils/transpose.ts 100% 100% 100% 100%
test/build.ts 96.29% 91.66% 100% 96.15% 78-80
Generated in workflow #91 for commit f5b03ce by the Vitest Coverage Report Action

Comment on lines 14 to 18
tr(
td(p('A')), // pos 1
td({ rowspan: 2 }, p('B')), // pos 6
td(p('C')), // pos 11
),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's unclear what pos 1 means here. I recommend using the comment shown here in the test file.

Comment on lines 33 to 35
console.log('Map:', map.map);
console.log('Rect:', rect);
console.log('Cells in rect:', map.cellsInRect(rect));
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove these console.log in the test file

Comment on lines 74 to 76
console.log('Map:', map.map);
console.log('Rect:', rect);
console.log('Cells in rect:', map.cellsInRect(rect));
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove these console.log in the test file

src/tablemap.ts Outdated
Comment on lines 201 to 216
if (cellRect.left < rect.left) {
rect.left = cellRect.left;
expanded = true;
}
if (cellRect.right > rect.right) {
rect.right = cellRect.right;
expanded = true;
}
if (cellRect.top < rect.top) {
rect.top = cellRect.top;
expanded = true;
}
if (cellRect.bottom > rect.bottom) {
rect.bottom = cellRect.bottom;
expanded = true;
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I remove all four expanded = true here, I can still pass all tests. Could you construct a test case that will fail once expanded = true lines are removed?

You can use the following case as an example:

CleanShot.2025-12-27.at.04.13.19.mp4
CleanShot.2025-12-27.at.04.14.20.mp4

src/tablemap.ts Outdated
Comment on lines 194 to 198
for (let row = rect.top; row < rect.bottom; row++) {
for (let col = rect.left; col < rect.right; col++) {
const index = row * this.width + col;
const cellPos = this.map[index];
const cellRect = this.findCell(cellPos);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe there is room for performance improvement:

  1. We don't need to check all cells in the current rectangle. We only need to check the cells at the four edges. This can reduce the check numbers from $\text{width} \times \text{height}$ to $O(\text{width} + \text{height})$.
  2. this.findCell() call is slow because it's an $O(\text{width} \times \text{height})$ operation. We should avoid unnecessary this.findCell() calls. One method is to use a seen object to cache all checked points like this example.

@ocavue
Copy link
Collaborator

ocavue commented Dec 26, 2025

This is a great improvement! I left some review comments.

- Optimize cell checking from O(width×height) to O(width+height) per iteration
- Add seen cache to avoid redundant findCell() calls
- Update position comments to use standard /* pos */ format
- Remove console.log statements from tests
- Add test case for complex rowspan/colspan expansion validation

Performance improvement: For a 10×10 table, reduced from 100 cells checked
to 40 cells per iteration (60% reduction). The optimization maintains
correctness by checking only edge cells, as interior cells cannot cause
rectangle expansion.
@liwenka1
Copy link
Contributor Author

Thank you @ocavue for the thorough review and valuable suggestions! I've addressed all the feedback points

@RatoX
Copy link
Collaborator

RatoX commented Dec 27, 2025

Quick context: This wasn't a bug but a feature.

forcing a rectangular selection will make some UX behaviours quite difficult to implement like: "change the background color for an entire row"

@ocavue
Copy link
Collaborator

ocavue commented Dec 27, 2025

Quick context: This wasn't a bug but a feature.

forcing a rectangular selection will make some UX behaviours quite difficult to implement like: "change the background color for an entire row"

That's a good point. I just checked Excel's behavior, and it does provide different behaviors:

  1. If I hold the mouse and drag a selection, it always creates a rectangular shape.

  2. If I click the column and row header, I can only select "part of" a merged cell thus I can "change the background color for an entire row".
CleanShot.2025-12-28.at.00.13.59.mov

@ocavue
Copy link
Collaborator

ocavue commented Dec 27, 2025

Perhaps we can add a new option to new CellSelection($anchorCell, $headCell) to indicate whether we want to force the selection to be rectangular. When creating row/column selection programmatically, we don't force it. When creating a selection via the mouse move event, we force it.

@liwenka1
Copy link
Contributor Author

Thanks for the valuable feedback! I've implemented the forceRectangular option as suggested.

Changes:

  • Added CellSelectionOptions interface with forceRectangular?: boolean
  • CellSelection constructor now accepts an optional third parameter
  • TableMap.rectBetween() accepts forceRectangular parameter (default: false)
  • Mouse drag and Shift+Arrow selections use forceRectangular: true
  • rowSelection() and colSelection() use default false, allowing partial merged cell selection

Behavior:

Selection Method forceRectangular Can select partial merged cells
Mouse drag true No
Shift+Arrow true No
rowSelection() false Yes
colSelection() false Yes
new CellSelection() false (default) Yes

This matches Excel's behavior where drag selections are rectangular, but row/column header selections can include partial merged cells.

Please review when you have time. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants