Skip to content

Conversation

@Mjboothaus
Copy link
Contributor

Phase 2.2: Structural Token Tests

Adds comprehensive tests for YAML structural token patterns and fixes dash handling in the lexer.

Changes

  • 7 new structural token tests:

    • Simple sequences and mappings
    • Empty values
    • Nested structures
    • Spacing variations around dashes and colons
  • Lexer improvements:

    • Fixed dash handling: (no space) → string, (space) → list indicator
    • Added dash to valid unquoted string characters

Test Results

  • 64 total tests (10 basic + 15 scalars + 17 values + 15 fixtures + 7 structural)
  • 100% passing

Next Steps

Phase 2.3 will implement indentation tracking with INDENT/DEDENT tokens.

- Added 7 structural token tests:
  - Simple sequences and mappings
  - Empty values
  - Nested structures
  - Spacing variations (dash and colon)

- Fixed lexer dash handling:
  - Dash without trailing space (e.g., '-one') correctly treated as unquoted string
  - Dash with space (e.g., '- item') correctly recognised as list indicator
  - Updated scan_unquoted_string to include dash in valid characters

Tests: 64 total (10 basic + 15 scalars + 17 values + 15 fixtures + 7 structural)
All tests passing (100% success rate)

Co-Authored-By: Warp <agent@warp.dev>
@semanticdiff-com
Copy link

Review changes with  SemanticDiff

@Mjboothaus Mjboothaus merged commit 19f1008 into main Jan 14, 2026
0 of 2 checks passed
@Mjboothaus Mjboothaus deleted the feature/phase-2.2-structural-tokens branch January 14, 2026 08:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants