Skip to content

Added Support for explicit-key and Refactored TokenStream#76

Open
HatoYuze wants to merge 16 commits intoHim188:masterfrom
HatoYuze:master
Open

Added Support for explicit-key and Refactored TokenStream#76
HatoYuze wants to merge 16 commits intoHim188:masterfrom
HatoYuze:master

Conversation

@HatoYuze
Copy link
Copy Markdown
Contributor

TokenStream Refactoring

  • Converted boolean state fields (such as quoted and newLined) into int-based bitmasks.
  • Modified the logic of nextToken:
    • Added ignoreIndex to ignore characters at specific positions when reading tokens from source, while simultaneously increasing currentIndent.
    • Updated the parsing conditions for Token: For Token.COLON, Token.MULTILINE_LIST_FLAG, and the newly added Token.EXPLICIT_MAPPING_KEY,
      the next character must be a space; otherwise, it will be parsed as a string.

      Additionally, the behavior for Token.COMMA depends on stopOnComma. If this parameter is true, the behavior remains unchanged; otherwise, it aligns with the above logic.

      As a result, the original compensation for misparsed tokens has been removed.
  • Modified the unquoted string escape logic (Escape.kt):
    • A string ending with a : character will only terminate if the next character is a space.
    • currentIndent is no longer reset upon encountering a newline character. Instead, cur is rolled back to the position before the newline (the responsibility of resetting currentIndent should be handled by the subsequent nextToken call).

New Features

explicit-key (complex key)

c-mapping-key ::= '?'

[189] c-l-block-map-explicit-entry(n) ::=

c-l-block-map-explicit-key(n)

(

l-block-map-explicit-value(n)

| e-node # ""

)

[190] c-l-block-map-explicit-key(n) ::=

c-mapping-key # '?' (not followed by non-ws char)

s-l+block-indented(n,BLOCK-OUT)

Enabled the use of non-primitive elements as mapping keys. Implemented parsing and serialization for the explicit-key specification (during serialization, explicit specification of whether to provide '?' is not supported; it will only appear automatically when the key is non-primitive).

Requirements:

  • A space must follow the '?'.
  • Supported only for map; not supported in (de)serializers for classes.
  • The key and value must be on separate lines (this requirement does not apply in flow format).
  • The indentation of the nested content of the key must be greater than the indentation of the '?' (this requirement does not apply in flow format).
  • ? and : must be at the same indentation level; otherwise, the colon will be treated as part of the nested element (in flow format, only requires all { and } to be matched).

Additionally, the following requirements from the specification have been implemented for BlockMap:

  • : <value> will be parsed as null: <value>.
  • For string keys, the length will be checked to ensure it does not exceed 1024 characters.

Changes to YamlDecodingException

The original error messages included meaningless stack traces, which could impair user judgment. Therefore, on the jvm platform, the corresponding stack traces have been removed.


Theoretically, this PR resolves the following issues:

close #70 #66 #45 #36

@HatoYuze
Copy link
Copy Markdown
Contributor Author

@Him188 ping❤️
I forgot to sync the main branch when I started writing, please forgive me.🙁

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Negative number in a list is not handled

1 participant