Skip to content

Parser/linter enhancements #79

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
k-schnickmann wants to merge 44 commits into
base: main
Choose a base branch
from
Open

Parser/linter enhancements #79

k-schnickmann wants to merge 44 commits into from

Conversation

@k-schnickmann
Copy link
Contributor

@k-schnickmann k-schnickmann commented Jan 12, 2026

Improvements across all parsers and linters based on evaluation of real-world search query data.

PubMed

  • Added Date – Publication field
  • Refined allowed year range values

Web of Science

  • Deprecated WOS_0002 (maximum NEAR distance)
  • Added WOS_0013 (invalid NEAR query)
  • Fixed maximum search term limit
  • Removed redundant code

EBSCOhost

  • Added support for additional search fields
  • Updated term regex and case-handling logic
  • Revised wildcard validation to reflect recent platform changes
  • Improved handling of ambiguous tokens
  • Improved detection of unbalanced quotes

All platforms

  • Enforced operator precedence during parsing (instead of using artificial parentheses)
  • Improved search string generation from token lists
  • Refined list tokenization
  • Handle invalid characters during tokenization
  • Added test cases for all newly encountered errors

@k-schnickmann
Add [Date - Publication] field
f09fc2d
@k-schnickmann
Adjust year value regex
bf7cff0
@k-schnickmann
Enforce operator precedence in the parsing step, remove artificial pa…
536eddd 
…rentheses
@k-schnickmann
Revise build_query_str() to consistently add artificial parentheses w…
0638473 
…hen replacing list references with query content
@k-schnickmann
Force numeric comparison in build_query_str()
154f68b
@k-schnickmann
Add test case for different year range notation
9076418
@k-schnickmann
Adjust tokenize_operator_node() to interpret gaps between list refs a…
50259fa 
…s single token
@k-schnickmann
Add test case for revised tokenize_operator_node() method
14574a6
@k-schnickmann
Deprecate WOS_0002
df1e10b
@k-schnickmann
Add WOS_0013 (Invalid-near-query)
0dedc37
@k-schnickmann
Fix WOS max term amount
c7a8d2b
@k-schnickmann
Remove redundant method overriding
5c08cc1
@k-schnickmann
Add support for additional Ebsco fields
9a836ca
@k-schnickmann
Adjust EBSCO tokenizer regex to other platforms to broaden range of p…
fd3eb14 
…arsed terms and simplify patterns
@k-schnickmann
Revise EBSCOHost wildcard validation
96beecc
@k-schnickmann
Merge branch 'CoLRev-Environment:main' into main
a9a8240
@k-schnickmann
Update fix_ambiguous_tokens() to identify FIELD + OPERATOR token sequ…
74afe1c 
…ences
@k-schnickmann
Update test cases
deada78
@k-schnickmann
Use inline flags in EBSCOHost regex
11cfd44
@k-schnickmann
Update check_unsupported_wildcards()
430f4f7
@k-schnickmann
Call fix_ambiguous_tokens() before combine_subsequent_tokens()
c6bc83d
@k-schnickmann
Use parser_base method for combining terms in EBSCO
3800919
@k-schnickmann
Adjust Ebsco term regex to exclude quotes
ee75535
@k-schnickmann
Revise combine_subsequent_terms
c49db82
@k-schnickmann
Add missing operator validation step in WOS
f0ec297
@k-schnickmann
Add test cases
f5ceace
@k-schnickmann
Handle invalid characters during token validation
16b034d
@k-schnickmann
Revise EBSCO quoted term tokenization
c31b3f9
@k-schnickmann
Merge branch 'CoLRev-Environment:main' into main
2723b68
@k-schnickmann
Revise quoted search term tokenization and validation
12c0c4c
@geritwagner geritwagner force-pushed the main branch 3 times, most recently from 60e4179 to 8ddab21 Compare January 26, 2026 20:08
@geritwagner geritwagner force-pushed the main branch 3 times, most recently from e42e620 to a5261c1 Compare January 29, 2026 08:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@k-schnickmann