Skip to content

fix: handle UTF-16 and BOM encoded requirements.txt files#1483

Open
sliortega295-ops wants to merge 3 commits intoosprey-oss:mainfrom
sliortega295-ops:fix/issue-1393
Open

fix: handle UTF-16 and BOM encoded requirements.txt files#1483
sliortega295-ops wants to merge 3 commits intoosprey-oss:mainfrom
sliortega295-ops:fix/issue-1393

Conversation

@sliortega295-ops
Copy link
Copy Markdown

Summary

Fixes #1393.

When a requirements.txt file is saved with UTF-16 encoding (e.g., by VS Code), deptry fails to parse it because the BOM (Byte Order Mark) characters are misinterpreted as part of the package name.

Changes

  • Modified get_dependencies_from_requirements_file() in requirements_files.py to read file bytes first and detect encoding via BOM before decoding
  • Added _read_requirements_file() helper that handles:
    • UTF-16 LE (BOM: \xff\xfe)
    • UTF-16 BE (BOM: \xfe\xff)
    • UTF-8 with BOM (BOM: \xef\xbb\xbf)
    • Plain UTF-8 (no BOM, default)

Testing

  • Added 3 new unit tests for UTF-16 LE, UTF-16 BE, and UTF-8 with BOM encoded requirements.txt files
  • All 10 unit tests pass (7 existing + 3 new)
  • No regressions in existing tests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Parsing Error when requirements.txt Has UTF-16 Encoding

1 participant