Skip to content

Fixes nirjas comment extraction#115

Open
AnujRewar wants to merge 1 commit intofossology:masterfrom
AnujRewar:fixes114
Open

Fixes nirjas comment extraction#115
AnujRewar wants to merge 1 commit intofossology:masterfrom
AnujRewar:fixes114

Conversation

@AnujRewar
Copy link
Copy Markdown

@AnujRewar AnujRewar commented Jan 16, 2026

Added try except to check if comment extraction fails from Nirjas , and continue scan without completly failing it.
fixes #114

Copy link
Copy Markdown
Member

@Kaushl2208 Kaushl2208 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PTAL

except Exception as e:
#if the nirjas may mis-parse file
with open(inputFile) as inFile:
lines = inFile.read().split('\n')
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are loading the entire file in memory and then just writing as it is in outfile. This is not optimised.

Look into shutil maybe? helps reduce python overload.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TF-IDF agent crashes on real-world projects due to unhandled comment extraction errors

2 participants