Feature Enhancement: Vocabulary Generation

[ItsRikan]

Problem

DsKit lacks a simple utility to generate a vocabulary from text columns, which is a common preprocessing step in NLP workflows.

Proposed Solution

• Introduce generate_vocabulary(df:pd.DataFrame,text_col:str,case:Literal['lower','upper']=None)
• Returns a list of unique words present in the specified text column.

Progress

✅ Both features are already implemented

🔄 Open to feedback and ready to refactor or split PRs if needed

Use-Case

• Useful when developing Bag of Words like an ML model
• Provide knowledge about a specific text field

Pull Request

#2 (comment)

[Sreoshh]

Would like to work on this as is already proposed by API clarity and edge cases handling (nulls, punctuation, tokenization), test coverage, and refactoring if needed for extensibility.

[Kritika11052005]

Hi! I’d like to work on this issue.

I plan to focus on API clarity, edge case handling (null values, punctuation, repeated tokens), and consistent return types. I’ll also add unit tests to ensure correctness across different text inputs and refactor if needed to keep the utility extensible for future NLP features.

Open to feedback and happy to align with the project’s design conventions.