Implement comprehensive search and index system for bash.d repository #18
Conversation
|
To generate Unit Tests for this PR, please click here. |
|
Important Review skippedBot user detected. To trigger a single review, invoke the You can disable this status message by setting the Comment |
Co-authored-by: cbwinslow <8528478+cbwinslow@users.noreply.github.com>
Co-authored-by: cbwinslow <8528478+cbwinslow@users.noreply.github.com>
Co-authored-by: cbwinslow <8528478+cbwinslow@users.noreply.github.com>
…ASCII chars Co-authored-by: cbwinslow <8528478+cbwinslow@users.noreply.github.com>
… population Co-authored-by: cbwinslow <8528478+cbwinslow@users.noreply.github.com>
Co-authored-by: cbwinslow <8528478+cbwinslow@users.noreply.github.com>
PR Compliance Guide 🔍Below is a summary of compliance checks for this PR:
Compliance status legend🟢 - Fully Compliant🟡 - Partial Compliant 🔴 - Not Compliant ⚪ - Requires Further Human Verification 🏷️ - Compliance label |
||||||||||||||||||||||||||
PR Code Suggestions ✨Explore these optional code suggestions:
|
||||||||||||||||||||||||
🧪 CI InsightsHere's what we observed from your CI run for 5ac76ae. 🟢 All jobs passed!But CI Insights is watching 👀 |
![sentry[bot]](https://avatars.githubusercontent.com/in/12637?s=60&v=4){kind=small}
| local verbose="$3" | ||
| local count_only="$4" | ||
|
|
||
| local index_file="${BASHD_INDEX_FILE}" |
There was a problem hiding this comment.
Bug: Helper functions like _bashd_search_indexed lack a fallback for the BASHD_INDEX_FILE variable, causing silent failures if it is unset, unlike the main bashd_search function.
Severity: HIGH
Suggested Fix
Update all functions that use the index file to include a consistent fallback for BASHD_INDEX_FILE. The variable should be defined using the pattern: local index_file="${BASHD_INDEX_FILE:-${BASHD_HOME:-$HOME/.bash.d}/.index/master_index.json}". This ensures the functions behave predictably even if the environment variable is not set.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent.
Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not
valid.
Location: bash_functions.d/core/search.sh#L86
Potential issue: Several helper functions, including `_bashd_search_indexed`,
`bashd_locate`, and `bashd_sort`, initialize the `index_file` variable directly from
`BASHD_INDEX_FILE` without a fallback default. This is inconsistent with the main
`bashd_search` function, which does provide a default. If `BASHD_INDEX_FILE` is unset,
which can happen if a user sources files individually without the main loader, these
functions will fail. Specifically, `_bashd_search_indexed` will pass an empty filename
to `jq`, causing the search to fail silently and return no results.
Did we get this right? 👍 / 👎 to inform future reviews.
There was a problem hiding this comment.
Pull request overview
This pull request implements a comprehensive search and index system for the bash.d repository, addressing the need for organized indexing and searchability across 2500+ functions, aliases, and scripts. The implementation adds JSON-based indexing, multiple search methods, navigation utilities, and session management capabilities.
Changes:
- JSON-based indexer that extracts metadata from function headers and builds a searchable database
- Five search commands: unified search, pattern matching, exact lookup, fuzzy search, and content search
- Organization utilities including sorting, navigation, recent/popular tracking, and session management
- Comprehensive help system and tab completions for all commands
Reviewed changes
Copilot reviewed 8 out of 9 changed files in this pull request and generated 15 comments.
Show a summary per file
| File | Description |
|---|---|
| docs/SEARCH_SYSTEM.md | Comprehensive 454-line documentation covering all commands, examples, and troubleshooting |
| completions/search_completions.bash | Tab completion definitions for all search commands with intelligent context-aware suggestions |
| bash_functions.d/core/utilities.sh | Utility functions for sorting, navigation, sessions, and metadata display |
| bash_functions.d/core/search_system_loader.sh | Auto-loader that initializes the search system and creates convenient aliases |
| bash_functions.d/core/search_help.sh | Help system with overview and command-specific documentation |
| bash_functions.d/core/search.sh | Core search functions implementing unified, fuzzy, pattern, and content search |
| bash_functions.d/core/indexer.sh | JSON indexer that extracts metadata and builds master index database |
| README.md | Updated to document the new search system with examples and short aliases |
| .gitignore | Excludes generated index files from version control |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
||
| # Extract function names defined in file (support both syntaxes) | ||
| local functions_defined | ||
| functions_defined=$(grep -oP '(^[a-zA-Z_][a-zA-Z0-9_]*\(\)|^function\s+[a-zA-Z_][a-zA-Z0-9_]*)' "$file" | \ |
There was a problem hiding this comment.
The -P option (Perl regex) for grep is a GNU extension and not portable to BSD/macOS systems. Since cross-platform compatibility (Linux/macOS) is mentioned in the PR description, use grep -E with extended regex instead, or add a platform-specific fallback.
| functions_defined=$(grep -oP '(^[a-zA-Z_][a-zA-Z0-9_]*\(\)|^function\s+[a-zA-Z_][a-zA-Z0-9_]*)' "$file" | \ | |
| functions_defined=$(grep -E -o '(^[a-zA-Z_][a-zA-Z0-9_]*[[:space:]]*\(\)|^[[:space:]]*function[[:space:]]+[a-zA-Z_][a-zA-Z0-9_]*)' "$file" | \ |
| --height=80%) | ||
| if [[ -n "$selected" ]]; then | ||
| local item_type=$(echo "$selected" | grep -oP '^\[\K[^\]]+') |
There was a problem hiding this comment.
The -P option (Perl regex) for grep is not portable to macOS/BSD. Use sed or awk for better cross-platform compatibility. For example: sed 's/^\[\([^]]*\)\].*/\1/'
| local item_type=$(echo "$selected" | grep -oP '^\[\K[^\]]+') | |
| local item_type=$(echo "$selected" | sed 's/^\[\([^]]*\)\].*/\1/') |
| local count_only=false | ||
|
|
||
| while [[ "$1" =~ ^- ]]; do | ||
| case "$1" in | ||
| -i|--interactive) interactive=true; shift ;; | ||
| -v|--verbose) verbose=true; shift ;; | ||
| -c|--count) count_only=true; shift ;; |
There was a problem hiding this comment.
The count_only parameter is parsed from command-line options and passed to _bashd_search_indexed, but it is never actually used in the function implementation. Either implement the count-only functionality or remove this option from the help text and code.
| local count_only=false | |
| while [[ "$1" =~ ^- ]]; do | |
| case "$1" in | |
| -i|--interactive) interactive=true; shift ;; | |
| -v|--verbose) verbose=true; shift ;; | |
| -c|--count) count_only=true; shift ;; | |
| while [[ "$1" =~ ^- ]]; do | |
| case "$1" in | |
| -i|--interactive) interactive=true; shift ;; | |
| -v|--verbose) verbose=true; shift ;; |
| case "$1" in | ||
| -i|--interactive) interactive=true; shift ;; | ||
| -v|--verbose) verbose=true; shift ;; | ||
| -c|--count) count_only=true; shift ;; |
There was a problem hiding this comment.
The count_only parameter is parsed from command-line options and passed to _bashd_search_indexed, but it is never actually used in the function implementation. Either implement the count-only functionality or remove this option from the help text and code.
|
|
||
| # Use index if available | ||
| if [[ -f "$index_file" ]]; then | ||
| _bashd_search_indexed "$search_term" "$search_type" "$verbose" "$count_only" |
There was a problem hiding this comment.
The count_only parameter is parsed from command-line options and passed to _bashd_search_indexed, but it is never actually used in the function implementation. Either implement the count-only functionality or remove this option from the help text and code.
| find "$path" -name "$pattern" -type f 2>/dev/null | while read -r file; do | ||
| local relative="${file#$repo_root/}" | ||
| local name=$(basename "$file") | ||
| local dir=$(basename "$(dirname "$file")") | ||
| echo " ✓ $relative" | ||
| echo " Category: $dir" | ||
| ((count++)) | ||
| done |
There was a problem hiding this comment.
The count variable is incremented inside a subshell (the while read loop created by the pipe), so the value will always be 0 when checked at line 309. Use process substitution instead: while read -r file; do ... done < <(find ...)
| find "$path" -name "$pattern" -type f 2>/dev/null | while read -r file; do | |
| local relative="${file#$repo_root/}" | |
| local name=$(basename "$file") | |
| local dir=$(basename "$(dirname "$file")") | |
| echo " ✓ $relative" | |
| echo " Category: $dir" | |
| ((count++)) | |
| done | |
| while read -r file; do | |
| local relative="${file#$repo_root/}" | |
| local name=$(basename "$file") | |
| local dir=$(basename "$(dirname "$file")") | |
| echo " ✓ $relative" | |
| echo " Category: $dir" | |
| ((count++)) | |
| done < <(find "$path" -name "$pattern" -type f 2>/dev/null) |
| ((count++)) | ||
| done | ||
| fi | ||
| done | ||
|
|
||
| if [[ $count -eq 0 ]]; then |
There was a problem hiding this comment.
The count variable is incremented inside a subshell (the while read loop created by the pipe), so the value will always be 0 when checked at line 309. Use process substitution instead: while read -r file; do ... done < <(find ...)
| # Search history for these functions | ||
| for func in $all_funcs; do | ||
| local usage_count=$(grep -c "^${func}" ~/.bash_history 2>/dev/null || echo 0) | ||
| if [[ "$usage_count" -gt 0 ]]; then | ||
| echo "$usage_count $func" | ||
| fi | ||
| done | sort -rn | head -n "$count" | while read -r count func; do |
There was a problem hiding this comment.
This searches the entire bash history file for each function name (potentially 2500+ grep operations). This will be very slow for large history files. Consider using a single pass with grep -f or inverting the approach by extracting all function names from history first, then matching.
| # Search history for these functions | |
| for func in $all_funcs; do | |
| local usage_count=$(grep -c "^${func}" ~/.bash_history 2>/dev/null || echo 0) | |
| if [[ "$usage_count" -gt 0 ]]; then | |
| echo "$usage_count $func" | |
| fi | |
| done | sort -rn | head -n "$count" | while read -r count func; do | |
| # Search history for these functions in a single pass | |
| awk 'NR==FNR { names[$1]; next } | |
| { | |
| cmd = $1 | |
| if (cmd in names) { | |
| counts[cmd]++ | |
| } | |
| } | |
| END { | |
| for (f in counts) { | |
| print counts[f], f | |
| } | |
| }' \ | |
| <(printf "%s\n" $all_funcs) ~/.bash_history 2>/dev/null | \ | |
| sort -rn | head -n "$count" | while read -r count func; do |
| # NOTES: Creates index in $BASHD_HOME/.index/ | ||
| # AUTHOR: bash.d project | ||
| # VERSION: 2.0.0 | ||
| # CREATED: $(date +'%Y-%m-%d') |
There was a problem hiding this comment.
The CREATED field uses command substitution $(date +'%Y-%m-%d') which will be evaluated when the file is sourced, not when it was originally created. This should either be a static date or removed, as it doesn't represent the actual creation date of the file.
| # CREATED: $(date +'%Y-%m-%d') | |
| # CREATED: 2024-01-01 |
| # shellcheck disable=SC2086 | ||
| grep -r $grep_opts -C "$context" "$pattern" "${repo_root}/bash_functions.d" \ |
There was a problem hiding this comment.
Disabling shellcheck SC2086 (unquoted variable expansion) for $grep_opts can lead to word splitting issues if the variable contains spaces or special characters. Consider using an array instead: local grep_opts_array=(-n) and then grep -r \"${grep_opts_array[@]}\", or ensure BASHD_GREP_OPTS is properly validated.
User description
The bash.d repository lacked organized indexing and searchability for its 2500+ functions, aliases, and scripts across 30+ categories. Users had no efficient way to locate, sort, or navigate repository content.
Implementation
Core Index System
jqfor proper JSON escaping and validationSearch Methods (5 commands)
bashd_search- Unified search across names, descriptions, categories, usagebashd_find- Pattern matching with wildcards (docker*,*network*)bashd_locate- Fast exact-name lookup using indexbashd_fuzzy- Interactive search with fzf and live previewbashd_grep- Content search with context linesOrganization & Navigation
bashd_sort- Sort by name, size, date, category, line countbashd_describe- Rich metadata displaybashd_next/prev/first/last- Navigate search resultsbashd_recent/popular- Recently modified/used functionsIntegration
Example Usage
Technical Details
Files Added
bash_functions.d/core/indexer.sh- Index builderbash_functions.d/core/search.sh- Search commandsbash_functions.d/core/utilities.sh- Organization utilitiesbash_functions.d/core/search_help.sh- Help systembash_functions.d/core/search_system_loader.sh- Auto-loadercompletions/search_completions.bash- Tab completionsdocs/SEARCH_SYSTEM.md- User documentationOriginal prompt
💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.
PR Type
Enhancement
Description
Implement comprehensive search and index system for bash.d repository
Add 5 search methods with different use cases
bashd_searchfor unified search,bashd_findfor patterns,bashd_locatefor exact namesbashd_fuzzyfor interactive fzf-based search,bashd_grepfor content searchAdd utility functions for organization and navigation
bashd_sortby name/size/date/category,bashd_describefor metadata displaybashd_next/prev/first/lastfor result navigation,bashd_recent/popularfor usage trackingAdd session management and help system
bashd_saveandbashd_recall_sessionfor search context persistencebashd_helpand tab completions for all commandsDiagram Walkthrough
File Walkthrough
5 files
indexer.sh
Master indexer for bash.d repositorybash_functions.d/core/indexer.sh
function headers
requirements, versions
bashd_index_buildto create complete index from scratch in ~6seconds
bashd_index_updatefor incremental updates andbashd_index_statsfor statisticsjqfor proper JSON escaping and validation to prevent injectionattacks
search.sh
Five search methods for repository contentbash_functions.d/core/search.sh
bashd_searchprovides unified search across names, descriptions,categories, usage
bashd_findsupports pattern matching with wildcards (docker*,*network*)
bashd_locateprovides fast exact-name lookup using indexbashd_fuzzyoffers interactive search with fzf and live previewbashd_grepenables content search with configurable context linesutilities.sh
Organization, sorting, and navigation utilitiesbash_functions.d/core/utilities.sh
bashd_sortsorts by name, size, date, category, line count withasc/desc order
bashd_describedisplays rich metadata including version, author,requirements
bashd_next/prev/first/lastbrowse through searchresults
bashd_recentandbashd_populartrack recently modified and frequentlyused functions
bashd_saveandbashd_recall_sessionmanage search context persistencebashd_editprovides quick editing of functions with $EDITORsearch_system_loader.sh
Auto-loader for search system initializationbash_functions.d/core/search_system_loader.sh
search_completions.bash
Tab completion for search system commandscompletions/search_completions.bash
commands
bashd_describe, bashd_edit
filesystem
3 files
search_help.sh
Comprehensive help system for search commandsbash_functions.d/core/search_help.sh
bashd_helpdisplays overview of all available commands with examplesindex, sort, describe
SEARCH_SYSTEM.md
Complete user documentation for search systemdocs/SEARCH_SYSTEM.md
criteria, pattern matching
options
README.md
Update README with search system documentationREADME.md
1 files
.gitignore
Ignore generated index files.gitignore