fix: add shell pipe support with security validation

[vitali87]

Summary

• Adds support for piped commands (e.g., find . -name "*.c" | wc -l)
• Validates ALL commands in a pipeline against the allowlist (not just the first one)
• Blocks subshell execution ($(...) and backticks) for security
• Adds common pipe-friendly commands to allowlist: wc, head, tail, sort, uniq, cut, tr, xargs, awk, sed, tee

Changes

• shell_command.py: Switch from create_subprocess_exec() to create_subprocess_shell() with proper validation
• config.py: Add pipe-friendly commands to allowlist
• constants.py: Add shell pipe operators and subshell patterns
• tool_errors.py: Add subshell rejection error message

Test plan

• Unit tests for _extract_commands() function
• Unit tests for _has_subshell() function
• Unit tests for piped command execution
• Integration tests for various pipe scenarios
• Security tests for disallowed commands in pipes
• Security tests for subshell rejection

Fixes #207

[gemini-code-assist]

Summary of Changes

Hello @vitali87, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the shell command execution functionality by introducing support for piped commands while simultaneously bolstering security measures. It ensures that all commands within a pipeline are validated against an allowlist and explicitly prevents potentially dangerous subshell executions. This change broadens the utility of the shell command tool without compromising system integrity.

Highlights

• Piped Command Support: The shell command execution now supports piped commands (e.g., cmd1 | cmd2), allowing for more complex and flexible command sequences.
• Enhanced Security Validation: All commands within a pipeline are now individually validated against the allowlist, ensuring that every component of a piped command adheres to security policies.
• Subshell Execution Blocked: Subshell execution using $(...) or backticks (`) is explicitly blocked to prevent potential security vulnerabilities.
• Expanded Allowlist: Several common pipe-friendly commands, including wc, head, tail, sort, uniq, cut, tr, xargs, awk, sed, and tee, have been added to the shell command allowlist.
• Subprocess Execution Method Change: The underlying subprocess execution mechanism has been switched from create_subprocess_exec() to create_subprocess_shell() to enable native shell features like pipes, while robust security checks are maintained.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request adds valuable support for shell pipelines, including comprehensive validation and security checks. The changes are well-tested with new unit and integration tests. However, I've identified a critical security vulnerability where the 'dangerous command' check (rm -rf) is bypassed for commands within a pipeline. This could allow for unintended destructive operations. I've provided a detailed comment with a suggested fix to ensure every command in a pipeline is properly vetted. Addressing this issue is crucial for the security of this feature.

[vitali87]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces support for piped shell commands, which is a great feature. The implementation correctly identifies the need for security validation by checking for subshells and validating all commands in a pipeline against an allowlist.

However, I've identified several critical security vulnerabilities related to command parsing and the choice of allowlisted commands.

1. The allowlisting of powerful commands like xargs, awk, and sed creates a command injection vulnerability because their arguments are not validated.
2. Malformed command segments can bypass both the allowlist check and the dangerous command check due to how parsing errors are currently handled.

I've provided detailed comments with suggestions to fix these critical issues. Additionally, I've included a medium-severity comment to improve the regex used for splitting piped commands. Addressing these security concerns is crucial before this change is merged.

[vitali87]

/gemini review

[gemini-code-assist]

Code Review

This pull request significantly enhances the shell command execution security by introducing a comprehensive set of dangerous command checks and expanding the allowlist for common Unix utilities. Key changes include adding bandit for security scanning, extending the SHELL_COMMAND_ALLOWLIST in config.py with commands like wc, head, tail, sort, uniq, cut, tr, xargs, awk, sed, and tee. New constants were added in constants.py to define SHELL_DANGEROUS_COMMANDS, SHELL_RM_DANGEROUS_FLAGS, and a detailed list of SHELL_DANGEROUS_PATTERNS using regex to detect malicious commands or patterns (e.g., rm -rf /, dd to device, remote script execution, awk system() calls, subshell patterns). The core logic in shell_command.py was refactored to parse complex shell commands with pipes and logical operators, check for subshell execution, validate all commands in a pipeline against the allowlist, and apply the new dangerous command and pattern checks. The execute method now uses asyncio.create_subprocess_shell to support these features. Extensive new unit and integration tests were added to cover these security validations and piped command execution scenarios. Review comments highlighted that the awk pipe regex r"awk\s+.*\|" in SHELL_DANGEROUS_PATTERNS is too broad and could cause false positives by blocking legitimate pipelines, suggesting its removal. Another comment noted redundant logic in _check_dangerous_patterns, as it's called on both the full command and individual segments, recommending a refactoring to use distinct pattern sets for clarity and efficiency.

[vitali87]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces support for piped shell commands, which is a significant feature enhancement. The implementation shows a strong focus on security, with multiple layers of validation including command allowlisting, subshell blocking, and dangerous pattern matching using regular expressions. The accompanying tests are thorough and cover a wide range of scenarios, including many security-related edge cases. My review includes a critical fix for the pre-commit configuration, a suggestion to harden one of the security regex patterns, and a recommendation to properly acknowledge a security scanner warning for maintainability. Overall, this is a well-executed and security-conscious implementation.

[gemini-code-assist]

The specified revision 1.9.2 for bandit does not exist in the PyCQA/bandit repository. This will cause pre-commit install to fail. Please use a valid tag, such as the latest version 1.7.8.

    rev: 1.7.8

[gemini-code-assist]

This regex to detect rm on a system directory is too restrictive and can be easily bypassed. The pattern /[a-z]+ will not match common system directories like /usr, /etc, or any directory with uppercase letters or numbers. I recommend using a more explicit pattern that lists common Unix/Linux system directories for better protection.

Suggested change

	(r"rm\s+.*-[rf]+\s+/[a-z]+($|\s)", "rm with system directory"),
	(r"rm\s+.*-[rf]+\s+/(bin|boot|dev|etc|home|lib|lib64|media|mnt|opt|proc|root|run|sbin|srv|sys|tmp|usr|var)($|/|\s)", "rm with system directory"),

[gemini-code-assist]

Using create_subprocess_shell will be flagged by security scanners like Bandit (rule B602). Since you've implemented extensive sanitization before this call, it's good practice to explicitly acknowledge and suppress this warning for future maintainers by adding a # nosec B602 comment. This signals that the use of shell=True is intentional and has been secured.

Suggested change

	process = await asyncio.create_subprocess_shell(
	command,
	stdout=asyncio.subprocess.PIPE,
	stderr=asyncio.subprocess.PIPE,
	cwd=self.project_root,
	process = await asyncio.create_subprocess_shell( # nosec B602
	command,
	stdout=asyncio.subprocess.PIPE,
	stderr=asyncio.subprocess.PIPE,
	cwd=self.project_root,