[Refactor]: SWE-agent integration and chat template fix #442
Conversation
Summary of ChangesHello @Ratish1, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly refactors the experimental SWE-agent integration, aiming to simplify its environment setup and enhance its operational stability. It achieves this by eliminating the Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request significantly refactors the SWE-agent example, simplifying the environment setup by removing the nemo-gym dependency and the need for a separate agent server. The introduction of asyncio.to_thread to run agent logic in a background thread is a great improvement for Ray cluster stability. The code changes are well-structured and the updated documentation is much clearer. I have a few minor suggestions to improve configurability and dependency management.
| cd miles/examples/swe-agent | ||
| python download_and_process_data.py --input SWE-Gym/SWE-Gym --output /root/swe_train.jsonl | ||
| # Install SWE-Gym harness | ||
| pip install "swegym @ git+https://github.com/sdevare-nv/nv-SWE-Bench-Package.git@31e1cb8f0241da1707d00faa633c3d6ce1a8ba3b" |
There was a problem hiding this comment.
Pinning to a specific commit hash is great for reproducibility. However, installing directly from a git URL in a command can be fragile. For better dependency management, consider adding this dependency to a requirements.txt file or managing it as a git submodule, similar to mini-swe-agent.
| step_limit=250, | ||
| collapse_limit=3, |
There was a problem hiding this comment.
The step_limit and collapse_limit for the DefaultAgent are hardcoded. It would be more flexible to make these configurable, for instance by passing them down from the args namespace. This would allow for easier experimentation with different agent behaviors without changing the code.
|
|
||
| # Environment configuration | ||
| image_name = get_swegym_docker_image_name(sample.metadata, subset) | ||
| output_dir = Path("results") / subset / model_name |
There was a problem hiding this comment.
The base output directory 'results' is hardcoded. Consider making this configurable via args to allow users to specify a different location for agent outputs, which can be useful for organizing experiments. For example: Path(args.swe_agent_output_dir).
|
Hey @yueming-yuan , I wanted to follow up on this in case you missed it. Thanks. |
1 participant
Description
This PR refactors the experimental swe-agent example to make environment setup significantly simpler and fix model reasoning collapse / infinite loops. #423
Key Changes
reducing setup complexity.