Skip to content

Conversation

@SkySingh04
Copy link
Contributor

@SkySingh04 SkySingh04 commented Feb 20, 2025

TestGPT: PR Analysis with Linting & Static Analysis

Linked Issue : keploy/keploy#2532

Changes

  • Added automatic PR file analysis using GitHub API
  • Integrated language-specific linters:
    • Go: golangci-lint, gosec
    • JavaScript: eslint
    • Python: flake8, bandit
  • Unified reporting system combining test results and analysis
  • Added language auto-detection

Technical Details

  • Detects file extensions: .go, .js, .ts, .py, .java
  • Generates analysis reports in analysis-reports directory
  • Posts combined report as PR comment
  • Preserves existing Keploy test functionality

Testing

  • Keploy test execution
  • Language detection
  • Linter integration
  • Report generation
  • PR commenting

Signed-off-by: Sky Singh akashsingh2210670@gmail.com

Signed-off-by: Sky Singh <akashsingh2210670@gmail.com>
@SkySingh04
Copy link
Contributor Author

@Sonichigo Do let me know how we can test this / refine this!

Signed-off-by: Sky Singh <akashsingh2210670@gmail.com>
@SkySingh04
Copy link
Contributor Author

@Sonichigo @gouravkrosx Following up on this!

with:
github-token: ${{ inputs.github-token }}
script: |
try {
Copy link
Member

@khareyash05 khareyash05 Mar 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets not make the yml file cluttered, encouraging to use shell files for same

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure Yash! I also took the chance to move the parsing to a new script as well

action.yml Outdated
id: language-detection
shell: bash
run: |
declare -A extensions=(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it would be great to support jsx and tsx

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, Adding this!

action.yml Outdated
- name: Install analysis tools
shell: bash
run: |
for lang in ${{ steps.language-detection.outputs.languages }}; do
Copy link
Member

@khareyash05 khareyash05 Mar 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should also check if the system has a linter already installed and if yes we should use the same linter, might cause issues if using different linter

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I have added a check for it as well

Copy link
Member

@khareyash05 khareyash05 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please address the comments above

…rsing

Signed-off-by: Sky Singh <akashsingh2210670@gmail.com>
…esult parsing

Signed-off-by: Sky Singh <akashsingh2210670@gmail.com>
Signed-off-by: Sky Singh <akashsingh2210670@gmail.com>
@SkySingh04
Copy link
Contributor Author

@khareyash05 Your requested changes have been made!

@Hermione2408
Copy link
Member

@SkySingh04 please mention the issue in the PR

parse-results.sh Outdated
@@ -0,0 +1,11 @@
#!/bin/bash

grep -oE "COMPLETE TESTRUN SUMMARY\.\s+Total tests: [0-9]+" ${GITHUB_WORKSPACE}/${WORKDIR}/report.txt | sed -r "s/\x1B\[[0-9;]*[mGK]//g" > ${GITHUB_WORKSPACE}/${WORKDIR}/final_total_tests.out
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a fallback here, what if no tests were executed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a fallback message : No tests were executed!

action.yml Outdated

- name: Grant permissions to scripts
run: |
chmod +x ${GITHUB_ACTION_PATH}/install.sh
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Repeating chmod +x multiple times. Anyway we can batch update permissions in one step?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has been updated!

@SkySingh04
Copy link
Contributor Author

Thanks for the review @Hermione2408 , Will pick this up over the weekend!

…ing logic

- Combined multiple chmod commands into a single line for efficiency in action.yml.
- Added a check in parse-results.sh to handle cases where no tests were executed, ensuring proper output generation.

Signed-off-by: Sky Singh <akashsingh2210670@gmail.com>
…tory checks

- Added logging functionality to provide better insights during linter installations and code analysis completion in analyze-code.sh.
- Ensured the existence of the analysis-reports directory before generating reports.
- Improved error handling in parse-results.sh to check for the presence of the report file and handle cases where no tests were executed.

Signed-off-by: Sky Singh <akashsingh2210670@gmail.com>
@SkySingh04 SkySingh04 requested a review from Hermione2408 March 15, 2025 19:00
@SkySingh04
Copy link
Contributor Author

@Hermione2408 @khareyash05 Your requested changes have been made!

@SkySingh04 SkySingh04 requested a review from khareyash05 March 15, 2025 19:09
Copy link
Member

@Hermione2408 Hermione2408 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please address the comments

- Updated error messages to provide clearer guidance when the test report file is not found and when no valid test summary is present.
- Enhanced user feedback to ensure proper understanding of issues related to test execution.

Signed-off-by: Sky Singh <akashsingh2210670@gmail.com>
Signed-off-by: Sky Singh <akashsingh2210670@gmail.com>
@SkySingh04 SkySingh04 requested a review from Hermione2408 March 21, 2025 14:38
@SkySingh04
Copy link
Contributor Author

@Hermione2408 Your requested Changes have been made!

Copy link
Member

@Hermione2408 Hermione2408 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@keploy
Copy link

keploy bot commented Apr 15, 2025

Nice Pr 😄. We currently support Unit Test Generation only for Go projects.
Stay tuned – support for more languages is coming soon! 🚀

1 similar comment
@keploy-staging
Copy link

Nice Pr 😄. We currently support Unit Test Generation only for Go projects.
Stay tuned – support for more languages is coming soon! 🚀

@SkySingh04 SkySingh04 closed this Jul 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants