perf(push): add parallel HTTP requests and compression optimization #63

puzhen-ryan · 2026-01-29T10:17:29Z

Add parallel execution utility (ThreadPoolExecutor) for concurrent tasks
Add pigz-based parallel compression with automatic fallback to tarfile
Execute check_env() and state_environment() concurrently in push command
Run compression and signed URL fetch in parallel during upload
Add AENV_DISABLE_PARALLEL env var to disable all optimizations

Performance improvement: ~34% faster push time in real-world testing:

Version	Command	User Time	System Time	CPU	Total Time	Improvement
Optimized	`aenv push`	10.08s	2.00s	31%	38.94s	34% faster
Optimized (disabled)	`AENV_DISABLE_PARALLEL=1 aenv push`	19.78s	1.61s	45%	47.10s	21% faster
Original	`aenv push` (v0.1.4)	24.62s	1.99s	44%	59.29s	baseline

- Add parallel execution utility (ThreadPoolExecutor) for concurrent tasks - Add pigz-based parallel compression with automatic fallback to tarfile - Execute check_env() and state_environment() concurrently in push command - Run compression and signed URL fetch in parallel during upload - Add AENV_DISABLE_PARALLEL env var to disable all optimizations Performance improvement: ~34% faster push time in real-world testing Co-Authored-By: Claude (claude-opus-4-5) <noreply@anthropic.com>

gemini-code-assist · 2026-01-29T10:17:52Z

Summary of Changes

Hello @puzhen-ryan, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces significant performance enhancements to the aenv push command by leveraging parallel processing for both network requests and file compression. The changes aim to reduce the overall execution time by performing independent operations concurrently, leading to a faster and more efficient user experience. A configurable option is also provided to disable these optimizations if needed.

Highlights

Parallel Execution Utility: Introduced a new utility module (cli.utils.parallel) that provides a ThreadPoolExecutor-based function (parallel_execute) for running multiple tasks concurrently with robust error handling and a sequential fallback mechanism. This utility also includes a check for the AENV_DISABLE_PARALLEL environment variable to optionally disable parallel execution.
Optimized Compression with pigz: Added a new utility module (cli.utils.compression) that implements pigz-based parallel compression for archiving directories. This feature automatically falls back to standard tarfile compression if pigz is not available on the system or if parallel execution is disabled. It also includes logic to exclude specified patterns like __pycache__.
Concurrent push Command Operations: The push command in aenv/src/cli/cmds/push.py has been refactored to execute check_env() and state_environment() calls concurrently using the new parallel execution utility. This reduces the total time spent waiting for these independent API calls.
Parallel Upload Process: The AEnvHubStorage's upload method in aenv/src/cli/extends/storage/storage_manager.py now performs directory compression and fetching of signed upload URLs in parallel. This significantly speeds up the preparation phase before the actual file transfer.
Performance Gains: These optimizations have resulted in a measured performance improvement of approximately 34% faster push times in real-world testing, making the aenv push command more efficient.
Disable Parallelism Option: A new environment variable, AENV_DISABLE_PARALLEL, has been added, allowing users to easily disable all parallel execution and compression optimizations for debugging or specific use cases.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request introduces significant performance improvements to the push command by parallelizing network requests and compression. The implementation of parallel utilities for task execution and compression is well-done and includes fallbacks for robustness. The changes are accompanied by a good set of unit and integration tests.

My review focuses on a few areas for improvement:

Error handling in the parallel workflow to ensure safe operation.
A minor bug in a new test file.
A potential regression in flexibility in one of the modified classes.

Overall, this is a great enhancement. Addressing these points will make the implementation even more robust.

gemini-code-assist · 2026-01-29T10:19:25Z

aenv/src/cli/cmds/push.py

+        if state_result and state_result.success:
+            env_state = EnvStatus.parse_state(state_result.result)
+            if env_state.running() and not force:
+                click.echo("❌ Environment is being prepared, use --force to overwrite")
+                raise click.Abort()


If the state_env task fails, the check for a running environment is skipped. This could lead to unintentionally overwriting a running environment without the --force flag because the failure is handled silently. For safety, the operation should be aborted if the state of an existing environment cannot be determined.

Suggested change

if state_result and state_result.success:

env_state = EnvStatus.parse_state(state_result.result)

if env_state.running() and not force:

click.echo("❌ Environment is being prepared, use --force to overwrite")

raise click.Abort()

if state_result and state_result.success:

env_state = EnvStatus.parse_state(state_result.result)

if env_state.running() and not force:

click.echo("❌ Environment is being prepared, use --force to overwrite")

raise click.Abort()

else:

click.echo("❌ Could not determine the state of the existing environment. Aborting to prevent accidental overwrite.", err=True)

if state_result and state_result.error:

raise state_result.error

raise click.Abort()

gemini-code-assist · 2026-01-29T10:19:25Z

aenv/src/cli/tests/test_push_optimized.py

+        yield tmpdir
+
+
+from pathlib import Path


The import from pathlib import Path is located here, but Path is used before this line (at line 47). This will cause a NameError and prevent these tests from running. To fix this and follow Python conventions, all imports should be moved to the top of the file.

gemini-code-assist · 2026-01-29T10:19:25Z

aenv/src/cli/extends/storage/storage_manager.py

+            if archive_path and os.path.exists(archive_path):
+                os.unlink(archive_path)


The finally block correctly cleans up the temporary archive file. However, it's worth noting that os.unlink can raise an exception (e.g., PermissionError). While unlikely in this context, wrapping it in a try...except block would make the cleanup even more robust and prevent a cleanup failure from masking the original exception if one occurred in the try block.

Suggested change

if archive_path and os.path.exists(archive_path):

os.unlink(archive_path)

if archive_path and os.path.exists(archive_path):

try:

os.unlink(archive_path)

except OSError:

pass

gemini-code-assist · 2026-01-29T10:19:25Z

aenv/src/cli/utils/archive_tool.py

+        if self.use_parallel:
+            from cli.utils.compression import pack_directory_parallel
+
+            self.archive_path = pack_directory_parallel(
+                self.source_dir,
+                exclude_patterns=self.pack_kwargs.get("exclude_patterns"),
+                use_parallel=True,
+            )


When use_parallel=True, TempArchive only passes exclude_patterns to pack_directory_parallel. This is a reduction in flexibility compared to the non-parallel path, which passes all **self.pack_kwargs. Other arguments like compression_level would be ignored. While this works for the current usage, it makes the class's interface inconsistent and less reusable.

JacksonMei

lgtm

gemini-code-assist bot reviewed Jan 29, 2026

View reviewed changes

JacksonMei approved these changes Jan 30, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

perf(push): add parallel HTTP requests and compression optimization #63

perf(push): add parallel HTTP requests and compression optimization #63

Uh oh!

puzhen-ryan commented Jan 29, 2026

Uh oh!

gemini-code-assist bot commented Jan 29, 2026

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

gemini-code-assist bot Jan 29, 2026

Uh oh!

gemini-code-assist bot Jan 29, 2026

Uh oh!

gemini-code-assist bot Jan 29, 2026

Uh oh!

gemini-code-assist bot Jan 29, 2026

Uh oh!

JacksonMei left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		if archive_path and os.path.exists(archive_path):
		os.unlink(archive_path)

perf(push): add parallel HTTP requests and compression optimization #63

Are you sure you want to change the base?

perf(push): add parallel HTTP requests and compression optimization #63

Uh oh!

Conversation

puzhen-ryan commented Jan 29, 2026

Uh oh!

gemini-code-assist bot commented Jan 29, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist bot Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

JacksonMei left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants