Improve Installer Build Dependency Handling #313
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…CI configurations. Update README with build instructions for ARM and Intel. Modify CMakeLists to handle architecture-specific package names. Improve dependency management in build script for architecture detection.
…work installation, improving reliability. Remove SDKROOT references and streamline environment variable setup for deployment targets.
…lds to use aqtinstall with consistent Python versioning. Add verification steps for aqtinstall installation.
…lds to use Python virtual environments. Upgrade pip and streamline aqtinstall installation verification.
…ty and consistency. Update quotes in command strings and replace 'let' with 'const' where applicable.
- Fix Windows virtual environment activation path (Scripts/activate vs bin/activate) - Fix macOS Qt Installer Framework PATH configuration (remove trailing slash) - Fix ESLint error in modalBclController.js (use Object.prototype.hasOwnProperty.call) - Add CMAKE_OSX_ARCHITECTURES environment variable for macOS builds - Remove corrupted OpenStudio-server archive to force re-download These changes should resolve the build failures seen in workflow run 16181273377.
Skip pip upgrade on Windows to avoid permission error when trying to upgrade pip.exe while it's being used in the virtual environment. The default pip version is sufficient for installing aqtinstall. Fixes error: [WinError 5] Access is denied: venv\scripts\pip.exe
Add a pre-build cleanup step that removes any existing OpenStudio-server files that might be corrupted from previous builds. This prevents the 'incorrect header check' error that occurs when trying to extract corrupted tar.gz files. The cleanup step runs before the build and ensures a fresh download of dependencies.
- Improved cleanup step to test all gzip files for corruption and remove them - Added comprehensive cleanup of temp files and partial downloads - Added pre-download verification step specifically for macOS ARM OpenStudio-server - Added retry mechanism for corrupted downloads with integrity verification - Added detailed logging to debug download and extraction issues This should resolve the persistent 'incorrect header check' error by ensuring only valid, uncorrupted files are used in the build process.
- Changes ARM64 OpenStudio-server dependency to use x64 version - Resolves 'incorrect header check' error during dependency extraction - Temporary fix while ARM64 OpenStudio-server file is being debugged
- Update tasks/build.js to use MATRIX_ARCH environment variable for proper architecture detection - Add better error handling and logging for download and extraction processes - Fix extractDeps function to properly handle Promise-based extraction - Update workflow to export MATRIX_ARCH environment variable for macOS builds - Verify file existence before attempting extraction to prevent zlib errors
- Remove redundant pre-download step that was causing file corruption - Improve environment variable handling for MATRIX_ARCH - Add better error handling and debugging in build.js - Skip downloading files that already exist and are valid - Add file integrity checks before extraction
…ntegrity checks - Remove redundant pre-download step that was causing file corruption - Improve file integrity checks in build.js to test entire file instead of just first 1KB - Add better error messages for debugging - Ensure MATRIX_ARCH environment variable is properly set for Node.js scripts
- Updated tasks/build.js to use the new fs-jetpack API
- Removed deprecated {size: true} option from jetpack.inspect() call
- Fixed variable naming conflict with fileInfo/fileStats
- This resolves the Ubuntu workflow failure: 'Unknown argument options.size passed to inspect'
…download timeout for dependencies
- Comment out AWS credentials configuration step that was causing failures - Add test step to verify S3 bucket accessibility without authentication - This allows the build to proceed without requiring AWS secrets - Dependencies should be accessible from public S3 bucket
- Remove ARM64 OpenStudio entry from manifest due to S3 access issues (403 error) - Add comprehensive file validation in build.js: - Check HTTP status codes before download - Validate file sizes to detect error pages (< 1KB) - Enhanced integrity checks with better error messages - Improve Qt Installer Framework installation: - Better path detection for version-specific subdirectories - Enhanced verification of binarycreator availability - Automatic PATH correction in macOS build step - Add robust error handling for corrupted downloads This resolves the "incorrect header check" errors for ARM64 builds and "Cannot find QtIFW compiler binarycreator" errors for Intel builds. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Fixed "TypeError: source.once is not a function" by restoring stream-based architecture while preserving enhanced error handling for HTTP status codes and file size validation. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Exclude development and coordination files from version control: - .claude/ directory (Claude Code configuration) - .hive-mind/ and .swarm/ directories (coordination files) - claude-flow executables and scripts - hive-mind-prompt-*.txt files - CLAUDE.md configuration file 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Include Claude Flow memory storage directory to prevent tracking of session data, agent coordination files, and runtime state. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Removed unnecessary debugging code while preserving core ARM64 functionality: - Removed AWS dependency accessibility test step - Removed dependency cleanup step (corruption resolved by improved download logic) - Simplified Qt Installer Framework installation (kept aqtinstall approach) - Simplified Windows build step back to cmd shell - Removed excessive PATH verification for macOS - Removed TROUBLESHOOTING-MACOS.md development documentation Core ARM64 functionality retained: - macOS-ARM build matrix with proper architecture detection - Architecture-specific cmake configuration - Improved Qt Installer Framework installation via aqtinstall - Architecture-aware dependency fallback in build.js 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Removed download robustness enhancements that go beyond ARM64 scope: - File integrity checking before/after download - Enhanced HTTP error handling and file size validation - Increased timeout from 5s to 30s - Extensive logging and corruption detection - Promise-based error handling improvements Retained only essential ARM64 changes: - Architecture detection: MATRIX_ARCH > CMAKE_OSX_ARCHITECTURES > os.arch() - Architecture-aware dependency lookup with x64 fallback - Updated extractDeps and cleanDeps to use architecture logic The removed enhancements should be submitted as a separate PR focused on "Enhanced dependency download robustness". 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Reverted JavaScript app code quality fixes that are unrelated to ARM64: - Arrow function conversions and const/let changes - Escape sequence and quote style fixes - hasOwnProperty and trailing space cleanups Kept .gitignore additions for development tools. Fixed manifest.json typo: "openstudio" -> "OpenStudio" for Linux entry. This focuses the PR purely on macOS ARM64 installer functionality. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Reverted all files in app/ directory to develop branch state: - app/app/analysis/analysisController.js - app/app/bcl/modalBclController.js - app/app/project/osServerService.js - app/app/project/projectService.js - app/app/reports/reportsController.js These were code quality/linting fixes unrelated to ARM64 support and should be in a separate PR focused on code improvements. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Applied GitHub Copilot suggestions to improve code quality: - Added getActualFileInfo() helper function to eliminate duplicate logic - Reduced repeated arch fallback code from 3 functions to 1 helper - Simplified downloadDeps(), extractDeps(), and cleanDeps() functions - Consistent variable naming throughout Benefits: - 30+ lines of duplicate code eliminated - Single source of truth for manifest lookup logic - Easier maintenance - changes only needed in one place - Cleaner, more focused ARM64 implementation 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
… for macOS and Windows
- Enhanced error handling for dependency downloads with detailed logging - Added file integrity checks with MD5 checksums and size validation - Improved progress reporting with download speeds and expected file sizes - Added retry mechanism wrapper for critical operations - Increased download timeout from 5s to 30s for better reliability - Enhanced extraction logging with better error messages and debugging info - Added User-Agent header for better download tracking These improvements provide better debugging capabilities and more reliable dependency management for installer builds across all platforms. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR enhances the dependency download and extraction process for installer builds with better error handling, debugging capabilities, and reliability improvements.
Changes Made
Benefits
Testing
Related Issues
These improvements address common dependency download issues that can cause builds to fail, particularly useful for troubleshooting the ARM64 macOS builds and other platform-specific dependency problems.
🤖 Generated with Claude Code