Skip to content

Conversation

@claymore666
Copy link

@claymore666 claymore666 commented Oct 4, 2025

Summary

Fixes #30

  • Recursively download all files from /data directory including subdirectories
  • Preserve directory structure locally (e.g., /data/logs/syslog./data/wt-xxx/logs/syslog)
  • Exclude /data/swap/ and /data/lost+found/ directories from downloads
  • Automatically create local subdirectories as needed

Changes

Recursive file discovery

  • Use find command to recursively discover all files in /data
  • Filter out swap and lost+found directories
  • Maintain relative paths for proper local directory structure

Download improvements

  • Calculate relative paths from /data to preserve directory structure
  • Create local subdirectories automatically with os.makedirs()
  • Hash comparison works with files in subdirectories

Test plan

  • Test downloading files from subdirectories: ceti whaletag -t wt-b827eb123456
  • Verify /data/logs/syslog files are downloaded to ./data/wt-xxx/logs/syslog
  • Verify swap and lost+found directories are excluded
  • Verify directory structure is preserved locally

Fixes bug where subdirectories like /data/logs/ were ignored during
whale tag data downloads. Now recursively downloads all files while
preserving directory structure.

Changes:
- Use find command instead of ls to discover files recursively
- Exclude /data/swap/ and /data/lost+found/ directories

Fixes: Project-CETI#30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

missing whale-tag data subfolder

1 participant