Implement Cache Entry Expiration/Cleanup Mechanism for services/artifactcache

[aweris]

Description

The existing cache mechanism in the services/artifactcache, which was reverse-engineered to locally support actions/cache, does not currently have a mechanism for cache entry expiration or cleanup. Cache entries only get deleted when the entire dagger cache volume is deleted. This behavior may lead to potential space constraints and inefficient cache usage over time.

To align more closely with typical caching patterns and potentially with how caching works in GitHub Actions, it's essential to introduce a cache entry expiration/cleanup mechanism.

Current Behaviour

• Cache entries persist indefinitely until the entire dagger cache volume is deleted.

Desired Behaviour

• Cache entries should have a defined expiration time, after which they are automatically removed.
• Periodic cleanup processes or mechanisms to remove stale or unused cache entries.

Tasks

• Research how caching expiration works in GitHub Actions for potential alignment.
• Implement an automated cleanup mechanism that periodically checks and deletes expired cache entries.
• Extend unit and integration tests to validate the cache cleanup mechanism.

Optional:

• Provide tools or commands for users to manually clean up cache or view cache status.
• Define a default expiration duration for cache entries, allowing users to potentially configure this duration.
• Update relevant documentation to reflect the changes and guide users on best practices.