Support for External Read-Only Data Stores

[berggren]

Feature Request: Support for External Read-Only Data Stores

Problem

The current architecture relies on data stored on a shared filesystem for both read and write. It is common to store evidence data in a separate read-only location. Duplicating this data for use within the application is inefficient, resource-intensive, and introduces potential synchronization issues.

Proposed Solution

A new feature to integrate with these existing external data stores directly. This would allow the application to read from a specific, pre-defined paths without requiring data migration.

The mapping from the VFS (virtual filesystem in the SQL database) File object and the physical file on disk is managed as a hybrid_property in the File ORM model:

    @hybrid_property
    def path(self):
        """Returns the full path of the file."""
        filename = self.uuid.hex
        if self.extension:
            filename = f"{filename}.{self.extension}"
        return os.path.join(self.folder.path, filename)

This is how the system know where to find the file. The reason this is a hybrid_property is that early on in the development of openrelik the ida of having multiple datastores was floated around. This implementation was put in place in anticipation of that. We can use this for this feature.

Add a field to the model, something like external_storage_name that points to a configuration for the specific external storage path. This to support chaning the base mount point without changing all file mappings in the database.

Example storage configuration:

case_smb_storage {
  "mount_point": "/mnt/cases"
}

And add another field, external_relative_path that specifies the full path from the base mount point.
Example: /mnt/smb/case/123/image.dd

Extend the path() property with to return the $external_storage_name.mount_point/$external_relative_path.

Mapping of files to the VFS

We still need to map the files to VFS entries in the database. Instead of uploading a file from the UI or programatically via the API we need to provide the user with an option to find the file from the external storage path and then create the database object with the appropriate fields set (see above).