Skip to content

feat(memory): add Redis and OpenSearch long-term memory backends#49

Merged
sjy3 merged 1 commit intovolcengine:mainfrom
wucm667:feat/long-term-memory-redis-opensearch
Mar 10, 2026
Merged

feat(memory): add Redis and OpenSearch long-term memory backends#49
sjy3 merged 1 commit intovolcengine:mainfrom
wucm667:feat/long-term-memory-redis-opensearch

Conversation

@wucm667
Copy link
Contributor

@wucm667 wucm667 commented Mar 2, 2026

Summary

Add Redis (via Redis Stack / RediSearch) and OpenSearch backends for long-term memory in VeADK-Go, closing the feature gap with the Python SDK.

Both backends implement the existing LongTermMemoryBackend interface and use vector similarity search (KNN) backed by the ARK Embedding model, mirroring the VikingDB backend pattern.

Motivation

The Python SDK already provides redis and opensearch long-term memory backends (via LlamaIndex VectorStore). This PR brings parity to the Go SDK.

Changes

New files

File Description
memory/long_term_memory_backends/embedding_config.go Shared EmbeddingConfig struct that decouples embedding model setup from individual backends. Reads model name / API key / dimensions from global VeADKConfig by default, or accepts explicit values.
memory/long_term_memory_backends/redis_backend.go RedisMemoryBackend — stores memories as Redis Hash entries with a FLOAT32 vector field and retrieves them via FT.SEARCH KNN query (requires Redis Stack).
memory/long_term_memory_backends/redis_backend_test.go Unit tests for the Redis backend (mocked with github.com/bytedance/mockey).
memory/long_term_memory_backends/opensearch_backend.go OpenSearchMemoryBackend — creates per-user indices with knn_vector HNSW mapping and stores/retrieves memories via the OpenSearch REST API. Supports SSL/TLS with optional custom CA cert.
memory/long_term_memory_backends/opensearch_backend_test.go Unit tests for the OpenSearch backend.

Modified files

File Change
common/consts.go Add DATABASE_REDIS_* and DATABASE_OPENSEARCH_* env key constants
configs/database.go Add RedisConfig and OpenSearchConfig structs; map env vars in MapEnvToConfig
configs/configs.go Initialize Redis and OpenSearch config structs in SetupVeADKConfig
memory/long_term_memory.go Extend NewLongTermMemoryService factory with BackendLongTermRedis and BackendLongTermOpenSearch cases
memory/long_term_memory_test.go Add test cases for both new backends
go.mod / go.sum Add github.com/redis/go-redis/v9 v9.18.0

Usage

Redis backend

Requires Redis Stack (or Redis with the RediSearch module) which supports FT.CREATE / FT.SEARCH.

Via environment variables:

DATABASE_REDIS_HOST=127.0.0.1
DATABASE_REDIS_PORT=6379
DATABASE_REDIS_PASSWORD=secret
DATABASE_REDIS_DB=0

Via config.yaml:

database:
  redis:
    host: 127.0.0.1
    port: 6379
    password: secret
    db: 0

In code:

svc, err := memory.NewLongTermMemoryService(memory.BackendLongTermRedis, nil)
// or with explicit config:
svc, err := memory.NewLongTermMemoryService(memory.BackendLongTermRedis, &long_term_memory_backends.RedisMemoryConfig{
    Host:  "127.0.0.1",
    Port:  6379,
    Index: "my-agent-ltm",
    EmbeddingConfig: &long_term_memory_backends.EmbeddingConfig{
        ModelName:  "ep-xxx",
        APIKey:     "...",
        Dimensions: 2048,
    },
})

OpenSearch backend

Via environment variables:

DATABASE_OPENSEARCH_HOST=127.0.0.1
DATABASE_OPENSEARCH_PORT=9200
DATABASE_OPENSEARCH_USERNAME=admin
DATABASE_OPENSEARCH_PASSWORD=secret
DATABASE_OPENSEARCH_USE_SSL=false

Via config.yaml:

database:
  opensearch:
    host: 127.0.0.1
    port: 9200
    username: admin
    password: secret
    use_ssl: false

In code:

svc, err := memory.NewLongTermMemoryService(memory.BackendLongTermOpenSearch, nil)
// or with explicit config:
svc, err := memory.NewLongTermMemoryService(memory.BackendLongTermOpenSearch, &long_term_memory_backends.OpenSearchMemoryConfig{
    Host:     "127.0.0.1",
    Port:     9200,
    UseSSL:   true,
    CertPath: "/path/to/ca.pem",
    Index:    "my-agent-ltm",
})

Testing

No real databases are required — all external calls are mocked via github.com/bytedance/mockey.

# Run new backend tests
go test -v -race ./memory/long_term_memory_backends/...

# Run factory tests
go test -v -race ./memory/...

# Run all tests
go test -mod=readonly -v -race -count=1 -shuffle=on ./...

Notes

  • Both backends follow the same interface contract as the existing VikingDB backend (SaveMemory / SearchMemory).
  • Per-user data isolation is achieved via index-name suffixing (<index>_<userId>) for OpenSearch and key-prefix scoping (<index>:<userId>:<uuid>) for Redis.
  • The Redis backend requires Redis Stack >= 2.0 for vector search support.
  • The OpenSearch backend requires the knn plugin (included in OpenSearch >= 1.0 by default).
  • Follows all existing patterns: uses mockey for tests, utils.GetEnvWithDefault for config, and NewDefaultEmbeddingConfig for zero-config startup.

- Add RedisMemoryBackend using Redis Stack vector search (FT.SEARCH KNN)
- Add OpenSearchMemoryBackend using knn_vector mapping with HNSW
- Add shared EmbeddingConfig to decouple embedding model setup
- Add RedisConfig and OpenSearchConfig to configs/database.go
- Register DATABASE_REDIS_* and DATABASE_OPENSEARCH_* env key constants
- Extend NewLongTermMemoryService factory with redis and opensearch cases
- Add full unit test coverage for both backends using mockey
@sjy3
Copy link
Collaborator

sjy3 commented Mar 10, 2026

LGTM

@sjy3 sjy3 merged commit d7608fa into volcengine:main Mar 10, 2026
2 checks passed
@wucm667 wucm667 deleted the feat/long-term-memory-redis-opensearch branch March 10, 2026 12:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants