Document SQLite snapshot data mapping

README.md

@@ -40,6 +40,36 @@ UV and PV data are stored in the following keys:
 | pagePv | ZSet        | bsz:page_pv:md5(host) / md5(path) |
 | pageUv | HyperLogLog | bsz:site_uv:md5(host):md5(path)   |
 
+## SQLite persistence
+
+Busuanzi can periodically snapshot Redis data into a SQLite database in order to keep
+long term history or recover from Redis failures. The following configuration options
+control the behaviour (all values can also be provided via environment variables):
+
+| key | type | default | description |
+| --- | --- | --- | --- |
+| `Persistence.Enable` | bool | `false` | Enable background SQLite snapshots. |
+| `Persistence.DBPath` | string | `data/busuanzi.db` | Path to the SQLite database file. |
+| `Persistence.Interval` | number | `300` | Snapshot interval in seconds. |
+| `Persistence.RestoreOnStart` | bool | `false` | Restore the most recent snapshot during startup. |
+
+Snapshots include PV, UV and HyperLogLog states. Restoring a snapshot replaces the
+corresponding Redis keys using the stored values and TTL information.
+
+### Snapshot payload mapping
+
+Each Redis data type is normalised into the SQLite `snapshots` table as follows:
+
+| Redis key | Redis type | SQLite fields | Notes |
+| --- | --- | --- | --- |
+| `site_pv` | String counter | `count` | The plain integer PV value. |
+| `site_uv` | HyperLogLog | `count`, `payload` | `payload` stores the raw bytes returned by `DUMP`, encoded as hexadecimal. |
+| `page_pv` | ZSet | `count`, `payload` | `payload` is a JSON array of `{path_unique,count}` pairs so that the ranking can be restored. |
+| `page_uv` | HyperLogLog | `count`, `payload` | Same as `site_uv`. |
+
+The `ttl_ms` column stores the remaining key lifetime in milliseconds (or `0` for keys without an expiration) so that the Redis
+TTL semantics are preserved when restoring a snapshot.
+
 ## Data Migration
 
 - You can use the [busuanzi-sync](https://github.com/soxft/busuanzi-sync) tool to sync data from the [original busuanzi](http://busuanzi.ibruce.info) to the self-hosted busuanzi.

README.zh_CN.md

@@ -32,6 +32,34 @@
 | pagePv | ZSet        | bsz:page_pv:md5(host) / md5(path) |
 | pageUv | HyperLogLog | bsz:site_uv:md5(host):md5(path)   |
 
+## SQLite 持久化
+
+Busuanzi 可以按照指定的时间间隔，将 Redis 中的 PV、UV 与 HyperLogLog 数据自动快照到
+SQLite 数据库文件中，便于长期保存与快速恢复。所有配置项均支持使用环境变量覆盖：
+
+| 配置项 | 类型 | 默认值 | 说明 |
+| --- | --- | --- | --- |
+| `Persistence.Enable` | bool | `false` | 是否开启后台快照任务 |
+| `Persistence.DBPath` | string | `data/busuanzi.db` | SQLite 数据库文件路径 |
+| `Persistence.Interval` | number | `300` | 快照间隔（秒） |
+| `Persistence.RestoreOnStart` | bool | `false` | 启动时是否自动恢复最近一次快照 |
+
+快照会保存每个统计键当前的数值、序列化的 HyperLogLog 数据以及剩余 TTL。恢复快照时
+会用存储的数据覆盖 Redis 中对应的键。
+
+### 快照数据映射
+
+不同的 Redis 数据类型会按下表映射到 SQLite `snapshots` 表中：
+
+| Redis 键 | Redis 类型 | SQLite 字段 | 说明 |
+| --- | --- | --- | --- |
+| `site_pv` | 字符串计数器 | `count` | 直接保存站点 PV 的整数值。 |
+| `site_uv` | HyperLogLog | `count`, `payload` | `payload` 中保存 `DUMP` 返回的原始二进制，并以十六进制编码。 |
+| `page_pv` | ZSet | `count`, `payload` | `payload` 是 `{path_unique,count}` 组成的 JSON 数组，用于恢复排名明细。 |
+| `page_uv` | HyperLogLog | `count`, `payload` | 与 `site_uv` 相同的序列化方式。 |
+
+`ttl_ms` 字段会记录剩余的过期时间（毫秒，永不过期时为 `0`），以便在恢复时继续保持 Redis 的 TTL 语义。
+
 
 ## 其他
 

config.yaml

@@ -13,6 +13,11 @@ Redis:
   MaxActive: 100  # 最大连接数
   MinIdle: 25     # 最小空闲连接数
   MaxRetries: 3   # 最大重试次数
+Persistence:
+  Enable: false           # 是否开启持久化
+  DBPath: data/busuanzi.db # SQLite 数据库存储路径
+  Interval: 300           # 自动持久化间隔(秒)
+  RestoreOnStart: false   # 启动时是否自动从最近的快照恢复数据
 Bsz:
   Expire: 0        # 统计数据过期时间 单位秒, 请输入整数 (无任何访问, 超过这个时间后, 统计数据将被清空, 0为不过期)
   Secret: "bsz"    # JWT签名密钥 // 请设置为任意长度的随机值

main.go

@@ -1,8 +1,11 @@
 package main
 
 import (
+	"context"
+
 	"github.com/soxft/busuanzi/config"
 	"github.com/soxft/busuanzi/core"
+	"github.com/soxft/busuanzi/process/persist"
 	"github.com/soxft/busuanzi/process/redisutil"
 	"github.com/soxft/busuanzi/process/webutil"
 )
@@ -13,5 +16,7 @@ func main() {
 
 	core.InitExpire()
 
+	persist.Init(context.Background())
+
 	webutil.Init()
 }