Skip to content

Add s3 url config#222

Open
githubzilla wants to merge 6 commits intoeloqdata:mainfrom
githubzilla:add_s3_url_config
Open

Add s3 url config#222
githubzilla wants to merge 6 commits intoeloqdata:mainfrom
githubzilla:add_s3_url_config

Conversation

@githubzilla
Copy link
Collaborator

@githubzilla githubzilla commented Oct 13, 2025

Summary by CodeRabbit

  • New Features

    • Added a configuration option to specify a TxLog RocksDB cloud S3 URL; when provided it populates cloud storage settings and takes precedence over legacy options.
  • Chores

    • Updated internal submodule references; no user-facing behavior changes.
  • Style

    • Minor comment/formatting cleanup with no functional impact.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Oct 13, 2025

Warning

Rate limit exceeded

@githubzilla has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 26 minutes and 46 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between d214eec and b612408.

📒 Files selected for processing (1)
  • store_handler (1 hunks)

Walkthrough

Adds a new txlog_rocksdb_cloud_s3_url command-line flag and an s3_url_ field in the TxLog RocksDB cloud config; InitTxLogService reads the flag (falls back to local config) and populates/overrides cloud config. Also advances store_handler and tx_service submodule pointers.

Changes

Cohort / File(s) Summary of Changes
TxLog S3 URL flag & cloud config
src/redis_service.cpp
Adds DEFINE_string(txlog_rocksdb_cloud_s3_url); InitTxLogService reads the flag (fallback to local config) and sets txlog_rocksdb_cloud_config.s3_url_ alongside existing fields (endpoint_url_, bucket_name_, bucket_prefix_/object_path_, region_); minor formatting/comment tweak.
Submodule update: store_handler
store_handler
Advances submodule pointer (commit updated). No functional/control-flow changes in this diff.
Submodule update: tx_service
tx_service
Advances submodule pointer (commit updated). No code/API changes observed.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  actor Proc as Process Startup
  participant RS as RedisService
  participant CFG as Flags / Local Config
  participant Parser as txlog::ParseS3Url
  participant LOG as Logger

  Proc->>RS: InitTxLogService()
  RS->>CFG: Read txlog_rocksdb_cloud_s3_url flag
  alt flag set
    RS->>Parser: Parse(s3_url_)
    Parser-->>RS: bucket, object_path, endpoint_url, region
    RS->>RS: Set txlog_rocksdb_cloud_config.s3_url_ and override fields
    RS->>LOG: Info: using URL-based TxLog config
  else flag empty
    RS->>CFG: Read legacy txlog_rocksdb_cloud_config fields
    RS->>RS: Use legacy cloud config values
  end
  RS-->>Proc: Continue initialization
Loading

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

Suggested reviewers

  • liunyl
  • thweetkomputer

Poem

A rabbit sniffed a tiny clue 🥕
It found a URL that led it through.
Buckets mapped and paths set right,
Config hopped in with quiet delight.
Submodules leapt — the patch is new.

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check ✅ Passed The PR title "Add s3 url config" directly and clearly relates to the main objective of the changeset. The primary changes in src/redis_service.cpp implement support for a new S3 URL configuration option through a command-line flag and extended cloud configuration handling. The title is concise, readable, and specific enough that a teammate reviewing the repository history would understand the core purpose: adding a new S3 URL configuration capability. The supporting submodule updates are maintenance in nature and not the primary focus.
Docstring Coverage ✅ Passed No functions found in the changes. Docstring coverage check skipped.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
src/redis_service.cpp (1)

2451-2451: Comment formatting nit

End-for comment spacing change only; no behavior impact.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between c8ddf31 and a86b287.

📒 Files selected for processing (2)
  • src/redis_service.cpp (4 hunks)
  • store_handler (1 hunks)
🔇 Additional comments (2)
src/redis_service.cpp (2)

300-307: New S3 URL flag looks good

Flag name, default, help text, and placement under LOG_STATE_TYPE_RKDB_CLOUD are consistent with existing options.


1753-1760: LGTM on sourcing the URL

Reads from flag with config fallback in "local" section, consistent with other txlog cloud options.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

♻️ Duplicate comments (1)
src/redis_service.cpp (1)

1709-1716: Parse only when non-empty; ensure precedence; never log the raw URL

Nice addition. Please ensure downstream code only parses/uses this when non-empty and that precedence over legacy fields is actually enforced, and that no logs print s3_url_ directly. This echoes an earlier review point.

Run to verify guards and logging:

#!/bin/bash
set -euo pipefail
# Where is the URL parsed?
rg -n -C3 -g '!**/build/**' -P '\bParseS3Url\s*\('
# Where is s3_url_ used and potentially logged?
rg -n -C3 -g '!**/build/**' -P 's3_url_'
rg -n -C2 -g '!**/build/**' -P 'LOG\\(.*s3_url_'
🧹 Nitpick comments (1)
src/redis_service.cpp (1)

300-307: Scope S3 URL flag to S3 builds; clarify help to discourage credentials

Expose this flag only when S3 is enabled to avoid confusing GCS-only builds, and make the help text explicit about precedence and not embedding credentials in the URL.

 #if defined(LOG_STATE_TYPE_RKDB_CLOUD)
 ...
-DEFINE_string(txlog_rocksdb_cloud_s3_url,
+#if defined(LOG_STATE_TYPE_RKDB_S3)
+DEFINE_string(txlog_rocksdb_cloud_s3_url,
               "",
-              "TxLog RocksDB cloud S3 URL. Format: s3://{bucket}/{path} or "
-              "http(s)://{host}:{port}/{bucket}/{path}. "
-              "Examples: s3://my-bucket/my-path, "
-              "http://localhost:9000/my-bucket/my-path. "
-              "This option takes precedence over legacy configuration options "
-              "if both are provided");
+              "TxLog RocksDB cloud S3 URL. Format: s3://{bucket}/{path} or "
+              "http(s)://{host}:{port}/{bucket}/{path}. Examples: "
+              "s3://my-bucket/my-path, http://localhost:9000/my-bucket/my-path. "
+              "Do NOT include credentials in the URL; prefer FLAGS_aws_* or env vars. "
+              "When non-empty, this overrides endpoint/bucket/bucket_prefix/object_path/region.")
+#endif
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between db12796 and 844c706.

📒 Files selected for processing (3)
  • src/redis_service.cpp (4 hunks)
  • store_handler (1 hunks)
  • tx_service (1 hunks)
✅ Files skipped from review due to trivial changes (1)
  • tx_service
🚧 Files skipped from review as they are similar to previous changes (1)
  • store_handler
🔇 Additional comments (1)
src/redis_service.cpp (1)

2363-2363: LGTM (formatting-only).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant