Skip to content

Commit 363bc93

Browse files
sfc-gh-ggengclaude
andauthored
[SNOW-3249917] JDBC removal Step 11a: Replicate RestRequest and HTTP retry infrastructure (#1143)
* Update JDBC removal plan: add Step 11 (final cleanup) Step 10c/10c2 done (remove SFSession). After those, 6 JDBC imports + ~70 FQN references remain. Split final cleanup into Step 11: - 11a: Create HttpRequestHelper, replace 6 JDBC HTTP calls - 11b: Remove 47 FQN SnowflakeSQLException from throws clauses - 11c: Clean up remaining FQN refs (HttpClientSettingsKey, HttpProtocol, OCSPMode, SnowflakeUtil, setSessionlessProxyForAzure, AwsSdkGCPSigner) - 11d: Demote JDBC to test scope + remove shade rules Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * [SNOW-3249917] JDBC removal Step 11a: Replicate RestRequest and HTTP retry infrastructure Replicate the following JDBC classes (v3.25.1) verbatim into the ingest fileTransferAgent package: - ThrowingBiFunction (functional interface for retry callbacks) - OCSPErrorCode (OCSP error code enum) - SFOCSPException (OCSP exception) - URLUtil (URL validation and request ID extraction) - DecorrelatedJitterBackoff (already existed, now tracked) - RetryContext (retry state holder) - RetryContextManager (retry callback registration) - HttpExecutingContext (HTTP request execution state) - HttpExecutingContextBuilder (builder for HttpExecutingContext) - SnowflakeUseDPoPNonceException (DPoP nonce error) - RestRequest (1265-line HTTP request retry engine) Import swaps applied per the JDBC removal plan. HttpUtil references kept as FQN (net.snowflake.client.core.HttpUtil) pending future replication. SessionUtil.isNewRetryStrategyRequest kept as FQN. SnowflakeUtil.logResponseDetails inlined as a local method since it requires JDBC's SFLogger type. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1 parent c3f03a8 commit 363bc93

File tree

12 files changed

+2321
-22
lines changed

12 files changed

+2321
-22
lines changed

.plans/JDBC_REMOVAL_PLAN.md

Lines changed: 66 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -278,8 +278,12 @@ is replaced.
278278
| Step 9c — Swap telemetry imports | ✅ Open | #1131 |
279279
| Step 10a — Replicate SFException, ExecTimeTelemetryData, etc. | ✅ Open | #1132 |
280280
| Step 10b — Swap SFException imports | ✅ Open | #1134 |
281-
| Step 10c — Remove SFSession/SFBaseSession | ⬜ TODO ||
282-
| Step 10d — Demote JDBC to test scope | ⬜ TODO ||
281+
| Step 10c — Remove SFSession from storage stack | ✅ Open | #1135 |
282+
| Step 10c2 — Remove SFSession from exceptions + telemetry | ✅ Open | #1136 |
283+
| Step 11a — Replace JDBC HTTP calls with HttpRequestHelper | ⬜ TODO ||
284+
| Step 11b — Remove FQN SnowflakeSQLException from throws | ⬜ TODO ||
285+
| Step 11c — Clean up remaining FQN JDBC references | ⬜ TODO ||
286+
| Step 11d — Demote JDBC to test scope | ⬜ TODO ||
283287

284288
**Closed PRs:** #1117 (reverted 7b approach), #1122 (reverted 8c approach)
285289
**Other PRs:** #1118 (error/exception tests on master), #1133 (Maven retry config)
@@ -616,34 +620,74 @@ NOT swapped — they interact with JDBC's `RestRequest.executeWithRetries()`.
616620

617621
---
618622

619-
### Step 10c — Remove SFSession/SFBaseSession ⬜ TODO
623+
### Step 10c — Remove SFSession from storage stack ✅ Open (PR #1135)
620624

621-
SFSession/SFBaseSession are always null from ingest callers. Not feasible
622-
to replicate (1498+1404 lines, 156-class transitive closure = 40K lines).
623-
Need to remove these parameter types.
625+
**Done:** Remove SFSession/SFBaseSession parameters and dead session-based
626+
code from storage clients, interface, strategies, factory, agent, config.
627+
Session was always null from ingest callers. -336 lines.
624628

625629
---
626630

627-
### Step 10dDemote JDBC to test scope ⬜ TODO
631+
### Step 10c2Remove SFSession from exceptions + telemetry ✅ Open (PR #1136)
628632

629-
Remaining 27 JDBC imports after Step 10b (all unreplicable due to massive
630-
dependency chains):
631-
- `SFSession`/`SFBaseSession` (15) — parameter types, always null
632-
- `HttpUtil` (2) — GCS client + TelemetryClient
633-
- `RestRequest` (1) — GCS client
634-
- `SnowflakeConnectionV1` (1) — TelemetryClient session path
635-
- `SnowflakeSQLException` (JDBC's, 1) — TelemetryClient
636-
- `ExecTimeTelemetryData`/`HttpResponseContextDto` (2) — GCS client
637-
(replicated but interact with JDBC RestRequest)
638-
- IB `Telemetry`/`TelemetryField`/`TelemetryUtil` (3) —
639-
interact with session.getTelemetryClient()
640-
- `SFSession` in `SnowflakeSQLLoggedException` (2) — parameter types
633+
**Done:** Remove SFSession/SFBaseSession from SnowflakeSQLLoggedException
634+
(all 15 constructors) and TelemetryClient (session-based code). Remove IB
635+
telemetry dead code. Update all callers (~12 files). -339 lines.
641636

642-
Then:
637+
After 10c2: 6 JDBC imports remain + ~70 FQN JDBC references in throws/params.
638+
639+
---
640+
641+
### Step 11a — Replace JDBC HTTP calls with HttpRequestHelper ⬜ TODO
642+
643+
Create `HttpRequestHelper` utility with retry logic (replaces JDBC's
644+
`RestRequest.executeWithRetries` and `HttpUtil.executeGeneralRequest`).
645+
Replace the 6 remaining JDBC imports:
646+
647+
- `TelemetryClient`: replace `HttpUtil.executeGeneralRequest()` +
648+
`SnowflakeSQLException` catch
649+
- `SnowflakeGCSClient`: replace `HttpUtil.getHttpClientWithoutDecompression()`,
650+
`HttpUtil.getHttpClient()`, `RestRequest.executeWithRetries()`,
651+
`HttpUtil.getSocketTimeout()`. Remove `ExecTimeTelemetryData`,
652+
`HttpResponseContextDto`, `RestRequest` imports.
653+
654+
---
655+
656+
### Step 11b — Remove FQN SnowflakeSQLException from throws ⬜ TODO
657+
658+
Mechanical removal of `, net.snowflake.client.jdbc.SnowflakeSQLException`
659+
from ~47 throws clauses across all replicated storage clients, interface,
660+
strategies, factory, and GCS client.
661+
662+
---
663+
664+
### Step 11c — Clean up remaining FQN JDBC references ⬜ TODO
665+
666+
Swap remaining FQN JDBC type references to ingest versions:
667+
- `net.snowflake.client.core.HttpClientSettingsKey``HttpClientSettingsKey`
668+
(same package, 8 occurrences in S3HttpUtil, SnowflakeFileTransferAgent,
669+
SnowflakeGCSClient, SnowflakeStorageClient)
670+
- `net.snowflake.client.core.HttpProtocol``HttpProtocol` (same package,
671+
1 occurrence in S3HttpUtil)
672+
- `net.snowflake.client.core.OCSPMode``net.snowflake.ingest.utils.OCSPMode`
673+
(2 occurrences in SnowflakeFileTransferAgent)
674+
- `net.snowflake.client.jdbc.SnowflakeUtil.convertProxyPropertiesToHttpClientKey`
675+
`StorageClientUtil.convertProxyPropertiesToHttpClientKey` (2 occurrences
676+
in SnowflakeFileTransferAgent)
677+
- `static import net.snowflake.client.core.HttpUtil.setSessionlessProxyForAzure`
678+
→ replicate method in StorageClientUtil (4 occurrences in SnowflakeAzureClient)
679+
- `net.snowflake.client.jdbc.cloud.storage.AwsSdkGCPSigner` — JDBC class
680+
reference in GCSAccessStrategyAwsSdk (string constant + class reference,
681+
3 occurrences)
682+
683+
---
684+
685+
### Step 11d — Demote JDBC to test scope ⬜ TODO
686+
687+
After all FQN references are cleaned up:
643688
1. Demote `snowflake-jdbc-thin` to `test` scope in `pom.xml`
644689
2. Remove JDBC shade relocation rules from Maven Shade plugin
645-
3. Remove `snowflake-jdbc-thin` from `public_pom.xml`
646-
4. Run full test suite
690+
3. Run full test suite
647691

648692
---
649693

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
/*
2+
* Replicated from snowflake-jdbc (v3.25.1)
3+
* Source: https://github.com/snowflakedb/snowflake-jdbc/blob/v3.25.1/src/main/java/net/snowflake/client/util/DecorrelatedJitterBackoff.java
4+
*
5+
* Permitted differences: package.
6+
*/
7+
package net.snowflake.ingest.streaming.internal.fileTransferAgent;
8+
9+
import java.util.concurrent.ThreadLocalRandom;
10+
11+
/**
12+
* Decorrelated Jitter backoff
13+
*
14+
* <p>https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/
15+
*/
16+
public class DecorrelatedJitterBackoff {
17+
private final long base;
18+
private final long cap;
19+
20+
public DecorrelatedJitterBackoff(long base, long cap) {
21+
this.base = base;
22+
this.cap = cap;
23+
}
24+
25+
public long nextSleepTime(long sleep) {
26+
long correctedSleep = sleep <= base ? base + 1 : sleep;
27+
return Math.min(cap, ThreadLocalRandom.current().nextLong(base, correctedSleep));
28+
}
29+
30+
public long getJitterForLogin(long currentTime) {
31+
double multiplicationFactor = chooseRandom(-1, 1);
32+
long jitter = (long) (multiplicationFactor * currentTime * 0.5);
33+
return jitter;
34+
}
35+
36+
public double chooseRandom(double min, double max) {
37+
return min + (Math.random() * (max - min));
38+
}
39+
}

0 commit comments

Comments
 (0)