Skip to content

Commit 97d355d

Browse files
sfc-gh-ggengclaude
sfc-gh-ggeng
and
claude
authored
[SNOW-3249917] JDBC removal Step 11a (part 2): Replicate HttpUtil, swap callers (#1144)
* [SNOW-3249917] JDBC removal Step 11a (part 2): Replicate HttpUtil, swap callers to replicated REST infrastructure Replicate JDBC's HttpUtil as JdbcHttpUtil (renamed to avoid collision with ingest's existing net.snowflake.ingest.utils.HttpUtil). Also replicate SnowflakeMutableProxyRoutePlanner and AttributeEnhancingHttpRequestRetryHandler as small helper classes. Key changes: - JdbcHttpUtil: verbatim replication of JDBC HttpUtil with import swaps. SFTrustManager replaced with null (ingest does not use OCSP trust manager). SFSSLConnectionSocketFactory -> IngestSSLConnectionSocketFactory. SystemUtil.convertSystemPropertyToIntValue inlined. SessionUtil.isNewRetryStrategyRequest inlined as static method. Deprecated S3 proxy wrapper methods omitted (S3HttpUtil callable directly). - RestRequest: all FQN net.snowflake.client.core.HttpUtil references replaced with JdbcHttpUtil. SessionUtil.isNewRetryStrategyRequest replaced with JdbcHttpUtil.isNewRetryStrategyRequest. - TelemetryClient: HttpUtil.executeGeneralRequest and HttpUtil.getSocketTimeout replaced with JdbcHttpUtil equivalents. SnowflakeSQLException import swapped to ingest's replicated version. - SnowflakeGCSClient: HttpUtil.getHttpClient, getHttpClientWithoutDecompression, getSocketTimeout replaced with JdbcHttpUtil equivalents. Added toIngestKey adapter to convert JDBC's HttpClientSettingsKey during transition. - SnowflakeAzureClient: setSessionlessProxyForAzure import swapped from JDBC HttpUtil to JdbcHttpUtil. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add SFTrustManager and dependencies to avoid temporary OCSP gap Replicate SFTrustManager (1689 lines) and its dependencies so JdbcHttpUtil uses the real OCSP trust manager from the start, rather than temporarily stubbing it to null. New files: SFTrustManager, Constants, FileCacheManager, HexUtil, OCSPTelemetryData. Updated: JdbcHttpUtil (use real SFTrustManager), FileUtil (add methods needed by FileCacheManager), IngestSSLConnectionSocketFactory, S3HttpUtil, StorageClientUtil. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add bcutil-jdk18on as direct dependency for SFTrustManager SFTrustManager uses BouncyCastle's bcutil (ASN1, OCSP types). It was a transitive dependency but maven-dependency-plugin:analyze-only requires it to be declared explicitly. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 363bc93 commit 97d355d

18 files changed

Lines changed: 3814 additions & 52 deletions

‎.plans/JDBC_REMOVAL_PLAN.md‎

Lines changed: 15 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,8 @@ eliminate the JDBC dependency entirely.
1212

1313
**This project is a pure mechanical migration. There must be no functional changes.**
1414

15+
- **Always replicate** — no matter how many dependencies, replicate the JDBC original
16+
verbatim. Never create "simplified replacements" or "minimal viable alternatives."
1517
- Copy JDBC classes verbatim into this repo. Compare against the actual source file
1618
in the JDBC repo (not decompiled output). Do not refactor, simplify, rename fields,
1719
change method signatures, strip comments/Javadoc, or improve logic — even where
@@ -253,7 +255,7 @@ is replaced.
253255

254256
---
255257

256-
### Progress Summary (updated 2026-04-01)
258+
### Progress Summary (updated 2026-04-06)
257259

258260
| Step | Status | PR |
259261
|---|---|---|
@@ -274,18 +276,20 @@ is replaced.
274276
| Step 8e — Swap ALL imports (pure import swaps) | ✅ MERGED | #1127 |
275277
| Step 9a — Mechanical import swaps in replicated classes | ✅ MERGED | #1128 |
276278
| Step 9b-prep — Replicate OOB telemetry deps | ✅ MERGED | #1129 |
277-
| Step 9b — Replicate TelemetryEvent + TelemetryService | ✅ Open | #1130 |
278-
| Step 9c — Swap telemetry imports | ✅ Open | #1131 |
279-
| Step 10a — Replicate SFException, ExecTimeTelemetryData, etc. | ✅ Open | #1132 |
280-
| Step 10b — Swap SFException imports | ✅ Open | #1134 |
279+
| Step 9b — Replicate TelemetryEvent + TelemetryService | ✅ MERGED | #1130 |
280+
| Step 9c — Swap telemetry imports | ✅ MERGED | #1131 |
281+
| Step 10a — Replicate SFException, ExecTimeTelemetryData, etc. | ✅ MERGED | #1132 |
282+
| Step 10b — Swap SFException imports | ✅ MERGED | #1134 |
281283
| Step 10c — Remove SFSession from storage stack | ✅ Open | #1135 |
282284
| Step 10c2 — Remove SFSession from exceptions + telemetry | ✅ Open | #1136 |
283-
| Step 11a — Replace JDBC HTTP calls with HttpRequestHelper | ⬜ TODO | — |
284-
| Step 11b — Remove FQN SnowflakeSQLException from throws | ⬜ TODO | — |
285-
| Step 11c — Clean up remaining FQN JDBC references | ⬜ TODO | — |
286-
| Step 11d — Demote JDBC to test scope | ⬜ TODO | — |
287-
288-
**Closed PRs:** #1117 (reverted 7b approach), #1122 (reverted 8c approach)
285+
| Step 11a — Replicate RestRequest and HTTP retry infrastructure | ✅ Open | #1143 |
286+
| Step 11a-p2 — Replicate HttpUtil, swap callers | ✅ Open | #1144 |
287+
| Step 11b — Clean up remaining FQN JDBC references | ✅ Open | #1145 |
288+
| Step 11d — Demote JDBC to test scope | ✅ Open | #1146 |
289+
290+
**Closed PRs:** #1117 (reverted 7b approach), #1122 (reverted 8c approach),
291+
#1137 (superseded 10d), #1138 (superseded 11a), #1139-#1141 (superseded 11b-d),
292+
#1142 (combined 11a-d, split into #1143-#1146)
289293
**Other PRs:** #1118 (error/exception tests on master), #1133 (Maven retry config)
290294

291295
---

‎pom.xml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -985,6 +985,10 @@
985985
<groupId>org.bouncycastle</groupId>
986986
<artifactId>bcprov-jdk18on</artifactId>
987987
</dependency>
988+
<dependency>
989+
<groupId>org.bouncycastle</groupId>
990+
<artifactId>bcutil-jdk18on</artifactId>
991+
</dependency>
988992
<dependency>
989993
<groupId>org.slf4j</groupId>
990994
<artifactId>slf4j-api</artifactId>

‎src/main/java/net/snowflake/ingest/connection/telemetry/TelemetryClient.java‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -11,9 +11,9 @@
1111
import java.util.LinkedList;
1212
import java.util.Objects;
1313
import java.util.concurrent.Future;
14-
import net.snowflake.client.core.HttpUtil;
15-
import net.snowflake.client.jdbc.SnowflakeSQLException;
14+
import net.snowflake.ingest.streaming.internal.fileTransferAgent.JdbcHttpUtil;
1615
import net.snowflake.ingest.streaming.internal.fileTransferAgent.ObjectMapperFactory;
16+
import net.snowflake.ingest.streaming.internal.fileTransferAgent.SnowflakeSQLException;
1717
import net.snowflake.ingest.streaming.internal.fileTransferAgent.TelemetryThreadPool;
1818
import net.snowflake.ingest.streaming.internal.fileTransferAgent.log.SFLogger;
1919
import net.snowflake.ingest.streaming.internal.fileTransferAgent.log.SFLoggerFactory;
@@ -275,11 +275,11 @@ private boolean sendBatch() throws IOException {
275275

276276
try {
277277
response =
278-
HttpUtil.executeGeneralRequest(
278+
JdbcHttpUtil.executeGeneralRequest(
279279
post,
280280
TELEMETRY_HTTP_RETRY_TIMEOUT_IN_SEC,
281281
0,
282-
(int) HttpUtil.getSocketTimeout().toMillis(),
282+
(int) JdbcHttpUtil.getSocketTimeout().toMillis(),
283283
0,
284284
this.httpClient);
285285
stopwatch.stop();

‎src/main/java/net/snowflake/ingest/streaming/internal/fileTransferAgent/AttributeEnhancingHttpRequestRetryHandler.java‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
/*
2+
* Replicated from snowflake-jdbc (v3.25.1)
3+
* Source: https://github.com/snowflakedb/snowflake-jdbc/blob/v3.25.1/src/main/java/net/snowflake/client/core/AttributeEnhancingHttpRequestRetryHandler.java
4+
*
5+
* Permitted differences: package declaration.
6+
*/
7+
package net.snowflake.ingest.streaming.internal.fileTransferAgent;
8+
9+
import java.io.IOException;
10+
import org.apache.http.impl.client.DefaultHttpRequestRetryHandler;
11+
import org.apache.http.protocol.HttpContext;
12+
13+
/**
14+
* Extends {@link DefaultHttpRequestRetryHandler} to store the current execution count (attempt
15+
* number) in the {@link HttpContext}. This allows interceptors to identify retry attempts.
16+
*
17+
* <p>The execution count is stored using the key defined by {@link #EXECUTION_COUNT_ATTRIBUTE}.
18+
*/
19+
class AttributeEnhancingHttpRequestRetryHandler extends DefaultHttpRequestRetryHandler {
20+
/**
21+
* The key used to store the current execution count (attempt number) in the {@link HttpContext}.
22+
* Interceptors can use this key to retrieve the count. The value stored will be an {@link
23+
* Integer}.
24+
*/
25+
static final String EXECUTION_COUNT_ATTRIBUTE = "net.snowflake.client.core.execution-count";
26+
27+
@Override
28+
public boolean retryRequest(IOException exception, int executionCount, HttpContext context) {
29+
context.setAttribute(EXECUTION_COUNT_ATTRIBUTE, executionCount);
30+
return super.retryRequest(exception, executionCount, context);
31+
}
32+
}

‎src/main/java/net/snowflake/ingest/streaming/internal/fileTransferAgent/Constants.java‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
/*
2+
* Replicated from snowflake-jdbc (v3.25.1)
3+
* Source: https://github.com/snowflakedb/snowflake-jdbc/blob/v3.25.1/src/main/java/net/snowflake/client/core/Constants.java
4+
*
5+
* Permitted differences: package declaration,
6+
* SnowflakeUtil.systemGetProperty -> StorageClientUtil.systemGetProperty.
7+
*/
8+
package net.snowflake.ingest.streaming.internal.fileTransferAgent;
9+
10+
import static net.snowflake.ingest.streaming.internal.fileTransferAgent.StorageClientUtil.systemGetProperty;
11+
12+
/*
13+
* Constants used in JDBC implementation
14+
*/
15+
public final class Constants {
16+
// Session expired error code as returned from Snowflake
17+
public static final int SESSION_EXPIRED_GS_CODE = 390112;
18+
19+
// Cloud storage credentials expired error code
20+
public static final int CLOUD_STORAGE_CREDENTIALS_EXPIRED = 240001;
21+
22+
// Session gone error code as returned from Snowflake
23+
public static final int SESSION_GONE = 390111;
24+
25+
// Error code for all invalid id token cases during login request
26+
public static final int ID_TOKEN_INVALID_LOGIN_REQUEST_GS_CODE = 390195;
27+
28+
public static final int OAUTH_ACCESS_TOKEN_EXPIRED_GS_CODE = 390318;
29+
30+
public static final int OAUTH_ACCESS_TOKEN_INVALID_GS_CODE = 390303;
31+
32+
// Error message for IOException when no space is left for GET
33+
public static final String NO_SPACE_LEFT_ON_DEVICE_ERR = "No space left on device";
34+
35+
public enum OS {
36+
WINDOWS,
37+
LINUX,
38+
MAC,
39+
SOLARIS
40+
}
41+
42+
private static OS os = null;
43+
44+
public static synchronized OS getOS() {
45+
if (os == null) {
46+
String operSys = systemGetProperty("os.name").toLowerCase();
47+
if (operSys.contains("win")) {
48+
os = OS.WINDOWS;
49+
} else if (operSys.contains("nix") || operSys.contains("nux") || operSys.contains("aix")) {
50+
os = OS.LINUX;
51+
} else if (operSys.contains("mac")) {
52+
os = OS.MAC;
53+
} else if (operSys.contains("sunos")) {
54+
os = OS.SOLARIS;
55+
}
56+
}
57+
return os;
58+
}
59+
60+
public static void clearOSForTesting() {
61+
os = null;
62+
}
63+
64+
public static final int MB = 1024 * 1024;
65+
public static final long GB = 1024 * 1024 * 1024;
66+
}

0 commit comments

Comments
 (0)