[SNOW-3249917] JDBC removal Step 11a: Replicate RestRequest and HTTP retry infrastructure

.plans/JDBC_REMOVAL_PLAN.md

@@ -12,6 +12,8 @@ eliminate the JDBC dependency entirely.
 
 **This project is a pure mechanical migration. There must be no functional changes.**
 
+- **Always replicate** — no matter how many dependencies, replicate the JDBC original
+  verbatim. Never create "simplified replacements" or "minimal viable alternatives."
 - Copy JDBC classes verbatim into this repo. Compare against the actual source file
   in the JDBC repo (not decompiled output). Do not refactor, simplify, rename fields,
   change method signatures, strip comments/Javadoc, or improve logic — even where
@@ -278,8 +280,12 @@ is replaced.
 | Step 9c — Swap telemetry imports | ✅ Open | #1131 |
 | Step 10a — Replicate SFException, ExecTimeTelemetryData, etc. | ✅ Open | #1132 |
 | Step 10b — Swap SFException imports | ✅ Open | #1134 |
-| Step 10c — Remove SFSession/SFBaseSession | ⬜ TODO | — |
-| Step 10d — Demote JDBC to test scope | ⬜ TODO | — |
+| Step 10c — Remove SFSession from storage stack | ✅ Open | #1135 |
+| Step 10c2 — Remove SFSession from exceptions + telemetry | ✅ Open | #1136 |
+| Step 11a — Replace JDBC HTTP calls with HttpRequestHelper | ⬜ TODO | — |
+| Step 11b — Remove FQN SnowflakeSQLException from throws | ⬜ TODO | — |
+| Step 11c — Clean up remaining FQN JDBC references | ⬜ TODO | — |
+| Step 11d — Demote JDBC to test scope | ⬜ TODO | — |
 
 **Closed PRs:** #1117 (reverted 7b approach), #1122 (reverted 8c approach)
 **Other PRs:** #1118 (error/exception tests on master), #1133 (Maven retry config)
@@ -616,34 +622,74 @@ NOT swapped — they interact with JDBC's `RestRequest.executeWithRetries()`.
 
 ---
 
-### Step 10c — Remove SFSession/SFBaseSession ⬜ TODO
+### Step 10c — Remove SFSession from storage stack ✅ Open (PR #1135)
 
-SFSession/SFBaseSession are always null from ingest callers. Not feasible
-to replicate (1498+1404 lines, 156-class transitive closure = 40K lines).
-Need to remove these parameter types.
+**Done:** Remove SFSession/SFBaseSession parameters and dead session-based
+code from storage clients, interface, strategies, factory, agent, config.
+Session was always null from ingest callers. -336 lines.
 
 ---
 
-### Step 10d — Demote JDBC to test scope ⬜ TODO
+### Step 10c2 — Remove SFSession from exceptions + telemetry ✅ Open (PR #1136)
 
-Remaining 27 JDBC imports after Step 10b (all unreplicable due to massive
-dependency chains):
-- `SFSession`/`SFBaseSession` (15) — parameter types, always null
-- `HttpUtil` (2) — GCS client + TelemetryClient
-- `RestRequest` (1) — GCS client
-- `SnowflakeConnectionV1` (1) — TelemetryClient session path
-- `SnowflakeSQLException` (JDBC's, 1) — TelemetryClient
-- `ExecTimeTelemetryData`/`HttpResponseContextDto` (2) — GCS client
-  (replicated but interact with JDBC RestRequest)
-- IB `Telemetry`/`TelemetryField`/`TelemetryUtil` (3) —
-  interact with session.getTelemetryClient()
-- `SFSession` in `SnowflakeSQLLoggedException` (2) — parameter types
+**Done:** Remove SFSession/SFBaseSession from SnowflakeSQLLoggedException
+(all 15 constructors) and TelemetryClient (session-based code). Remove IB
+telemetry dead code. Update all callers (~12 files). -339 lines.
 
-Then:
+After 10c2: 6 JDBC imports remain + ~70 FQN JDBC references in throws/params.
+
+---
+
+### Step 11a — Replace JDBC HTTP calls with HttpRequestHelper ⬜ TODO
+
+Create `HttpRequestHelper` utility with retry logic (replaces JDBC's
+`RestRequest.executeWithRetries` and `HttpUtil.executeGeneralRequest`).
+Replace the 6 remaining JDBC imports:
+
+- `TelemetryClient`: replace `HttpUtil.executeGeneralRequest()` +
+  `SnowflakeSQLException` catch
+- `SnowflakeGCSClient`: replace `HttpUtil.getHttpClientWithoutDecompression()`,
+  `HttpUtil.getHttpClient()`, `RestRequest.executeWithRetries()`,
+  `HttpUtil.getSocketTimeout()`. Remove `ExecTimeTelemetryData`,
+  `HttpResponseContextDto`, `RestRequest` imports.
+
+---
+
+### Step 11b — Remove FQN SnowflakeSQLException from throws ⬜ TODO
+
+Mechanical removal of `, net.snowflake.client.jdbc.SnowflakeSQLException`
+from ~47 throws clauses across all replicated storage clients, interface,
+strategies, factory, and GCS client.
+
+---
+
+### Step 11c — Clean up remaining FQN JDBC references ⬜ TODO
+
+Swap remaining FQN JDBC type references to ingest versions:
+- `net.snowflake.client.core.HttpClientSettingsKey` → `HttpClientSettingsKey`
+  (same package, 8 occurrences in S3HttpUtil, SnowflakeFileTransferAgent,
+  SnowflakeGCSClient, SnowflakeStorageClient)
+- `net.snowflake.client.core.HttpProtocol` → `HttpProtocol` (same package,
+  1 occurrence in S3HttpUtil)
+- `net.snowflake.client.core.OCSPMode` → `net.snowflake.ingest.utils.OCSPMode`
+  (2 occurrences in SnowflakeFileTransferAgent)
+- `net.snowflake.client.jdbc.SnowflakeUtil.convertProxyPropertiesToHttpClientKey`
+  → `StorageClientUtil.convertProxyPropertiesToHttpClientKey` (2 occurrences
+  in SnowflakeFileTransferAgent)
+- `static import net.snowflake.client.core.HttpUtil.setSessionlessProxyForAzure`
+  → replicate method in StorageClientUtil (4 occurrences in SnowflakeAzureClient)
+- `net.snowflake.client.jdbc.cloud.storage.AwsSdkGCPSigner` — JDBC class
+  reference in GCSAccessStrategyAwsSdk (string constant + class reference,
+  3 occurrences)
+
+---
+
+### Step 11d — Demote JDBC to test scope ⬜ TODO
+
+After all FQN references are cleaned up:
 1. Demote `snowflake-jdbc-thin` to `test` scope in `pom.xml`
 2. Remove JDBC shade relocation rules from Maven Shade plugin
-3. Remove `snowflake-jdbc-thin` from `public_pom.xml`
-4. Run full test suite
+3. Run full test suite
 
 ---
 

pom.xml

@@ -887,11 +887,6 @@
       <groupId>net.minidev</groupId>
       <artifactId>json-smart</artifactId>
     </dependency>
-    <!-- Snowflake JDBC used to connect to the service-->
-    <dependency>
-      <groupId>net.snowflake</groupId>
-      <artifactId>snowflake-jdbc-thin</artifactId>
-    </dependency>
     <dependency>
       <groupId>org.apache.commons</groupId>
       <artifactId>commons-lang3</artifactId>
@@ -1017,6 +1012,12 @@
       <artifactId>junit</artifactId>
       <scope>test</scope>
     </dependency>
+    <!-- Snowflake JDBC — test scope only (used by TestUtils for IT result verification) -->
+    <dependency>
+      <groupId>net.snowflake</groupId>
+      <artifactId>snowflake-jdbc-thin</artifactId>
+      <scope>test</scope>
+    </dependency>
     <dependency>
       <groupId>org.assertj</groupId>
       <artifactId>assertj-core</artifactId>
@@ -1514,14 +1515,7 @@
                   <pattern>com.nimbusds</pattern>
                   <shadedPattern>${shadeBase}.com.nimbusds</shadedPattern>
                 </relocation>
-                <relocation>
-                  <pattern>net.snowflake.client</pattern>
-                  <shadedPattern>${shadeBase}.net.snowflake.client</shadedPattern>
-                </relocation>
-                <relocation>
-                  <pattern>com.snowflake.client.jdbc</pattern>
-                  <shadedPattern>${shadeBase}.com.snowflake.client.jdbc</shadedPattern>
-                </relocation>
+                <!-- JDBC shade relocations removed — JDBC is now test-scope only -->
                 <relocation>
                   <pattern>org.bouncycastle</pattern>
                   <shadedPattern>${shadeBase}.org.bouncycastle</shadedPattern>

src/main/java/net/snowflake/ingest/connection/telemetry/TelemetryClient.java

@@ -11,9 +11,9 @@
 import java.util.LinkedList;
 import java.util.Objects;
 import java.util.concurrent.Future;
-import net.snowflake.client.core.HttpUtil;
-import net.snowflake.client.jdbc.SnowflakeSQLException;
+import net.snowflake.ingest.streaming.internal.fileTransferAgent.JdbcHttpUtil;
 import net.snowflake.ingest.streaming.internal.fileTransferAgent.ObjectMapperFactory;
+import net.snowflake.ingest.streaming.internal.fileTransferAgent.SnowflakeSQLException;
 import net.snowflake.ingest.streaming.internal.fileTransferAgent.TelemetryThreadPool;
 import net.snowflake.ingest.streaming.internal.fileTransferAgent.log.SFLogger;
 import net.snowflake.ingest.streaming.internal.fileTransferAgent.log.SFLoggerFactory;
@@ -275,11 +275,11 @@ private boolean sendBatch() throws IOException {
 
       try {
         response =
-            HttpUtil.executeGeneralRequest(
+            JdbcHttpUtil.executeGeneralRequest(
                 post,
                 TELEMETRY_HTTP_RETRY_TIMEOUT_IN_SEC,
                 0,
-                (int) HttpUtil.getSocketTimeout().toMillis(),
+                (int) JdbcHttpUtil.getSocketTimeout().toMillis(),
                 0,
                 this.httpClient);
         stopwatch.stop();

src/main/java/net/snowflake/ingest/streaming/internal/fileTransferAgent/AttributeEnhancingHttpRequestRetryHandler.java

@@ -0,0 +1,32 @@
+/*
+ * Replicated from snowflake-jdbc (v3.25.1)
+ * Source: https://github.com/snowflakedb/snowflake-jdbc/blob/v3.25.1/src/main/java/net/snowflake/client/core/AttributeEnhancingHttpRequestRetryHandler.java
+ *
+ * Permitted differences: package declaration.
+ */
+package net.snowflake.ingest.streaming.internal.fileTransferAgent;
+
+import java.io.IOException;
+import org.apache.http.impl.client.DefaultHttpRequestRetryHandler;
+import org.apache.http.protocol.HttpContext;
+
+/**
+ * Extends {@link DefaultHttpRequestRetryHandler} to store the current execution count (attempt
+ * number) in the {@link HttpContext}. This allows interceptors to identify retry attempts.
+ *
+ * <p>The execution count is stored using the key defined by {@link #EXECUTION_COUNT_ATTRIBUTE}.
+ */
+class AttributeEnhancingHttpRequestRetryHandler extends DefaultHttpRequestRetryHandler {
+  /**
+   * The key used to store the current execution count (attempt number) in the {@link HttpContext}.
+   * Interceptors can use this key to retrieve the count. The value stored will be an {@link
+   * Integer}.
+   */
+  static final String EXECUTION_COUNT_ATTRIBUTE = "net.snowflake.client.core.execution-count";
+
+  @Override
+  public boolean retryRequest(IOException exception, int executionCount, HttpContext context) {
+    context.setAttribute(EXECUTION_COUNT_ATTRIBUTE, executionCount);
+    return super.retryRequest(exception, executionCount, context);
+  }
+}

src/main/java/net/snowflake/ingest/streaming/internal/fileTransferAgent/AwsSdkGCPSigner.java

@@ -0,0 +1,56 @@
+/*
+ * Replicated from snowflake-jdbc (v3.25.1)
+ * Source: https://github.com/snowflakedb/snowflake-jdbc/blob/v3.25.1/src/main/java/net/snowflake/client/jdbc/cloud/storage/AwsSdkGCPSigner.java
+ *
+ * Permitted differences: package. @SnowflakeJdbcInternalApi removed.
+ */
+package net.snowflake.ingest.streaming.internal.fileTransferAgent;
+
+import com.amazonaws.SignableRequest;
+import com.amazonaws.auth.AWS4Signer;
+import com.amazonaws.auth.AWSCredentials;
+import com.amazonaws.http.HttpMethodName;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+public class AwsSdkGCPSigner extends AWS4Signer {
+  private static final Map<String, String> headerMap =
+      new HashMap<String, String>() {
+        {
+          put("x-amz-storage-class", "x-goog-storage-class");
+          put("x-amz-acl", "x-goog-acl");
+          put("x-amz-date", "x-goog-date");
+          put("x-amz-copy-source", "x-goog-copy-source");
+          put("x-amz-metadata-directive", "x-goog-metadata-directive");
+          put("x-amz-copy-source-if-match", "x-goog-copy-source-if-match");
+          put("x-amz-copy-source-if-none-match", "x-goog-copy-source-if-none-match");
+          put("x-amz-copy-source-if-unmodified-since", "x-goog-copy-source-if-unmodified-since");
+          put("x-amz-copy-source-if-modified-since", "x-goog-copy-source-if-modified-since");
+        }
+      };
+
+  @Override
+  public void sign(SignableRequest<?> request, AWSCredentials credentials) {
+    if (credentials.getAWSAccessKeyId() != null && !"".equals(credentials.getAWSAccessKeyId())) {
+      request.addHeader("Authorization", "Bearer " + credentials.getAWSAccessKeyId());
+    }
+
+    if (request.getHttpMethod() == HttpMethodName.GET) {
+      request.addHeader("Accept-Encoding", "gzip,deflate");
+    }
+
+    Map<String, String> headerCopy =
+        request.getHeaders().entrySet().stream()
+            .collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue));
+
+    for (Map.Entry<String, String> entry : headerCopy.entrySet()) {
+      String entryKey = entry.getKey().toLowerCase();
+      if (headerMap.containsKey(entryKey)) {
+        request.addHeader(headerMap.get(entryKey), entry.getValue());
+      } else if (entryKey.startsWith("x-amz-meta-")) {
+        request.addHeader(entryKey.replace("x-amz-meta-", "x-goog-meta-"), entry.getValue());
+      }
+    }
+  }
+}

src/main/java/net/snowflake/ingest/streaming/internal/fileTransferAgent/Constants.java

@@ -0,0 +1,66 @@
+/*
+ * Replicated from snowflake-jdbc (v3.25.1)
+ * Source: https://github.com/snowflakedb/snowflake-jdbc/blob/v3.25.1/src/main/java/net/snowflake/client/core/Constants.java
+ *
+ * Permitted differences: package declaration,
+ * SnowflakeUtil.systemGetProperty -> StorageClientUtil.systemGetProperty.
+ */
+package net.snowflake.ingest.streaming.internal.fileTransferAgent;
+
+import static net.snowflake.ingest.streaming.internal.fileTransferAgent.StorageClientUtil.systemGetProperty;
+
+/*
+ * Constants used in JDBC implementation
+ */
+public final class Constants {
+  // Session expired error code as returned from Snowflake
+  public static final int SESSION_EXPIRED_GS_CODE = 390112;
+
+  // Cloud storage credentials expired error code
+  public static final int CLOUD_STORAGE_CREDENTIALS_EXPIRED = 240001;
+
+  // Session gone error code as returned from Snowflake
+  public static final int SESSION_GONE = 390111;
+
+  // Error code for all invalid id token cases during login request
+  public static final int ID_TOKEN_INVALID_LOGIN_REQUEST_GS_CODE = 390195;
+
+  public static final int OAUTH_ACCESS_TOKEN_EXPIRED_GS_CODE = 390318;
+
+  public static final int OAUTH_ACCESS_TOKEN_INVALID_GS_CODE = 390303;
+
+  // Error message for IOException when no space is left for GET
+  public static final String NO_SPACE_LEFT_ON_DEVICE_ERR = "No space left on device";
+
+  public enum OS {
+    WINDOWS,
+    LINUX,
+    MAC,
+    SOLARIS
+  }
+
+  private static OS os = null;
+
+  public static synchronized OS getOS() {
+    if (os == null) {
+      String operSys = systemGetProperty("os.name").toLowerCase();
+      if (operSys.contains("win")) {
+        os = OS.WINDOWS;
+      } else if (operSys.contains("nix") || operSys.contains("nux") || operSys.contains("aix")) {
+        os = OS.LINUX;
+      } else if (operSys.contains("mac")) {
+        os = OS.MAC;
+      } else if (operSys.contains("sunos")) {
+        os = OS.SOLARIS;
+      }
+    }
+    return os;
+  }
+
+  public static void clearOSForTesting() {
+    os = null;
+  }
+
+  public static final int MB = 1024 * 1024;
+  public static final long GB = 1024 * 1024 * 1024;
+}

src/main/java/net/snowflake/ingest/streaming/internal/fileTransferAgent/DecorrelatedJitterBackoff.java

@@ -0,0 +1,39 @@
+/*
+ * Replicated from snowflake-jdbc (v3.25.1)
+ * Source: https://github.com/snowflakedb/snowflake-jdbc/blob/v3.25.1/src/main/java/net/snowflake/client/util/DecorrelatedJitterBackoff.java
+ *
+ * Permitted differences: package.
+ */
+package net.snowflake.ingest.streaming.internal.fileTransferAgent;
+
+import java.util.concurrent.ThreadLocalRandom;
+
+/**
+ * Decorrelated Jitter backoff
+ *
+ * <p>https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/
+ */
+public class DecorrelatedJitterBackoff {
+  private final long base;
+  private final long cap;
+
+  public DecorrelatedJitterBackoff(long base, long cap) {
+    this.base = base;
+    this.cap = cap;
+  }
+
+  public long nextSleepTime(long sleep) {
+    long correctedSleep = sleep <= base ? base + 1 : sleep;
+    return Math.min(cap, ThreadLocalRandom.current().nextLong(base, correctedSleep));
+  }
+
+  public long getJitterForLogin(long currentTime) {
+    double multiplicationFactor = chooseRandom(-1, 1);
+    long jitter = (long) (multiplicationFactor * currentTime * 0.5);
+    return jitter;
+  }
+
+  public double chooseRandom(double min, double max) {
+    return min + (Math.random() * (max - min));
+  }
+}