[🍒][PLUGIN-1847] Add SqlServerErrorDetailsProvider

[psainics]

🍒 [cherrypick]

Commits :

• 53cfd3a

PR:

• [PLUGIN-1847] Add SqlServerErrorDetailsProvider #539 [ Reviewer @itsankit-google ]

---

ErrorDetailsProvider - SqlServer [Source|Sink] plugin

Jira : PLUGIN-1847

Description

Implement Program Failure Exception Handling in SqlServer Source/Sink plugin to catch known errors

Code change

• Added SqlServerErrorDetailsProvider.java
• Modified DBUtils.java
• Modified SqlServerSink.java
• Modified SqlServerSource.java

Tests

• Test Case (Violate check constrain)

Raw Logs

2025-01-29 09:20:32,606 - ERROR [Executor task launch worker for task 0.0 in stage 0.0 (TID 0):o.a.s.u.Utils@98] - Aborting task
io.cdap.cdap.api.exception.WrappedStageException: Stage 'SQL Server4' encountered : io.cdap.cdap.api.exception.ProgramFailureException: Error occurred in the phase: 'Writing' with sqlState: '23000', errorCode: '547', errorMessage: The INSERT statement conflicted with the CHECK constraint "CHK_Gender". The conflict occurred in database "master", table "dbo.users", column 'gender'.
  at io.cdap.cdap.etl.common.ErrorDetails.handleException(ErrorDetails.java:77)
  at io.cdap.cdap.etl.spark.io.StageTrackingRecordWriter.close(StageTrackingRecordWriter.java:67)
  at org.apache.spark.internal.io.HadoopMapReduceWriteConfigUtil.closeWriter(SparkHadoopWriter.scala:373)
  at org.apache.spark.internal.io.SparkHadoopWriter$.$anonfun$executeTask$1(SparkHadoopWriter.scala:145)
  at org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1538)
  at org.apache.spark.internal.io.SparkHadoopWriter$.executeTask(SparkHadoopWriter.scala:135)
  at org.apache.spark.internal.io.SparkHadoopWriter$.$anonfun$write$1(SparkHadoopWriter.scala:88)
  at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
  at org.apache.spark.scheduler.Task.run(Task.scala:136)
  at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:548)
  at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1504)
  at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:551)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  at java.lang.Thread.run(Thread.java:750)
Caused by: io.cdap.cdap.api.exception.ProgramFailureException: Error occurred in the phase: 'Writing' with sqlState: '23000', errorCode: '547', errorMessage: The INSERT statement conflicted with the CHECK constraint "CHK_Gender". The conflict occurred in database "master", table "dbo.users", column 'gender'.
  at io.cdap.cdap.api.exception.ProgramFailureException$Builder.build(ProgramFailureException.java:229)
  at io.cdap.cdap.api.exception.ErrorUtils.getProgramFailureException(ErrorUtils.java:198)
  at io.cdap.plugin.common.db.DBErrorDetailsProvider.getProgramFailureException(DBErrorDetailsProvider.java:196)
  at io.cdap.plugin.common.db.DBErrorDetailsProvider.getExceptionDetails(DBErrorDetailsProvider.java:170)
  at io.cdap.cdap.etl.common.ErrorDetails.handleException(ErrorDetails.java:75)
  ... 14 common frames omitted
Caused by: java.sql.BatchUpdateException: The INSERT statement conflicted with the CHECK constraint "CHK_Gender". The conflict occurred in database "master", table "dbo.users", column 'gender'.
  at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeBatch(SQLServerPreparedStatement.java:2101)
  at io.cdap.plugin.db.sink.ETLDBOutputFormat$1.close(ETLDBOutputFormat.java:99)
  at io.cdap.cdap.etl.spark.io.TrackingRecordWriter.close(TrackingRecordWriter.java:46)
  at io.cdap.cdap.etl.spark.io.StageTrackingRecordWriter.close(StageTrackingRecordWriter.java:65)
  ... 13 common frames omitted
2025-01-29 09:20:32,609 - WARN  [Executor task launch worker for task 0.0 in stage 0.0 (TID 0):i.c.p.d.s.ETLDBOutputFormat@106] - com.microsoft.sqlserver.jdbc.SQLServerException: The connection is closed.
  at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:237)
  at com.microsoft.sqlserver.jdbc.SQLServerConnection.checkClosed(SQLServerConnection.java:1369)
  at com.microsoft.sqlserver.jdbc.SQLServerConnection.rollback(SQLServerConnection.java:3724)
  at io.cdap.plugin.db.sink.ETLDBOutputFormat$1.close(ETLDBOutputFormat.java:104)
  at io.cdap.cdap.etl.spark.io.TrackingRecordWriter.close(TrackingRecordWriter.java:46)
  at io.cdap.cdap.etl.spark.io.StageTrackingRecordWriter.close(StageTrackingRecordWriter.java:65)
  at org.apache.spark.internal.io.HadoopMapReduceWriteConfigUtil.closeWriter(SparkHadoopWriter.scala:373)
  at org.apache.spark.internal.io.SparkHadoopWriter$.$anonfun$executeTask$2(SparkHadoopWriter.scala:150)
  at org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1549)
  at org.apache.spark.internal.io.SparkHadoopWriter$.executeTask(SparkHadoopWriter.scala:135)
  at org.apache.spark.internal.io.SparkHadoopWriter$.$anonfun$write$1(SparkHadoopWriter.scala:88)
  at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
  at org.apache.spark.scheduler.Task.run(Task.scala:136)
  at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:548)
  at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1504)
  at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:551)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  at java.lang.Thread.run(Thread.java:750)
  

POST v3/namespaces/{namespace-id}/apps/{app-id}/workflows/DataPipelineWorkflow/runs/{run-id}/classify

[
{
  "stageName": "SQL Server4",
  "errorCategory": "Plugin-'DB Integrity Constraint Violation'-'SQL Server4'",
  "errorReason": "The INSERT statement conflicted with the CHECK constraint \"CHK_Gender\". The conflict occurred in database \"master\", table \"dbo.users\", column 'gender'. For more details, see https://docs.microsoft.com/en-us/sql/relational-databases/errors-events/database-engine-events-and-errors",
  "errorMessage": "Error occurred in the phase: 'Writing' with sqlState: '23000', errorCode: '547', errorMessage: The INSERT statement conflicted with the CHECK constraint \"CHK_Gender\". The conflict occurred in database \"master\", table \"dbo.users\", column 'gender'.",
  "errorType": "USER",
  "dependency": "true",
  "errorCodeType": "SQLSTATE",
  "errorCode": "23000",
  "supportedDocumentationUrl": "https://docs.microsoft.com/en-us/sql/relational-databases/errors-events/database-engine-events-and-errors"
}
]