[CORE] Refactor task error logging into dedicated utility class

[pratham76]

What changes are proposed in this pull request?

Extract task error logging from TaskResources into new TaskErrorLogger utility, resolving a TODO here. Improves separation of concerns while maintaining identical functionality for filtering speculative execution errors.

How was this patch tested?

Existing UTs

Was this patch authored or co-authored using generative AI tooling?

No

[github-actions]

Run Gluten Clickhouse CI on x86

[github-actions]

Run Gluten Clickhouse CI on x86

[pratham76]

@zhztheplayer Could you please review?

[zhztheplayer]

Hi @pratham76, thanks for helping out.

The intention of the TODO could mean we need to rely on Spark's scheduler layer to log the error. But we had to keep the TaskFailureListener here because Gluten may cause crash in CompletionListener so Spark's scheduler code will never have a chance to log the error.

So I think the discussion here is not about how to log the error. We may need to consider open PR in upstream Spark to make logging work even CompletionListener crashed.

I might not recall everything so let me know if you have different findings here.

[zhztheplayer]

Apart from the above comment, PR change itself LGTM.

[github-actions]

Run Gluten Clickhouse CI on x86

[pratham76]

Hi @zhztheplayer, Thanks for the clarification, I do get that the proper solution for the above mentioned TODO would be make spark's error logging more resilient when CompletionListener crashes.

In this PR, I have tried to refactor the logging logic into separate utility class, to handle both scenarios of task failures and task recompute/retries, just to make the code a bit cleaner. I'm happy to explore the upstream Spark changes for this TODO (which I have updated here to give better context). For now, could we proceed with this refactoring as an incremental improvement?

[github-actions]

Run Gluten Clickhouse CI on x86

[zhztheplayer]

Hi @zhztheplayer, Thanks for the clarification, I do get that the proper solution for the above mentioned TODO would be make spark's error logging more resilient when CompletionListener crashes.

In this PR, I have tried to refactor the logging logic into separate utility class, to handle both scenarios of task failures and task recompute/retries, just to make the code a bit cleaner. I'm happy to explore the upstream Spark changes for this TODO (which I have updated here to give better context). For now, could we proceed with this refactoring as an incremental improvement?

Sounds reasonable. Thanks for the patch.