💥Enable Worker heartbeating, update Core to 44a6576 #570

yuandrew · 2025-12-04T22:08:43Z

What was changed

Enabled worker heartbeat
plumb plugin names (both client and worker) to core
update core to 44a6576
- 💥temporal_sdk_core_c_bridge.dll got renamed to temporalio_sdk_core_c_bridge.dll

Why?

New worker heartbeating feature

Checklist

Closes [Feature Request] Enable Worker Heartbeating #551
How was this tested:
Manually tested
Any docs updates needed?

Note

Enables worker heartbeating, plumbs plugin names to Core, updates interop/bindings and build paths, and renames the native bridge library to temporalio_sdk_core_c_bridge across code, CI, and docs.

Runtime/Worker:
- Add TemporalRuntimeOptions.WorkerHeartbeatInterval and propagate to Core (worker_heartbeat_interval_millis).
- Pass plugin names (client + worker) to Core via TemporalCoreByteArrayRefArray on client/worker options.
- Replace no_remote_activities with TemporalCoreWorkerTaskTypes in TemporalCoreWorkerOptions and populate based on registered work.
- Bridge.Worker constructor accepts clientPlugins and forwards to interop conversion.
Interop/Core bindings:
- Update header path to sdk-core/crates/sdk-core-c-bridge and adjust all DllImport names and library path to temporalio_sdk_core_c_bridge.
- Add new interop structs/fields: TemporalCoreWorkerTaskTypes, extra fields on client/runtime/worker options.
- Rename Rust crate log/targets in telemetry defaults from temporal_* to temporalio_*.
Build/CI/Packaging:
- Rename emitted artifacts to temporalio_sdk_core_c_bridge.*; update .csproj, build/net462/Temporalio.targets, and GitHub Actions workflow to new paths and Cargo manifest.
- Adjust CI matrices/includes and smoke-test config; use Manylinux Alpine/ARM containers.
Docs/Tests:
- Update README and test expectations for new library name and log targets; minor test fixes (unique task queues, metric assertions).

^{Written by Cursor Bugbot for commit 2bbb1f4. This will update automatically on new commits. Configure here.}

src/Temporalio/Runtime/TemporalRuntimeOptions.cs

cretz · 2025-12-05T13:32:14Z

src/Temporalio/Temporalio.csproj


  <PropertyGroup Condition="$([MSBuild]::IsOSPlatform('Windows'))">
-    <BridgeLibraryFile>temporal_sdk_core_c_bridge.dll</BridgeLibraryFile>
+    <BridgeLibraryFile>temporalio_sdk_core_c_bridge.dll</BridgeLibraryFile>


This is considered a backwards incompatible change. We have to mark the PR with a 💥 in the title of the PR at least so we know to update release notes.

Thanks, updated PR title and desc

cretz · 2025-12-05T13:33:15Z

src/Temporalio/Temporalio.csproj


  <PropertyGroup Condition="$([MSBuild]::IsOSPlatform('Windows'))">
-    <BridgeLibraryFile>temporal_sdk_core_c_bridge.dll</BridgeLibraryFile>
+    <BridgeLibraryFile>temporalio_sdk_core_c_bridge.dll</BridgeLibraryFile>


This change was not made to the PackBridgeRuntimes from below. Should probably also temporarily enable the package.yml GH workflow in this branch to confirm it still works.

package.yml now passes after a fixup, https://github.com/temporalio/sdk-dotnet/actions/runs/19974335695

cretz · 2025-12-05T13:37:11Z

src/Temporalio/Bridge/OptionsExtensions.cs

            options.NexusTaskPollerBehavior ??= new PollerBehavior.SimpleMaximum(options.MaxConcurrentNexusTaskPolls);
+            var workerPluginNames = options.Plugins?
+                .Select(p => p.Name)
+                .Where(name => !string.IsNullOrEmpty(name)) ?? Enumerable.Empty<string>();


Pedantic, but I don't think we need the IsNullOrEmpty check, it is a required property. If we're concerned about empty plugin names, we should validate that somewhere else, but I don't think we're really that concerned about it.

cretz · 2025-12-05T13:39:23Z

src/Temporalio/Runtime/TemporalRuntimeOptions.cs

+        /// </summary>
+        /// <param name="telemetry"><see cref="Telemetry" />.</param>
+        /// <param name="workerHeartbeatInterval">Worker heartbeat interval. If 0 is specified, heartbeat is disabled.</param>
+        public TemporalRuntimeOptions(TelemetryOptions telemetry, TimeSpan workerHeartbeatInterval)


I don't think you need a constructor for this, workerHeartbeatInterval would be rarely set, they can use the parameterless or post-construction set approaches

sounds good, removed

cretz · 2025-12-05T13:39:39Z

src/Temporalio/Runtime/TemporalRuntimeOptions.cs

+        /// <summary>
+        /// Gets or sets the worker heartbeat duration in milliseconds.
+        /// </summary>
+        public TimeSpan WorkerHeartbeatInterval { get; set; } = TimeSpan.FromSeconds(60);


This should be nullable IMO (i.e. ?)

sounds good, added in

cretz · 2025-12-05T13:40:21Z

src/Temporalio/Bridge/OptionsExtensions.cs

            return new Interop.TemporalCoreRuntimeOptions()
            {
                telemetry = scope.Pointer(options.Telemetry.ToInteropOptions(scope)),
+                worker_heartbeat_interval_millis = (ulong)options.WorkerHeartbeatInterval.TotalMilliseconds,


This should support unset (if 0 means unset in bridge, then can convert here from null to unset)

converted null and unset to 0, lmk if I misunderstood the comment

cretz · 2025-12-05T13:46:15Z

It looks like something about the Core upgrade broke the ExecuteWorkflowAsync_CustomMetrics_FloatsAndDurations test. I think we have to investigate this (let me know if help is needed here).

cretz · 2025-12-05T13:51:40Z

src/Temporalio/Runtime/TemporalRuntimeOptions.cs

Note, you'll have to update the TelemetryFilterOptions class to account for the crate name changes

ty, I didn't do a thorough search through the repo to update crate names. Kinda thought AI woulda taken care of that, my bad

All good, I missed this in Ruby which was pointed out to me at temporalio/sdk-ruby#368 (comment) which reminded me to point it out here

.github/workflows/package.yml

cursor

Bug: Client plugin names not passed to Core

The TemporalCoreClientOptions struct now has a plugin_names field added by this PR, but the ToInteropOptions method for TemporalConnectionOptions doesn't set this field. The PR description mentions "plumb plugin names (both client and worker) to core", and while worker plugin names are properly passed (via the worker options), client plugin names are not being sent to Core during connection. The field will be default-initialized to an empty array, which may cause the worker heartbeat feature to not properly report client-side plugins.

src/Temporalio/Bridge/OptionsExtensions.cs#L246-L269

sdk-dotnet/src/Temporalio/Bridge/OptionsExtensions.cs

Lines 246 to 269 in 0d016b3

    
           var scheme = options.Tls == null ? "http" : "https"; 
        
           return new Interop.TemporalCoreClientOptions() 
        
           { 
        
               target_url = scope.ByteArray($"{scheme}://{options.TargetHost}"), 
        
               client_name = ClientName.Ref, 
        
               client_version = ClientVersion.Ref, 
        
               metadata = scope.Metadata(options.RpcMetadata), 
        
               api_key = scope.ByteArray(options.ApiKey), 
        
               identity = scope.ByteArray(options.Identity), 
        
               tls_options = 
        
                   options.Tls == null ? null : scope.Pointer(options.Tls.ToInteropOptions(scope)), 
        
               retry_options = 
        
                   options.RpcRetry == null 
        
                       ? null 
        
                       : scope.Pointer(options.RpcRetry.ToInteropOptions()), 
        
               keep_alive_options = 
        
                   options.KeepAlive == null 
        
                       ? null 
        
                       : scope.Pointer(options.KeepAlive.ToInteropOptions()), 
        
               http_connect_proxy_options = 
        
                   options.HttpConnectProxy == null 
        
                       ? null 
        
                       : scope.Pointer(options.HttpConnectProxy.ToInteropOptions(scope)), 
        
           };

src/Temporalio/Bridge/Interop/Interop.cs#L217-L219

sdk-dotnet/src/Temporalio/Bridge/Interop/Interop.cs

Lines 217 to 219 in 0d016b3

    
           [NativeTypeName("struct TemporalCoreByteArrayRefArray")] 
        
           public TemporalCoreByteArrayRefArray plugin_names;

src/Temporalio/Bridge/OptionsExtensions.cs

cretz

LGTM, only blocking concern is to remove the temporary enablement of the package.yml GH workflow

.github/workflows/package.yml

cretz · 2025-12-05T20:26:34Z

src/Temporalio/Bridge/OptionsExtensions.cs

            }
+            var pluginNames = options.Plugins?
+                .Select(p => p.Name)
+                .Where(name => !string.IsNullOrEmpty(name))


As cursor bot mentions above, probably don't need this line

cretz · 2025-12-05T20:27:51Z

src/Temporalio/Runtime/TemporalRuntimeOptions.cs

All good, I missed this in Ruby which was pointed out to me at temporalio/sdk-ruby#368 (comment) which reminded me to point it out here

cretz · 2025-12-05T20:29:33Z

tests/Temporalio.Tests/Worker/WorkflowWorkerTests.cs

                // We'll also test the metric prefix
                Metrics = new() { Prometheus = new(promAddr), MetricPrefix = "foo_" },
            },
+            WorkerHeartbeatInterval = null,


I'm curious why these had to be set to null in these tests (just trying to make sure our users won't have to do this in their tests)

ahh this is due to the metrics test failing from the Worker heartbeating configured for runtime, but server version does not support it warning message, the solution is either to disable it for now, or we can set the dynamic config flag to support heartbeating

the test fails from seeing an additional log statement

Hrmm, I don't see where these tests are checking logs. Can you explain a bit more how these tests were failing before setting this to null? I want to make sure our users don't have to do the same thing in the same situations. I am not seeing where the metrics would clash with worker heartbeat metrics, but I may be missing it. If unclear where it is failing, I can help debug.

Discussed off-PR, provided diff for TestUtils.cs to help this situation

yuandrew added 2 commits December 2, 2025 13:37

bump core to 45b1d7e

e72cf8e

rename to heartbeat_interval, need core update for this

2c9eac1

yuandrew requested a review from a team as a code owner December 4, 2025 22:08

Update core to 7419ada, plumb plugin to bridge

fbc46ea

yuandrew force-pushed the worker-heartbeating branch from 23d3dc6 to fbc46ea Compare December 4, 2025 22:13

cursor bot reviewed Dec 4, 2025

View reviewed changes

src/Temporalio/Runtime/TemporalRuntimeOptions.cs Outdated Show resolved Hide resolved

Fix breaking change

3367bb6

cretz reviewed Dec 5, 2025

View reviewed changes

yuandrew changed the title ~~Enable Worker heartbeating, update Core to 7419ada~~ 💥Enable Worker heartbeating, update Core to 7419ada Dec 5, 2025

yuandrew added 2 commits December 5, 2025 06:20

add missing DLL renames, enable package.yml for this branch

080fa76

Make WorkerHeartbeatInterval nullable, fix test

c057406

cursor bot reviewed Dec 5, 2025

View reviewed changes

.github/workflows/package.yml Outdated Show resolved Hide resolved

yuandrew added 3 commits December 5, 2025 13:46

fix other metrics test

bcdabca

i think need to use PR trigger instead of branch bc of fork

01d95b7

fix paths for crate reshuffle

0d016b3

cursor bot reviewed Dec 5, 2025

View reviewed changes

src/Temporalio/Bridge/OptionsExtensions.cs Show resolved Hide resolved

cretz reviewed Dec 5, 2025

View reviewed changes

yuandrew and others added 2 commits December 5, 2025 16:20

Package.yml confirmed, remove change. Address cursor feedback

81ab6f9

Merge branch 'main' into worker-heartbeating

52b81dc

yuandrew changed the title ~~💥Enable Worker heartbeating, update Core to 7419ada~~ 💥Enable Worker heartbeating, update Core to 44a6576 Dec 5, 2025

jmaeagle99 mentioned this pull request Dec 10, 2025

Support gRPC binary metadata headers in C bridge temporalio/sdk-core#1076

Merged

Merge branch 'main' into worker-heartbeating

0cc4dc1

cretz approved these changes Dec 10, 2025

View reviewed changes

yuandrew and others added 3 commits December 10, 2025 14:28

test fix

526e01c

Merge branch 'main' into worker-heartbeating

c762282

Test fix

2bbb1f4

yuandrew merged commit f384f20 into temporalio:main Dec 10, 2025
11 checks passed

yuandrew deleted the worker-heartbeating branch December 10, 2025 21:47

cretz mentioned this pull request Dec 11, 2025

Fix api generator and sdk-core protos path #574

Merged

	var scheme = options.Tls == null ? "http" : "https";
	return new Interop.TemporalCoreClientOptions()
	{
	target_url = scope.ByteArray($"{scheme}://{options.TargetHost}"),
	client_name = ClientName.Ref,
	client_version = ClientVersion.Ref,
	metadata = scope.Metadata(options.RpcMetadata),
	api_key = scope.ByteArray(options.ApiKey),
	identity = scope.ByteArray(options.Identity),
	tls_options =
	options.Tls == null ? null : scope.Pointer(options.Tls.ToInteropOptions(scope)),
	retry_options =
	options.RpcRetry == null
	? null
	: scope.Pointer(options.RpcRetry.ToInteropOptions()),
	keep_alive_options =
	options.KeepAlive == null
	? null
	: scope.Pointer(options.KeepAlive.ToInteropOptions()),
	http_connect_proxy_options =
	options.HttpConnectProxy == null
	? null
	: scope.Pointer(options.HttpConnectProxy.ToInteropOptions(scope)),
	};


	[NativeTypeName("struct TemporalCoreByteArrayRefArray")]
	public TemporalCoreByteArrayRefArray plugin_names;

💥Enable Worker heartbeating, update Core to 44a6576 #570

💥Enable Worker heartbeating, update Core to 44a6576 #570

Uh oh!

Conversation

yuandrew commented Dec 4, 2025 • edited by cursor bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What was changed

Why?

Checklist

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cretz commented Dec 5, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Bug: Client plugin names not passed to Core

Uh oh!

Uh oh!

cretz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cretz Dec 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

yuandrew commented Dec 4, 2025 •

edited by cursor bot

Loading

cretz Dec 8, 2025 •

edited

Loading