perf: Hydra performance optimizations for datum-heavy workloads by awcjack · Pull Request #22 · awcjack/hydra

awcjack · 2026-01-09T01:11:16Z

Summary

This PR introduces performance optimizations for Hydra Head protocol, specifically targeting datum-heavy transaction workloads.

Changes

1. Batch Transaction Validation Optimization

Convert UTxO to Shelley format once per batch instead of per transaction
Reduces conversion overhead from O(n × txCount) to O(n)
File: hydra-node/src/Hydra/Ledger/Cardano.hs

2. adjustUTxO Optimization

Use Map.withoutKeys for O(m log n) instead of O(n log n)
Where m = consumed inputs, n = UTxO set size
File: hydra-node/src/Hydra/Ledger/Cardano.hs

Performance Impact

Optimization	Before	After	Impact
adjustUTxO	O(n log n)	O(m log n)	Faster UTxO updates
Batch validation	O(n × txCount) conversions	O(n) conversions	Significant for multi-tx batches

Testing

✅ All Hydra.Ledger.Cardano tests pass (18 tests)
✅ All Hydra.Model tests pass (6 tests)
✅ Treefmt formatting check passes
✅ Weeder dead code check passes

Previous Commits (from priority-queue-docker branch)

This branch also includes:

Priority queue for protocol messages
Hybrid snapshot throttling implementation
CI/Docker build improvements

github-actions · 2026-01-09T01:25:23Z

Transaction costs

Sizes and execution budgets for Hydra protocol transactions. Note that unlisted parameters are currently using arbitrary values and results are not fully deterministic and comparable to previous runs.

Metadata
Generated at	2026-01-12 08:54:45.625013857 UTC
Max. memory units	14000000
Max. CPU units	10000000000
Max. tx size (kB)	16384

Script summary

Name	Hash	Size (Bytes)
νInitial	c8a101a5c8ac4816b0dceb59ce31fc2258e387de828f02961d2f2045	2652
νCommit	61458bc2f297fff3cc5df6ac7ab57cefd87763b0b7bd722146a1035c	685
νHead	a1442faf26d4ec409e2f62a685c1d4893f8d6bcbaf7bcb59d6fa1340	14599
μHead	fd173b993e12103cd734ca6710d364e17120a5eb37a224c64ab2b188*	5284
νDeposit	ae01dade3a9c346d5c93ae3ce339412b90a0b8f83f94ec6baa24e30c	1102

The minting policy hash is only usable for comparison. As the script is parameterized, the actual script is unique per head.

`Init` transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	5834	10.47	3.32	0.52
2	6035	12.44	3.94	0.54
3	6239	14.31	4.52	0.57
5	6641	18.43	5.81	0.63
10	7646	29.14	9.19	0.79
43	14279	98.95	30.93	1.80

`Commit` transaction costs

This uses ada-only outputs for better comparability.

UTxO	Tx size	% max Mem	% max CPU	Min fee ₳
1	561	2.44	1.16	0.20
2	742	3.38	1.73	0.22
3	917	4.36	2.33	0.24
5	1280	6.41	3.60	0.28
10	2166	12.13	7.25	0.40
54	10061	98.61	68.52	1.88

`CollectCom` transaction costs

Parties	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
1	57	525	24.42	7.12	0.42
2	114	636	33.33	9.64	0.52
3	171	747	42.46	12.22	0.61
4	227	862	53.53	15.24	0.73
5	282	974	57.43	16.56	0.78
6	337	1085	73.46	20.82	0.94
7	395	1196	80.41	22.97	1.02
8	449	1307	90.02	25.68	1.12
9	505	1414	93.04	26.87	1.16

Cost of Increment Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	1790	23.92	7.60	0.48
2	1924	25.92	8.80	0.51
3	2112	28.10	10.09	0.54
5	2431	32.41	12.62	0.61
10	3181	41.98	18.63	0.76
41	7754	99.75	55.35	1.70

Cost of Decrement Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	631	22.84	7.40	0.42
2	767	24.35	8.48	0.44
3	957	26.95	9.86	0.48
5	1168	27.93	11.45	0.51
10	1983	40.53	18.32	0.70
42	6746	98.38	55.73	1.64

`Close` transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	639	29.17	8.91	0.48
2	846	31.61	10.27	0.52
3	939	32.72	11.23	0.54
5	1244	37.02	13.77	0.60
10	2030	48.45	20.35	0.78
34	5634	97.52	50.07	1.55

`Contest` transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	674	33.87	10.16	0.53
2	819	35.88	11.39	0.56
3	892	37.16	12.39	0.58
5	1307	43.25	15.47	0.67
10	2131	55.09	22.12	0.85
29	4936	98.12	46.74	1.50

`Abort` transaction costs

There is some variation due to the random mixture of initial and already committed outputs.

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	5793	27.00	9.06	0.69
2	5945	35.91	12.07	0.79
3	6187	46.82	15.84	0.92
4	6290	54.58	18.40	1.00
5	6444	63.54	21.39	1.10
6	6529	71.62	24.09	1.19
7	6850	85.63	28.96	1.35
8	6960	93.59	31.64	1.44
9	7021	97.75	32.94	1.49

`FanOut` transaction costs

Involves spending head output and burning head tokens. Uses ada-only UTXO for better comparability.

Parties	UTxO	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
10	0	0	5835	18.75	6.26	0.60
10	1	56	5868	20.34	6.91	0.62
10	20	1139	6514	59.54	22.38	1.08
10	30	1707	6854	79.78	30.37	1.32
10	36	2051	7059	92.53	35.37	1.46

End-to-end benchmark results

This page is intended to collect the latest end-to-end benchmark results produced by Hydra's continuous integration (CI) system from the latest master code.

Please note that these results are approximate as they are currently produced from limited cloud VMs and not controlled hardware. Rather than focusing on the absolute results, the emphasis should be on relative results, such as how the timings for a scenario evolve as the code changes.

Generated at 2026-01-12 08:59:02.682818829 UTC

Baseline Scenario

Number of nodes	1
Number of txs	300
Avg. Confirmation Time (ms)	5.480719966
P99	14.337489459999997ms
P95	6.6964656000000025ms
P50	5.0877115ms
Number of Invalid txs	0

Memory data

Time	Used	Free
2026-01-12 08:57:35.022028684 UTC	1658M	6990M
2026-01-12 08:57:36.021927049 UTC	1686M	6927M
2026-01-12 08:57:37.021847902 UTC	1700M	6912M
2026-01-12 08:57:38.021910408 UTC	1709M	6903M
2026-01-12 08:57:39.021859903 UTC	1761M	6775M
2026-01-12 08:57:40.021905881 UTC	1766M	6770M
2026-01-12 08:57:41.021862202 UTC	1777M	6756M
2026-01-12 08:57:42.02185914 UTC	1784M	6744M
2026-01-12 08:57:43.021853925 UTC	1784M	6743M
2026-01-12 08:57:44.021866844 UTC	1784M	6743M
2026-01-12 08:57:45.021846323 UTC	1789M	6738M
2026-01-12 08:57:46.021850135 UTC	1789M	6738M
2026-01-12 08:57:47.021858399 UTC	1789M	6738M
2026-01-12 08:57:48.021852695 UTC	1790M	6737M
2026-01-12 08:57:49.021840341 UTC	1790M	6737M
2026-01-12 08:57:50.021867792 UTC	1790M	6737M
2026-01-12 08:57:51.021843546 UTC	1790M	6737M
2026-01-12 08:57:52.021858115 UTC	1790M	6736M
2026-01-12 08:57:53.021856258 UTC	1791M	6735M
2026-01-12 08:57:54.021863293 UTC	1793M	6733M
2026-01-12 08:57:55.021858608 UTC	1794M	6732M
2026-01-12 08:57:56.021813584 UTC	1794M	6732M
2026-01-12 08:57:57.021851804 UTC	1794M	6732M
2026-01-12 08:57:58.021856965 UTC	1794M	6732M
2026-01-12 08:57:59.021864578 UTC	1794M	6732M
2026-01-12 08:58:00.021875886 UTC	1794M	6732M
2026-01-12 08:58:01.021854004 UTC	1794M	6732M
2026-01-12 08:58:02.02185195 UTC	1794M	6731M
2026-01-12 08:58:03.021829329 UTC	1794M	6731M
2026-01-12 08:58:04.021869956 UTC	1795M	6730M

Three local nodes

Number of nodes	3
Number of txs	900
Avg. Confirmation Time (ms)	34.024970910
P99	73.81702264999997ms
P95	46.37346784999995ms
P50	31.428618ms
Number of Invalid txs	0

Memory data

Time	Used	Free
2026-01-12 08:58:15.806243518 UTC	1679M	6884M
2026-01-12 08:58:16.806251421 UTC	1681M	6882M
2026-01-12 08:58:17.806228205 UTC	1694M	6869M
2026-01-12 08:58:18.80624006 UTC	1701M	6862M
2026-01-12 08:58:19.806202975 UTC	1701M	6862M
2026-01-12 08:58:20.806212708 UTC	1701M	6861M
2026-01-12 08:58:21.806243296 UTC	1706M	6856M
2026-01-12 08:58:22.806309511 UTC	1746M	6788M
2026-01-12 08:58:23.806318348 UTC	1818M	6688M
2026-01-12 08:58:24.806780984 UTC	1869M	6609M
2026-01-12 08:58:25.806311921 UTC	1919M	6559M
2026-01-12 08:58:26.809368081 UTC	1929M	6548M
2026-01-12 08:58:27.807282408 UTC	1968M	6500M
2026-01-12 08:58:28.807237042 UTC	1973M	6484M
2026-01-12 08:58:29.807325216 UTC	1985M	6461M
2026-01-12 08:58:30.808398698 UTC	2001M	6436M
2026-01-12 08:58:31.806629367 UTC	2008M	6418M
2026-01-12 08:58:32.80774479 UTC	2029M	6388M
2026-01-12 08:58:33.808234891 UTC	2055M	6352M
2026-01-12 08:58:34.806724137 UTC	2062M	6335M
2026-01-12 08:58:35.806783709 UTC	2065M	6321M
2026-01-12 08:58:36.808260138 UTC	2081M	6296M
2026-01-12 08:58:37.806233659 UTC	2083M	6292M
2026-01-12 08:58:38.806383261 UTC	2088M	6286M
2026-01-12 08:58:39.806292848 UTC	2090M	6285M
2026-01-12 08:58:40.806309601 UTC	2089M	6285M
2026-01-12 08:58:41.806265095 UTC	2091M	6283M
2026-01-12 08:58:42.806282877 UTC	2091M	6283M
2026-01-12 08:58:43.806343376 UTC	2093M	6281M
2026-01-12 08:58:44.806369999 UTC	2093M	6281M
2026-01-12 08:58:45.806359176 UTC	2093M	6280M
2026-01-12 08:58:46.806317804 UTC	2094M	6279M
2026-01-12 08:58:47.806324384 UTC	2094M	6278M
2026-01-12 08:58:48.806322737 UTC	2096M	6276M
2026-01-12 08:58:49.806351255 UTC	2096M	6276M
2026-01-12 08:58:50.806413735 UTC	2096M	6276M
2026-01-12 08:58:51.806373325 UTC	2096M	6276M
2026-01-12 08:58:52.806245735 UTC	2096M	6276M
2026-01-12 08:58:53.806308695 UTC	2096M	6276M
2026-01-12 08:58:54.806300378 UTC	2096M	6276M
2026-01-12 08:58:55.806318226 UTC	2096M	6276M
2026-01-12 08:58:56.80635981 UTC	2096M	6275M
2026-01-12 08:58:57.80652833 UTC	2096M	6275M
2026-01-12 08:58:58.806225138 UTC	2101M	6270M
2026-01-12 08:58:59.806326281 UTC	2101M	6269M
2026-01-12 08:59:00.806450212 UTC	2104M	6266M
2026-01-12 08:59:01.806366803 UTC	2104M	6266M

github-actions · 2026-01-09T01:26:33Z

Transaction cost differences

No cost or size differences found

Implement priority-based message processing to prevent protocol starvation: - Add PriorityTMQueue with configurable priority levels - Protocol messages (AckSn, ReqSn) get highest priority (0) - Client/chain messages get lower priorities - Use Data.Heap (pairing heap) for O(log n) operations - Add comprehensive tests for priority ordering

Add batch transaction validation to Ledger interface: - applyTransactionBatch for efficient multi-tx validation - Avoids repeated set difference operations - Simple fold-based implementation for correctness

…n once Reduce redundant CBOR serialization during signature verification: - Compute signable representation once before iterating over parties - Amortizes serialization cost across all signature verifications - Significant improvement when verifying multi-party snapshots

- Replace list-based localTxs with Data.Sequence for O(1) append - Update TxReceived and TxValid state changes to use Seq - Fix maybeRequestNextSnapshot to properly handle deposits: - Use currentDepositTxId if already in progress - Fall back to nextActiveDeposit for new active deposits - Request snapshot when either localTxs or deposits need processing - Ensure consistent Seq handling in snapshot request flow

- Update BehaviorSpec, HeadLogicSpec, HeadLogicSnapshotSpec for Seq tx - Update MockChain, RotationSpec, NodeSpec for priority queue - Add test helpers for Seq-based localTxs assertions

- Add v1.2.0-base to workflow triggers - Enable CI for performance optimization branch testing

awcjack changed the base branch from master to v1.2.0-base January 9, 2026 01:13

awcjack changed the base branch from v1.2.0-base to master January 9, 2026 01:14

awcjack changed the base branch from master to v1.2.0-base January 9, 2026 01:15

awcjack force-pushed the performance-optimizations branch from 77b6608 to 23bfdf9 Compare January 12, 2026 04:55

awcjack added 6 commits January 12, 2026 16:45

perf: optimize UTxO operations for batch transaction validation

8cadaa2

Add batch transaction validation to Ledger interface: - applyTransactionBatch for efficient multi-tx validation - Avoids repeated set difference operations - Simple fold-based implementation for correctness

test: update test infrastructure for priority queue and Seq changes

2b936ab

- Update BehaviorSpec, HeadLogicSpec, HeadLogicSnapshotSpec for Seq tx - Update MockChain, RotationSpec, NodeSpec for priority queue - Add test helpers for Seq-based localTxs assertions

ci: trigger workflow for v1.2.0-base branch and performance testing

c05219e

- Add v1.2.0-base to workflow triggers - Enable CI for performance optimization branch testing

awcjack force-pushed the performance-optimizations branch from 23bfdf9 to c05219e Compare January 12, 2026 08:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

perf: Hydra performance optimizations for datum-heavy workloads#22

perf: Hydra performance optimizations for datum-heavy workloads#22
awcjack wants to merge 6 commits intov1.2.0-basefrom
performance-optimizations

awcjack commented Jan 9, 2026

Uh oh!

github-actions bot commented Jan 9, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Jan 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

awcjack commented Jan 9, 2026

Summary

Changes

Performance Impact

Testing

Previous Commits (from priority-queue-docker branch)

Uh oh!

github-actions bot commented Jan 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Transaction costs

Script summary

Init transaction costs

Commit transaction costs

CollectCom transaction costs

Cost of Increment Transaction

Cost of Decrement Transaction

Close transaction costs

Contest transaction costs

Abort transaction costs

FanOut transaction costs

End-to-end benchmark results

Baseline Scenario

Memory data

Three local nodes

Memory data

Uh oh!

github-actions bot commented Jan 9, 2026

Transaction cost differences

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

github-actions bot commented Jan 9, 2026 •

edited

Loading

`Init` transaction costs

`Commit` transaction costs

`CollectCom` transaction costs

`Close` transaction costs

`Contest` transaction costs

`Abort` transaction costs

`FanOut` transaction costs