Skip to content

[testing] Bug Fix: Do not leave lingering qtokens #1457

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
iyzhang merged 1 commit into from
Jan 30, 2025
Merged

[testing] Bug Fix: Do not leave lingering qtokens #1457

iyzhang merged 1 commit into from
Jan 30, 2025
+49 −161

Conversation

@iyzhang
Copy link
Contributor

@iyzhang iyzhang commented Nov 13, 2024

Remove data structure from tcp-echo that impacts performance.

@iyzhang iyzhang requested a review from anandbonde November 13, 2024 21:19
@iyzhang iyzhang force-pushed the enhancement-testing-tcp-echo branch from 916314c to 9194a6a Compare November 14, 2024 00:14
@github-actions
Copy link

github-actions bot commented Nov 14, 2024

libos = catnap
commit id = 9194a6a

collapsed_stack num_calls cycles_per_call nanoseconds_per_call percent_time
bgc::catnap::transport::epoll 13360296.10 2006.72 889.33 32.06
register 14016.25 41.61 18.30 0.00
unregister 14015.25 50.33 22.14 0.00
demikernel::sgafree 11536.09 73.15 31.65 0.04
demikernel::sgaalloc 11283.68 115.73 51.31 0.09
ioc::network::libos::push;catnap::linux::transport::push 9066.12 664.71 294.81 4.16
ioc::network::libos::push 8800.49 16633.40 7385.04 5.45
demikernel::push 8800.49 561.63 247.89 0.14
ioc::network::libos::pop;catnap::linux::transport::pop 8024.13 421.55 184.09 4.23
ioc::network::libos::pop 7815.65 17722653.37 7906350.23 25.67
demikernel::pop 7815.65 550.57 238.59 0.10
ioc::network::libos::pushto 607.67 7354.68 3082.27 23.54
ioc::network::libos::pushto;catnap::linux::transport::push 607.67 454.61 190.39 6.18
demikernel::pushto 607.67 441.10 184.91 1.42
demikernel::async_close 32.50 1140.58 468.11 0.01
ioc::network::libos::connect 24.95 6927586.56 3083425.16 27.98
ioc::network::libos::connect;catnap::linux::transport::connect 24.95 75968.89 33572.92 2.57
demikernel::connect 24.95 1976.44 880.30 0.01
ioc::network::libos::close 18.35 24736.55 10815.14 0.07
ioc::network::libos::close;catnap::linux::transport::close 18.35 23362.12 10212.29 94.54
demikernel::accept 15.42 1447.89 631.41 0.00
ioc::network::libos::accept 15.39 909191451.93 395944537.39 46.18
ioc::network::libos::accept;catnap::linux::transport::accept 15.39 5088.10 2280.52 0.00
demikernel::socket 9.50 153126.52 67804.40 0.31
demikernel::socket;catnap::linux::transport::socket 9.50 150675.85 66733.15 96.39
demikernel::new 1.00 262683.56 114636.24 0.14
demikernel::bind 1.00 28978.02 12580.07 0.01
demikernel::bind;catnap::linux::transport::bind 1.00 24362.58 10575.86 84.04
demikernel::listen 1.00 14968.29 6496.53 0.00
demikernel::listen;catnap::linux::transport::listen 1.00 14041.08 6095.77 93.78
demikernel::close 0.00 nan nan 0.00

@anandbonde anandbonde force-pushed the enhancement-testing-tcp-echo branch from 9194a6a to f3a2109 Compare November 22, 2024 23:07
@github-actions
Copy link

github-actions bot commented Nov 22, 2024

libos = catnap
commit id = f3a2109

collapsed_stack num_calls cycles_per_call nanoseconds_per_call percent_time
bgc::catnap::transport::epoll 13405989.77 2018.97 882.54 32.37
register 14016.25 42.07 18.42 0.00
unregister 14015.25 52.26 22.91 0.00
demikernel::sgafree 11536.49 74.76 31.92 0.04
demikernel::sgaalloc 11284.78 114.65 49.53 0.09
ioc::network::libos::push;catnap::linux::transport::push 9066.12 609.82 267.26 3.84
ioc::network::libos::push 8800.49 16444.21 7191.23 5.94
demikernel::push 8800.49 531.93 231.66 0.14
ioc::network::libos::pop;catnap::linux::transport::pop 8023.19 386.07 166.90 3.84
ioc::network::libos::pop 7814.73 17602903.90 8302438.92 25.88
demikernel::pop 7814.73 544.81 233.29 0.11
ioc::network::libos::pushto 623.33 8458.04 3962.93 23.34
ioc::network::libos::pushto;catnap::linux::transport::push 623.33 518.48 243.18 6.14
demikernel::pushto 623.33 497.94 233.36 1.44
demikernel::async_close 32.50 1065.58 420.66 0.01
ioc::network::libos::connect 24.95 4268132.83 1903517.82 26.59
ioc::network::libos::connect;catnap::linux::transport::connect 24.95 77660.10 33718.74 2.81
demikernel::connect 24.95 1988.76 876.88 0.01
ioc::network::libos::close 18.35 25135.75 10923.79 0.06
ioc::network::libos::close;catnap::linux::transport::close 18.35 23450.46 10183.04 93.61
demikernel::accept 15.42 1450.45 627.98 0.00
ioc::network::libos::accept 15.41 890559610.91 383544088.69 45.57
ioc::network::libos::accept;catnap::linux::transport::accept 15.39 3342.09 1425.93 0.00
demikernel::socket 9.50 156548.05 70259.11 0.31
demikernel::socket;catnap::linux::transport::socket 9.50 153837.82 69077.31 96.12
demikernel::new 1.00 262002.11 113060.61 0.11
demikernel::bind 1.00 28751.45 12506.64 0.01
demikernel::bind;catnap::linux::transport::bind 1.00 24448.15 10645.57 84.56
demikernel::listen 1.00 15250.26 6575.60 0.00
demikernel::listen;catnap::linux::transport::listen 1.00 14036.79 6054.50 91.82
demikernel::close 0.00 nan nan 0.00

@anandbonde anandbonde force-pushed the enhancement-testing-tcp-echo branch from f3a2109 to c16875b Compare November 25, 2024 20:53
@github-actions
Copy link

github-actions bot commented Nov 25, 2024

libos = catnap
commit id = c16875b

collapsed_stack num_calls cycles_per_call nanoseconds_per_call percent_time
bgc::catnap::transport::epoll 13354840.29 1699.32 737.60 33.27
register 14016.25 41.01 18.13 0.00
unregister 14015.25 49.06 21.68 0.00
demikernel::sgafree 11540.77 74.91 32.35 0.05
demikernel::sgaalloc 11296.40 143.41 60.74 0.11
ioc::network::libos::push;catnap::linux::transport::push 9066.12 649.09 286.20 4.09
ioc::network::libos::push 8800.49 16604.04 7311.48 6.67
demikernel::push 8800.49 553.94 242.40 0.16
ioc::network::libos::pop;catnap::linux::transport::pop 8023.10 399.86 175.06 3.17
ioc::network::libos::pop 7814.64 16211576.09 6312607.51 23.69
demikernel::pop 7814.64 543.37 235.69 0.13
ioc::network::libos::pushto 790.33 7300.63 2990.99 23.93
demikernel::pushto 790.33 443.98 181.98 1.48
ioc::network::libos::pushto;catnap::linux::transport::push 790.33 434.24 177.88 5.96
demikernel::async_close 32.50 1023.69 424.68 0.01
ioc::network::libos::connect 24.95 6523848.58 2816040.98 29.45
ioc::network::libos::connect;catnap::linux::transport::connect 24.95 78202.81 33927.28 2.16
demikernel::connect 24.95 2077.07 909.63 0.01
ioc::network::libos::close 18.35 24971.75 10862.92 0.05
ioc::network::libos::close;catnap::linux::transport::close 18.35 23556.32 10238.38 94.48
demikernel::accept 15.42 1521.45 669.95 0.00
ioc::network::libos::accept 15.39 878877770.73 387695128.77 44.78
ioc::network::libos::accept;catnap::linux::transport::accept 15.39 8794.58 3814.25 0.00
demikernel::socket 9.50 154998.04 67479.44 0.25
demikernel::socket;catnap::linux::transport::socket 9.50 152271.71 66296.18 96.02
demikernel::new 1.00 259804.02 112933.39 0.11
demikernel::bind 1.00 29082.28 12664.41 0.01
demikernel::bind;catnap::linux::transport::bind 1.00 24693.90 10747.48 84.72
demikernel::listen 1.00 15641.66 6844.39 0.00
demikernel::listen;catnap::linux::transport::listen 1.00 14277.53 6244.71 92.55
demikernel::close 0.00 nan nan 0.00

@anandbonde
Copy link
Contributor

anandbonde commented Nov 27, 2024

This change doesn’t seem to have any effect on the performance so we will just leave it here.

@anandbonde anandbonde closed this Nov 27, 2024
@iyzhang iyzhang reopened this Jan 28, 2025
@iyzhang iyzhang force-pushed the enhancement-testing-tcp-echo branch from c16875b to dd78aae Compare January 28, 2025 19:25
@iyzhang
Copy link
Contributor Author

iyzhang commented Jan 28, 2025

This PR fixes #1492

@iyzhang iyzhang changed the title [testing] Enhancement: Improve tcp-echo [testing] Bug Fix: Do not leave lingering qtokens Jan 28, 2025
anandbonde
anandbonde approved these changes Jan 28, 2025
View reviewed changes
Copy link
Contributor

@anandbonde anandbonde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

@github-actions
Copy link

github-actions bot commented Jan 29, 2025

libos = catnap
commit id = 525fe54

collapsed_stack num_calls cycles_per_call nanoseconds_per_call total_duration percent_total_duration total_duration_exclusive percent_total_duration_exclusive
bgc::catnap::transport::epoll 13177670 3156 1328 6954250727 100 6954250727 100
ioc::network::libos::push 17591 483 203 7374957 0 7374957 0
ioc::network::libos::pop 15318 467 196 5406182 0 5406182 0
ioc::network::libos::connect 50 46800 19454 1853818 0 1853818 0
demikernel::sgaalloc 11305 108 45 1138340 0 1138340 0
demikernel::sgafree 11543 97 41 859900 0 859900 0
ioc::network::libos::pushto 2199 335 135 737362 0 737362 0
ioc::network::libos::accept 26 12235 5182 376564 0 376564 0
ioc::network::libos::close 16 21638 9068 273749 0 273749 0

@iyzhang iyzhang force-pushed the enhancement-testing-tcp-echo branch from 525fe54 to 173ca10 Compare January 30, 2025 09:03
@github-actions
Copy link

github-actions bot commented Jan 30, 2025

libos = catnap
commit id = 173ca10

collapsed_stack num_calls cycles_per_call nanoseconds_per_call total_duration percent_total_duration total_duration_exclusive percent_total_duration_exclusive
bgc::catnap::transport::epoll 13172864 3980 2172 6998773758 100 6998773758 100
ioc::network::libos::push 17595 526 221 8001811 0 8001811 0
ioc::network::libos::pop 15263 501 210 5827486 0 5827486 0
ioc::network::libos::connect 50 62963 28178 2416494 0 2416494 0
demikernel::sgaalloc 11283 144 66 1241407 0 1241407 0
demikernel::sgafree 11533 80 34 957681 0 957681 0
ioc::network::libos::accept 26 12427 5085 397720 0 397720 0
ioc::network::libos::pushto 1188 335 141 392425 0 392425 0
ioc::network::libos::close 17 26170 10955 356149 0 356149 0

@iyzhang iyzhang force-pushed the enhancement-testing-tcp-echo branch from 173ca10 to 7ba95d7 Compare January 30, 2025 10:32
@github-actions
Copy link

github-actions bot commented Jan 30, 2025

libos = catpowder
commit id = 7ba95d7

collapsed_stack num_calls cycles_per_call nanoseconds_per_call total_duration percent_total_duration total_duration_exclusive percent_total_duration_exclusive
bgc::inetstack::poll;inetstack::layer4::poll_once 19462321 707 283 12045740442 46 12036460253 86
bgc::inetstack::poll 9731160 1621 649 13943470578 53 1897730135 14
ioc::network::libos::push 16145 2946 1229 48526892 0 48526892 0
bgc::inetstack::poll;inetstack::layer4::poll_once;inetstack::layer4::receive_batch 6083 2468 971 9437481 0 9434595 0
bgc::inetstack::tcp::established::background;tcp::established::background::retransmitter 11844 406 163 5872142 0 5872142 0
ioc::network::libos::pushto 794 4818 2034 3842365 0 3842365 0
ioc::network::libos::pop 9319 552 222 3759954 0 3759954 0
bgc::inetstack::tcp::established::background 11856 1705 673 13573564 0 3439528 0
bgc::inetstack::tcp::established::background;tcp::established::background::acknowledger 11856 572 224 3260379 0 3260379 0
ioc::network::libos::connect 42 40738 16551 1597253 0 1597253 0
demikernel::sgaalloc 6511 141 59 1003060 0 1003060 0
bgc::inetstack::tcp::established::background;tcp::established::background::sender 11801 858 331 1001516 0 1001516 0
demikernel::sgafree 8755 128 52 660543 0 660543 0
bgc::inetstack::tcp::passiveopen::background 42 33346 13017 540476 0 540476 0
ioc::network::libos::close 46 6326 2551 282250 0 282250 0
bgc::inetstack::poll;inetstack::layer4::poll_once;inetstack::layer4::receive_batch;udp::receive 257 237 100 56761 0 56761 0
ioc::network::libos::accept 40 2266 888 52026 0 52026 0
bgc::inetstack::arp::background 7 2044 815 9967 0 9967 0
bgc::inetstack::icmp::background 1 2315 920 2426 0 2426 0

@github-actions
Copy link

github-actions bot commented Jan 30, 2025

libos = catnap
commit id = 7ba95d7

collapsed_stack num_calls cycles_per_call nanoseconds_per_call total_duration percent_total_duration total_duration_exclusive percent_total_duration_exclusive
bgc::catnap::transport::epoll 13252988 3702 1510 6983188518 100 6983188518 100
ioc::network::libos::push 17600 508 206 7863859 0 7863859 0
ioc::network::libos::pop 15235 467 190 5875223 0 5875223 0
ioc::network::libos::connect 50 48724 20200 1962766 0 1962766 0
demikernel::sgaalloc 11283 110 45 1205344 0 1205344 0
demikernel::sgafree 11536 75 31 907248 0 907248 0
ioc::network::libos::pushto 1199 349 135 429953 0 429953 0
ioc::network::libos::accept 26 12636 5164 404594 0 404594 0
ioc::network::libos::close 18 22535 9171 335767 0 335767 0

@github-actions
Copy link

github-actions bot commented Jan 30, 2025

libos = catnip
commit id = 7ba95d7

collapsed_stack num_calls cycles_per_call nanoseconds_per_call total_duration percent_total_duration total_duration_exclusive percent_total_duration_exclusive
bgc::inetstack::poll;inetstack::layer4::poll_once;catnip::runtime::receive 27808783 208 85 5756803342 20 5756803342 45
bgc::inetstack::poll;inetstack::layer4::poll_once 27808783 366 150 9941839714 35 4179837659 32
bgc::inetstack::poll 13904392 928 379 12862027203 45 2920187488 23
ioc::network::libos::push 16145 1208 508 17627909 0 15504319 0
bgc::inetstack::poll;inetstack::layer4::poll_once;inetstack::layer4::receive_batch 2400 2561 1029 5286828 0 5152730 0
bgc::inetstack::tcp::established::background;tcp::established::background::retransmitter 11406 332 136 4527290 0 4526399 0
bgc::inetstack::tcp::established::background;tcp::established::background::acknowledger 11417 308 127 3000612 0 2998330 0
bgc::inetstack::tcp::established::background 11417 1080 442 11155324 0 2902447 0
ioc::network::libos::pop 9064 386 158 2698329 0 2698329 0
ioc::network::libos::push;catnip::runtime::transmit 8763 325 138 2305612 0 2305612 0
ioc::network::libos::connect 42 35936 14823 1527838 0 1471466 0
ioc::network::libos::pushto 1285 1413 577 1809990 0 1451810 0
demikernel::sgafree 8783 119 49 718519 0 718519 0
bgc::inetstack::tcp::established::background;tcp::established::background::sender 11364 313 122 724975 0 714603 0
demikernel::sgaalloc 6539 138 58 652108 0 652108 0
bgc::inetstack::tcp::passiveopen::background 42 34920 14511 446355 0 392456 0
ioc::network::libos::pushto;catnip::runtime::transmit 1284 283 115 358180 0 358180 0
ioc::network::libos::close 49 5522 2266 256194 0 256194 0
bgc::inetstack::poll;inetstack::layer4::poll_once;inetstack::layer4::receive_batch;udp::receive 529 275 108 139110 0 139110 0
bgc::inetstack::poll;inetstack::layer4::poll_once;inetstack::layer4::receive_batch;catnip::runtime::transmit 382 634 264 133830 0 133830 0
ioc::network::libos::connect;catnip::runtime::transmit 42 5925 2455 56372 0 56372 0
bgc::inetstack::tcp::passiveopen::background;catnip::runtime::transmit 21 16739 6903 53899 0 53899 0
ioc::network::libos::accept 37 2257 940 45162 0 45162 0
bgc::inetstack::arp::background 7 2331 946 21321 0 21321 0
bgc::inetstack::tcp::established::background;tcp::established::background::acknowledger;catnip::runtime::transmit 14 1788 755 15975 0 15975 0
bgc::inetstack::tcp::established::background;tcp::established::background::sender;catnip::runtime::transmit 21 550 226 10372 0 10372 0
bgc::inetstack::tcp::established::background;tcp::established::background::retransmitter;catnip::runtime::transmit 18 423 180 7130 0 7130 0
bgc::inetstack::icmp::background 1 2341 953 2508 0 2508 0

@iyzhang iyzhang merged commit 4435797 into dev Jan 30, 2025
14 checks passed
@iyzhang iyzhang deleted the enhancement-testing-tcp-echo branch January 30, 2025 11:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@anandbonde anandbonde anandbonde approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@iyzhang @anandbonde