YOLO inference takes too long, causing video stuttering and lag

[lida2003]

• test video(people 30FPS): https://drive.google.com/file/d/1c90Ip1jVieWfEQjmw67LsWxTn7Xd1sYi/view?usp=sharing
• test video(cars 60FPS): https://drive.google.com/file/d/1VxqCZFpaa13BBLIqKZG7_ehj4wcp-k-Z/view?usp=sharing

inference time: 51 ~ 499ms/ 63 ~ 510ms
deepsort time: 1.1 ~ 10.5ms/ 1.1 ~ 20 ms
7% frame loss when 60FPS

The video playback is inconsistent, with stuttering or jittering.

• cars.mp4 // 60 FPS 1280x720
• log: cars.log

daniel@daniel-nvidia:~/Work/jetson-fpv$ ./scripts/tools/analysis_raw_log.sh cars.log
dos2unix: converting file cars.log to Unix format...

### Video log reading ...
Processing: [====================] 100%

### Video frame consistency check ...
Processing: [====================] 100%
Video(totl): 1347 frames
Video(lost): 0 frames
Video: frames are continuous

### Inference consistency check ...
Processing: [====================] 100%
Inference(totl): 1330 frames
Inference(eval): 344 frames - 25%
Inference(skip): 893 frames - 67%
Inference(lost): 94 frames - 7%
Inference(inf_min): 0.05106854438781738 second
Inference(inf_max): 0.4998948574066162 second
Inference(track_min): 0.001172780990600586 second
Inference(track_max): 0.01059722900390625 second
Inference: frames experience loss
Inference: specific missing frame are 21 22 23 24 25 26 27 28 29 30 31 301 302 333 352 389 410 411 419 469 486 502 547 548 559 564 573 578 579 580 581 582 600 601 615 653 658 659 683 684 695 710 715 725 730 736 751 752 781 790 795 804 805 806 822 857 858 887 888 889 910 911 961 976 977 995 1019 1034 1050 1073 1099 1110 1115 1116 1137 1146 1151 1156 1157 1169 1190 1191 1197 1198 1215 1222 1245 1269 1278 1279 1301 1306 1307 1308

• people.mp4 // 25 FPS 1280x720
• log: people.log

daniel@daniel-nvidia:~/Work/jetson-fpv$ ./scripts/tools/analysis_raw_log.sh people.log
dos2unix: converting file people.log to Unix format...

### Video log reading ...
Processing: [====================] 100%

### Video frame consistency check ...
Processing: [====================] 100%
Video(totl): 200 frames
Video(lost): 0 frames
Video: frames are continuous

### Inference consistency check ...
Processing: [====================] 100%
Inference(totl): 188 frames
Inference(eval): 165 frames - 87%
Inference(skip): 24 frames - 12%
Inference(lost): 0 frames - 0%
Inference(inf_min): 0.06393694877624512 second
Inference(inf_max): 0.5100603103637695 second
Inference(track_min): 0.0011720657348632812 second
Inference(track_max): 0.002038717269897461 second
Inference: frames are continuous
daniel@daniel-nvidia:~/Work/jetson-fpv$

• Test Env

Software part of jetson-stats 4.3.1 - (c) 2024, Raffaello Bonghi
Model: NVIDIA Jetson Orin Nano Developer Kit - Jetpack 6.2 [L4T 36.4.3]
NV Power Mode[0]: 15W
Serial Number: [XXX Show with: jetson_release -s XXX]
Hardware:
 - P-Number: p3767-0005
 - Module: NVIDIA Jetson Orin Nano (Developer kit)
Platform:
 - Distribution: Ubuntu 22.04 Jammy Jellyfish
 - Release: 5.15.148-tegra
jtop:
 - Version: 4.3.1
 - Service: Active
Libraries:
 - CUDA: 12.6.68
 - cuDNN: 9.3.0.75
 - TensorRT: 10.3.0.30
 - VPI: 3.2.4
 - OpenCV: 4.11.0 - with CUDA: YES

--------------------------------
NVIDIA SDK:
DeepStream C/C++ SDK version: 7.1
    jetson-inference version: c038530 (dirty)
        jetson-utils version: 6d5471c

--------------------------------
Python Environment:
Python 3.10.12
    GStreamer:                   YES (1.20.3)
  NVIDIA CUDA:                   YES (ver 12.6, CUFFT CUBLAS FAST_MATH)
         OpenCV version: 4.11.0  CUDA True
           YOLO version: 8.3.89
         PYCUDA version: 2024.1.2
          Torch version: 2.5.0a0+872d972e41.nv24.08
    Torchvision version: 0.20.0a0+afc54f7
 DeepStream SDK version: 1.2.0
onnxruntime     version: 1.21.0
onnxruntime-gpu version: 1.20.0

--------------------------------
FPV Environment:
jetson-fpv version: d2a0558 dirty
    WFB-ng version: 25.2.25.64703
    MSPOSD version: c28d645 20250313_214314

[lida2003]

version: 7e3cfb0

1. cars.mp4 60FPS source //simulate RTP source
2. 1/2 inference frame ratio, drop 42% frames
3. inference time from CUDA: 9.8 ~ 19.8 ms
4. Deepsort time from CPU: 6.6 ~ 13.3 ms
5. inference time from CPU: max 203 ms???

daniel@daniel-nvidia:~/Work/jetson-fpv$ ./scripts/tools/analysis_raw_log.sh  cars_5nu.log
dos2unix: converting file cars_5nu.log to Unix format...

### Video log reading ...
Processing: [====================] 100%

### Video frame consistency check ...
Processing: [====================] 100%
Video(totl): 1347 frames
Video: frames capture are continuous
Video: frames forward experience loss 570 frames - 42%
Video: specific missing 570 frame are 21 22 23 24 27 28 29 30 31 32 33 34 35 36 37 38 39 40 43 44 45 46 49 50 53 62 63 66 67 74 75 81 82 103 131 134 135 142 149 150 153 154 157 158 162 167 168 171 172 176 181 182 185 186 189 190 196 197 200 201 204 205 229 242 251 252 254 255 256 261 262 263 266 267 270 271 274 275 279 280 281 284 285 286 289 290 291 294 295 296 297 301 302 305 306 307 309 310 311 312 313 316 317 318 321 322 328 329 330 335 336 337 345 347 353 369 370 372 378 391 392 400 404 405 408 409 412 413 416 417 420 421 424 425 430 431 434 435 439 442 443 444 449 450 453 454 457 458 459 468 469 472 474 485 492 501 504 505 508 509 512 513 515 516 517 520 523 524 528 529 532 533 534 537 538 541 543 544 545 550 551 552 555 556 557 559 560 561 562 566 567 568 571 572 573 576 577 583 584 586 587 588 590 591 597 598 600 601 602 605 606 610 611 613 614 617 618 619 622 623 626 627 628 632 633 634 635 638 639 640 641 644 645 649 650 651 652 655 656 657 660 661 662 665 666 667 668 669 671 672 673 676 677 691 693 694 699 700 701 703 704 705 708 709 712 713 716 717 722 728 733 735 737 742 743 746 747 750 751 755 761 762 765 766 768 769 770 772 773 774 776 777 778 781 782 785 786 787 790 791 794 795 798 799 800 802 803 804 807 808 811 812 813 816 817 823 826 827 828 830 831 832 833 836 837 838 839 840 843 844 845 850 853 855 858 859 862 863 864 865 868 869 871 881 885 887 889 890 891 894 895 898 899 902 903 905 906 907 910 911 914 915 918 919 922 923 925 926 927 930 931 933 934 935 936 938 939 941 944 945 948 951 952 953 955 956 957 958 961 964 965 970 972 975 979 988 992 1003 1029 1032 1033 1036 1037 1040 1041 1044 1045 1046 1049 1050 1053 1054 1057 1058 1061 1062 1065 1066 1069 1070 1072 1073 1074 1077 1078 1079 1087 1088 1095 1100 1101 1102 1106 1107 1109 1110 1111 1112 1115 1116 1117 1120 1121 1122 1125 1126 1127 1128 1131 1132 1135 1136 1137 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1151 1152 1155 1156 1162 1163 1164 1165 1168 1169 1171 1172 1173 1174 1180 1183 1184 1185 1187 1188 1189 1192 1193 1196 1197 1198 1201 1202 1205 1206 1207 1210 1211 1212 1215 1216 1219 1220 1221 1224 1225 1229 1230 1232 1233 1234 1236 1237 1238 1239 1242 1243 1247 1248 1249 1252 1254 1255 1256 1260 1263 1264 1265 1267 1268 1271 1272 1273 1276 1277 1281 1282 1285 1286 1287 1290 1291 1292 1294 1295 1296 1301 1302 1306 1307 1309 1310 1311 1312 1315 1316 1317 1319 1320 1321 1322 1325 1326 1329 1330 1333 1334 1335 1337 1338 1339 1344

### Inference consistency check ...
Processing: [====================] 100%
Inference(totl): 1346 frames
Inference(eval): 397 frames - 29%
Inference(skip): 380 frames - 28%
Inference(lost): 570 frames - 42%
Inference(int_min): 2.0 inference/p
Inference(int_max): 2.0 inference/p
Inference(inf_min): 0.009840395003266167 second
Inference(inf_max): 0.019820411998807685 second
Inference(track_min): 0.0006642341613769531 second
Inference(track_max): 0.013324499130249023 second
Inference: frames experience loss
Inference: specific missing 570 frame are 21 22 23 24 27 28 29 30 31 32 33 34 35 36 37 38 39 40 43 44 45 46 49 50 53 62 63 66 67 74 75 81 82 103 131 134 135 142 149 150 153 154 157 158 162 167 168 171 172 176 181 182 185 186 189 190 196 197 200 201 204 205 229 242 251 252 254 255 256 261 262 263 266 267 270 271 274 275 279 280 281 284 285 286 289 290 291 294 295 296 297 301 302 305 306 307 309 310 311 312 313 316 317 318 321 322 328 329 330 335 336 337 345 347 353 369 370 372 378 391 392 400 404 405 408 409 412 413 416 417 420 421 424 425 430 431 434 435 439 442 443 444 449 450 453 454 457 458 459 468 469 472 474 485 492 501 504 505 508 509 512 513 515 516 517 520 523 524 528 529 532 533 534 537 538 541 543 544 545 550 551 552 555 556 557 559 560 561 562 566 567 568 571 572 573 576 577 583 584 586 587 588 590 591 597 598 600 601 602 605 606 610 611 613 614 617 618 619 622 623 626 627 628 632 633 634 635 638 639 640 641 644 645 649 650 651 652 655 656 657 660 661 662 665 666 667 668 669 671 672 673 676 677 691 693 694 699 700 701 703 704 705 708 709 712 713 716 717 722 728 733 735 737 742 743 746 747 750 751 755 761 762 765 766 768 769 770 772 773 774 776 777 778 781 782 785 786 787 790 791 794 795 798 799 800 802 803 804 807 808 811 812 813 816 817 823 826 827 828 830 831 832 833 836 837 838 839 840 843 844 845 850 853 855 858 859 862 863 864 865 868 869 871 881 885 887 889 890 891 894 895 898 899 902 903 905 906 907 910 911 914 915 918 919 922 923 925 926 927 930 931 933 934 935 936 938 939 941 944 945 948 951 952 953 955 956 957 958 961 964 965 970 972 975 979 988 992 1003 1029 1032 1033 1036 1037 1040 1041 1044 1045 1046 1049 1050 1053 1054 1057 1058 1061 1062 1065 1066 1069 1070 1072 1073 1074 1077 1078 1079 1087 1088 1095 1100 1101 1102 1106 1107 1109 1110 1111 1112 1115 1116 1117 1120 1121 1122 1125 1126 1127 1128 1131 1132 1135 1136 1137 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1151 1152 1155 1156 1162 1163 1164 1165 1168 1169 1171 1172 1173 1174 1180 1183 1184 1185 1187 1188 1189 1192 1193 1196 1197 1198 1201 1202 1205 1206 1207 1210 1211 1212 1215 1216 1219 1220 1221 1224 1225 1229 1230 1232 1233 1234 1236 1237 1238 1239 1242 1243 1247 1248 1249 1252 1254 1255 1256 1260 1263 1264 1265 1267 1268 1271 1272 1273 1276 1277 1281 1282 1285 1286 1287 1290 1291 1292 1294 1295 1296 1301 1302 1306 1307 1309 1310 1311 1312 1315 1316 1317 1319 1320 1321 1322 1325 1326 1329 1330 1333 1334 1335 1337 1338 1339 1344
Inference(A_min): 0.000 second
Inference(A_max): 0.001 second
Inference(B_min): 0.000 second
Inference(B_max): 0.004 second
Inference(C_min): 0.000 second
Inference(C_max): 0.203 second
Inference(D_min): 0.000 second
Inference(D_max): 0.012 second
Inference(E_min): 0.001 second
Inference(E_max): 0.015 second
Inference(t_min): 0.001 second
Inference(t_min_id): 1103-th frame
Inference(t_max): 0.208 second
Inference(t_max_id): 1094-th frame

cars_5nu.log

[lida2003]

1. cars.mp4 60FPS source //simulate RTP source
2. 1/2 inference frame ratio, drop 54% frames
3. inference time from CUDA: 8.6 ~ 22.8 ms
4. Deepsort time from CPU: 0.6 ~ 14.6 ms
5. inference time from CPU: max 212 ms???

daniel@daniel-nvidia:~/Work/jetson-fpv$ ./scripts/tools/analysis_raw_log.sh  cars_8n.log
dos2unix: converting file cars_8n.log to Unix format...

### Video log reading ...
Processing: [====================] 100%

### Video frame consistency check ...
Processing: [====================] 100%
Video(totl): 1346 frames
Video: frames capture are continuous
Video: frames forward experience loss 739 frames - 54%
Video: specific missing 739 frame are 21 22 23 26 27 28 29 30 31 32 33 34 35 36 39 40 41 44 45 46 47 50 51 54 55 64 65 68 69 71 80 81 88 89 92 93 96 97 100 101 103 104 105 108 111 112 115 116 119 120 125 126 127 130 131 132 135 136 137 140 141 142 145 146 148 149 150 154 157 158 161 162 165 166 169 170 171 173 174 175 179 180 181 183 184 185 186 190 191 192 196 197 198 202 205 206 207 208 211 212 215 216 218 219 220 222 223 224 225 228 229 230 231 233 234 235 238 239 240 244 245 246 250 251 255 256 259 260 261 264 265 266 270 271 272 273 274 277 278 279 280 282 283 284 285 286 287 288 289 290 291 292 293 297 298 302 305 306 307 310 311 312 315 316 318 319 320 324 325 326 330 331 334 335 338 339 342 343 344 348 350 351 352 353 357 358 362 366 369 370 373 374 377 378 379 383 384 385 389 390 393 394 395 396 397 400 401 402 403 407 408 412 413 416 417 418 421 422 424 428 429 430 433 436 437 438 439 441 442 443 444 447 448 449 450 453 454 455 458 459 460 463 464 465 469 470 474 475 479 480 482 483 484 487 488 489 490 493 494 495 498 499 500 501 503 504 505 506 510 511 512 514 515 516 517 518 521 522 523 526 527 530 531 534 535 536 540 543 544 546 547 548 549 552 553 554 557 558 559 560 565 567 568 569 572 573 576 577 581 582 586 587 590 591 592 593 594 598 599 602 603 606 607 608 610 611 612 613 614 615 616 618 619 620 623 624 627 628 629 632 633 634 637 638 639 640 641 644 645 646 650 651 654 655 658 659 667 669 670 671 675 677 678 679 683 686 687 690 691 694 695 698 699 702 703 706 707 709 710 711 713 714 715 718 719 721 722 723 725 726 727 730 731 733 734 735 738 739 742 743 744 747 748 751 752 753 756 757 758 761 762 763 765 766 767 769 770 771 774 775 776 778 779 780 781 783 784 785 788 789 791 792 793 796 797 798 802 805 806 808 809 810 814 816 817 818 819 820 822 823 824 826 827 828 829 830 834 835 837 838 839 842 843 845 846 847 851 855 859 869 872 873 874 875 877 878 879 882 883 884 886 887 888 891 892 893 894 896 897 898 899 902 903 904 906 907 908 909 910 913 914 915 918 919 920 923 924 925 926 929 930 932 933 934 935 938 939 940 942 943 944 945 946 951 952 955 956 957 962 963 965 966 970 973 974 975 976 979 980 981 982 987 990 991 994 996 997 998 1003 1004 1007 1008 1012 1015 1018 1019 1022 1023 1024 1026 1027 1028 1029 1032 1033 1034 1035 1040 1043 1044 1045 1046 1047 1050 1051 1052 1055 1056 1057 1061 1062 1064 1065 1066 1067 1070 1071 1072 1076 1077 1078 1081 1082 1085 1086 1090 1091 1092 1094 1095 1096 1097 1098 1102 1103 1105 1106 1107 1108 1111 1112 1113 1114 1117 1118 1119 1120 1124 1125 1129 1130 1133 1134 1137 1138 1139 1144 1147 1148 1151 1152 1155 1156 1161 1162 1165 1168 1169 1172 1173 1175 1176 1177 1180 1181 1185 1188 1189 1192 1193 1196 1197 1200 1201 1204 1205 1207 1208 1209 1212 1213 1216 1217 1219 1220 1221 1224 1225 1228 1229 1231 1232 1233 1236 1237 1240 1241 1242 1243 1246 1247 1248 1249 1252 1253 1254 1255 1258 1259 1260 1261 1262 1264 1265 1266 1267 1270 1271 1274 1275 1277 1278 1279 1282 1283 1286 1287 1289 1290 1291 1294 1295 1298 1299 1302 1303 1306 1307 1311 1314 1315 1318 1319 1320 1322 1323 1324 1327 1328 1331 1332 1334 1335 1336 1339 1340 1341 1344 1345

### Inference consistency check ...
Processing: [====================] 100%
Inference(totl): 1346 frames
Inference(eval): 301 frames - 22%
Inference(skip): 307 frames - 22%
Inference(lost): 739 frames - 54%
Inference(int_min): 2.0 inference/p
Inference(int_max): 2.0 inference/p
Inference(inf_min): 0.008606048999354243 second
Inference(inf_max): 0.022875648002809612 second
Inference(track_min): 0.0006525516510009766 second
Inference(track_max): 0.014650106430053711 second
Inference: frames experience loss
Inference: specific missing 739 frame are 21 22 23 26 27 28 29 30 31 32 33 34 35 36 39 40 41 44 45 46 47 50 51 54 55 64 65 68 69 71 80 81 88 89 92 93 96 97 100 101 103 104 105 108 111 112 115 116 119 120 125 126 127 130 131 132 135 136 137 140 141 142 145 146 148 149 150 154 157 158 161 162 165 166 169 170 171 173 174 175 179 180 181 183 184 185 186 190 191 192 196 197 198 202 205 206 207 208 211 212 215 216 218 219 220 222 223 224 225 228 229 230 231 233 234 235 238 239 240 244 245 246 250 251 255 256 259 260 261 264 265 266 270 271 272 273 274 277 278 279 280 282 283 284 285 286 287 288 289 290 291 292 293 297 298 302 305 306 307 310 311 312 315 316 318 319 320 324 325 326 330 331 334 335 338 339 342 343 344 348 350 351 352 353 357 358 362 366 369 370 373 374 377 378 379 383 384 385 389 390 393 394 395 396 397 400 401 402 403 407 408 412 413 416 417 418 421 422 424 428 429 430 433 436 437 438 439 441 442 443 444 447 448 449 450 453 454 455 458 459 460 463 464 465 469 470 474 475 479 480 482 483 484 487 488 489 490 493 494 495 498 499 500 501 503 504 505 506 510 511 512 514 515 516 517 518 521 522 523 526 527 530 531 534 535 536 540 543 544 546 547 548 549 552 553 554 557 558 559 560 565 567 568 569 572 573 576 577 581 582 586 587 590 591 592 593 594 598 599 602 603 606 607 608 610 611 612 613 614 615 616 618 619 620 623 624 627 628 629 632 633 634 637 638 639 640 641 644 645 646 650 651 654 655 658 659 667 669 670 671 675 677 678 679 683 686 687 690 691 694 695 698 699 702 703 706 707 709 710 711 713 714 715 718 719 721 722 723 725 726 727 730 731 733 734 735 738 739 742 743 744 747 748 751 752 753 756 757 758 761 762 763 765 766 767 769 770 771 774 775 776 778 779 780 781 783 784 785 788 789 791 792 793 796 797 798 802 805 806 808 809 810 814 816 817 818 819 820 822 823 824 826 827 828 829 830 834 835 837 838 839 842 843 845 846 847 851 855 859 869 872 873 874 875 877 878 879 882 883 884 886 887 888 891 892 893 894 896 897 898 899 902 903 904 906 907 908 909 910 913 914 915 918 919 920 923 924 925 926 929 930 932 933 934 935 938 939 940 942 943 944 945 946 951 952 955 956 957 962 963 965 966 970 973 974 975 976 979 980 981 982 987 990 991 994 996 997 998 1003 1004 1007 1008 1012 1015 1018 1019 1022 1023 1024 1026 1027 1028 1029 1032 1033 1034 1035 1040 1043 1044 1045 1046 1047 1050 1051 1052 1055 1056 1057 1061 1062 1064 1065 1066 1067 1070 1071 1072 1076 1077 1078 1081 1082 1085 1086 1090 1091 1092 1094 1095 1096 1097 1098 1102 1103 1105 1106 1107 1108 1111 1112 1113 1114 1117 1118 1119 1120 1124 1125 1129 1130 1133 1134 1137 1138 1139 1144 1147 1148 1151 1152 1155 1156 1161 1162 1165 1168 1169 1172 1173 1175 1176 1177 1180 1181 1185 1188 1189 1192 1193 1196 1197 1200 1201 1204 1205 1207 1208 1209 1212 1213 1216 1217 1219 1220 1221 1224 1225 1228 1229 1231 1232 1233 1236 1237 1240 1241 1242 1243 1246 1247 1248 1249 1252 1253 1254 1255 1258 1259 1260 1261 1262 1264 1265 1266 1267 1270 1271 1274 1275 1277 1278 1279 1282 1283 1286 1287 1289 1290 1291 1294 1295 1298 1299 1302 1303 1306 1307 1311 1314 1315 1318 1319 1320 1322 1323 1324 1327 1328 1331 1332 1334 1335 1336 1339 1340 1341 1344 1345
Inference(A_min): 0.000 second
Inference(A_max): 0.000 second
Inference(B_min): 0.000 second
Inference(B_max): 0.013 second
Inference(C_min): 0.000 second
Inference(C_max): 0.208 second
Inference(D_min): 0.000 second
Inference(D_max): 0.013 second
Inference(E_min): 0.001 second
Inference(E_max): 0.015 second
Inference(t_min): 0.002 second
Inference(t_min_id): 25-th frame
Inference(t_max): 0.212 second
Inference(t_max_id): 236-th frame

cars_8n.log

[lida2003]

1. cars.mp4 60FPS source //simulate RTP source
2. 1/2 inference frame ratio, drop 36% frames
3. inference time from CUDA: 9.4 ~ 29.9 ms
4. Deepsort time from CPU: 0.6 ~ 13.8 ms
5. inference time from CPU: max 191 ms???

daniel@daniel-nvidia:~/Work/jetson-fpv$ ./scripts/tools/analysis_raw_log.sh  cars_11n.log
dos2unix: converting file cars_11n.log to Unix format...

### Video log reading ...
Processing: [====================] 100%

### Video frame consistency check ...
Processing: [====================] 100%
Video(totl): 1345 frames
Video: frames capture are continuous
Video: frames forward experience loss 496 frames - 36%
Video: specific missing 496 frame are 21 24 25 26 27 28 29 30 31 32 33 34 37 38 39 40 41 44 45 48 49 52 53 56 59 60 63 64 67 68 72 76 78 79 80 83 84 87 90 91 127 132 133 136 137 140 143 146 147 150 151 156 157 160 161 163 164 167 168 171 172 175 184 185 188 191 192 193 196 197 198 201 202 203 206 207 211 214 231 235 249 252 253 256 257 259 260 263 264 266 267 268 271 272 275 276 279 280 283 284 287 288 291 292 293 294 296 297 298 299 302 303 304 307 308 309 310 313 314 315 316 320 321 322 325 330 331 332 337 340 341 375 376 379 380 383 384 387 388 391 398 399 402 403 406 407 414 415 418 419 423 426 427 438 439 448 450 451 454 455 462 467 468 471 472 475 476 479 480 484 487 488 506 510 511 518 519 522 523 526 527 530 531 534 535 536 537 538 539 540 541 542 545 546 550 551 554 555 558 559 562 565 566 571 572 575 576 580 583 584 591 592 596 599 600 603 604 607 608 611 615 618 619 624 625 628 629 633 636 637 640 643 644 646 647 648 651 652 654 655 656 659 660 663 664 667 668 671 672 676 677 678 681 703 710 711 714 715 724 725 728 729 732 733 735 736 737 752 753 761 766 767 770 771 776 781 782 785 786 789 792 793 795 796 797 800 801 804 805 808 809 810 813 814 818 826 827 828 829 832 833 835 836 837 838 841 842 845 846 850 851 852 855 856 857 858 862 863 864 867 868 871 872 873 874 875 878 881 882 885 886 887 890 891 892 893 896 897 901 904 905 909 912 913 915 916 920 921 924 925 927 928 931 932 935 936 939 940 941 943 944 945 948 949 950 951 954 955 956 958 959 962 963 964 969 970 980 982 1045 1046 1055 1058 1059 1062 1063 1066 1067 1070 1071 1074 1086 1095 1096 1100 1103 1104 1107 1108 1111 1112 1115 1116 1120 1129 1130 1134 1137 1138 1141 1142 1147 1168 1169 1171 1176 1194 1195 1198 1199 1203 1206 1207 1210 1211 1212 1215 1218 1219 1222 1223 1226 1227 1230 1231 1234 1235 1238 1239 1242 1243 1245 1246 1247 1250 1251 1255 1256 1258 1259 1260 1263 1264 1267 1268 1271 1272 1277 1280 1281 1284 1285 1287 1288 1289 1292 1293 1294 1297 1301 1302 1303 1306 1307 1308 1311 1312 1313 1316 1317 1318 1321 1322 1323 1325 1326 1327 1328 1329 1332 1333 1334 1337 1338 1340 1341 1342

### Inference consistency check ...
Processing: [====================] 100%
Inference(totl): 1345 frames
Inference(eval): 423 frames - 31%
Inference(skip): 427 frames - 31%
Inference(lost): 496 frames - 36%
Inference(int_min): 2.0 inference/p
Inference(int_max): 2.0 inference/p
Inference(inf_min): 0.009433435003302293 second
Inference(inf_max): 0.029913353002484655 second
Inference(track_min): 0.0006115436553955078 second
Inference(track_max): 0.013831377029418945 second
Inference: frames experience loss
Inference: specific missing 496 frame are 21 24 25 26 27 28 29 30 31 32 33 34 37 38 39 40 41 44 45 48 49 52 53 56 59 60 63 64 67 68 72 76 78 79 80 83 84 87 90 91 127 132 133 136 137 140 143 146 147 150 151 156 157 160 161 163 164 167 168 171 172 175 184 185 188 191 192 193 196 197 198 201 202 203 206 207 211 214 231 235 249 252 253 256 257 259 260 263 264 266 267 268 271 272 275 276 279 280 283 284 287 288 291 292 293 294 296 297 298 299 302 303 304 307 308 309 310 313 314 315 316 320 321 322 325 330 331 332 337 340 341 375 376 379 380 383 384 387 388 391 398 399 402 403 406 407 414 415 418 419 423 426 427 438 439 448 450 451 454 455 462 467 468 471 472 475 476 479 480 484 487 488 506 510 511 518 519 522 523 526 527 530 531 534 535 536 537 538 539 540 541 542 545 546 550 551 554 555 558 559 562 565 566 571 572 575 576 580 583 584 591 592 596 599 600 603 604 607 608 611 615 618 619 624 625 628 629 633 636 637 640 643 644 646 647 648 651 652 654 655 656 659 660 663 664 667 668 671 672 676 677 678 681 703 710 711 714 715 724 725 728 729 732 733 735 736 737 752 753 761 766 767 770 771 776 781 782 785 786 789 792 793 795 796 797 800 801 804 805 808 809 810 813 814 818 826 827 828 829 832 833 835 836 837 838 841 842 845 846 850 851 852 855 856 857 858 862 863 864 867 868 871 872 873 874 875 878 881 882 885 886 887 890 891 892 893 896 897 901 904 905 909 912 913 915 916 920 921 924 925 927 928 931 932 935 936 939 940 941 943 944 945 948 949 950 951 954 955 956 958 959 962 963 964 969 970 980 982 1045 1046 1055 1058 1059 1062 1063 1066 1067 1070 1071 1074 1086 1095 1096 1100 1103 1104 1107 1108 1111 1112 1115 1116 1120 1129 1130 1134 1137 1138 1141 1142 1147 1168 1169 1171 1176 1194 1195 1198 1199 1203 1206 1207 1210 1211 1212 1215 1218 1219 1222 1223 1226 1227 1230 1231 1234 1235 1238 1239 1242 1243 1245 1246 1247 1250 1251 1255 1256 1258 1259 1260 1263 1264 1267 1268 1271 1272 1277 1280 1281 1284 1285 1287 1288 1289 1292 1293 1294 1297 1301 1302 1303 1306 1307 1308 1311 1312 1313 1316 1317 1318 1321 1322 1323 1325 1326 1327 1328 1329 1332 1333 1334 1337 1338 1340 1341 1342
Inference(A_min): 0.000 second
Inference(A_max): 0.012 second
Inference(B_min): 0.000 second
Inference(B_max): 0.010 second
Inference(C_min): 0.000 second
Inference(C_max): 0.187 second
Inference(D_min): 0.000 second
Inference(D_max): 0.013 second
Inference(E_min): 0.002 second
Inference(E_max): 0.017 second
Inference(t_min): 0.002 second
Inference(t_min_id): 23-th frame
Inference(t_max): 0.191 second
Inference(t_max_id): 502-th frame

cars_11n.log

[lida2003]

yolo: change time.time() to time.perf_counter() as yolo uses, and there is no big difference from previous numbers.

Here is a link about Why is there such a big difference between CUDA inference time and CPU inference time? #19851

1. cars.mp4 60FPS source //simulate RTP source
2. 1/2 inference frame ratio, drop 38% frames
3. inference time from CUDA: 5.4 ~ 25.7 ms
4. Deepsort time from CPU: 0.6 ~ 12.5 ms
5. inference time from CPU: max 194 ms???

daniel@daniel-nvidia:~/Work/jetson-fpv$ ./scripts/tools/analysis_raw_log.sh cars_11n.log
dos2unix: converting file cars_11n.log to Unix format...

### Video log reading ...
Processing: [====================] 100%

### Video frame consistency check ...
Processing: [====================] 100%
Video(totl): 1346 frames
Video: frames capture are continuous
Video: frames forward experience loss 519 frames - 38%
Video: specific missing 519 frame are 21 24 25 26 27 28 29 30 31 32 33 34 35 36 39 40 41 42 43 46 47 50 51 54 55 58 59 64 65 68 69 72 73 75 79 81 82 83 86 89 90 93 94 106 110 112 126 129 141 142 145 146 151 152 157 160 161 164 165 168 169 172 173 186 187 189 190 193 194 195 198 201 202 203 221 226 244 247 248 254 257 260 261 264 265 268 269 272 273 276 280 283 284 287 288 292 294 295 296 299 300 303 304 307 308 311 315 316 322 323 337 340 343 344 349 363 378 379 382 383 386 389 390 392 393 394 397 398 401 402 405 406 409 410 413 414 417 418 421 422 423 424 427 428 429 433 441 444 445 450 455 456 458 459 460 461 465 466 467 470 471 476 477 479 480 481 484 485 488 489 490 491 499 503 510 511 513 514 515 520 521 524 525 528 529 532 533 535 536 537 540 541 542 543 544 545 546 547 548 549 551 552 553 556 557 560 561 562 565 566 567 569 570 573 574 575 578 579 580 583 584 585 588 589 590 591 594 595 596 598 599 600 601 604 605 607 608 609 612 613 614 618 619 621 622 623 626 627 628 629 631 632 633 634 635 639 640 641 643 644 645 646 649 650 651 652 653 657 658 661 662 663 664 666 667 668 669 670 674 675 676 679 680 681 683 684 687 688 690 694 698 701 702 712 713 714 728 729 734 735 738 739 743 746 747 754 768 769 770 773 774 786 789 790 793 794 797 798 802 807 808 811 812 817 818 822 827 828 831 832 836 839 840 843 844 847 848 851 852 855 856 860 863 864 867 868 874 885 886 890 893 894 897 903 906 907 910 911 914 915 918 919 922 923 927 930 931 935 936 939 940 942 943 944 947 948 951 952 955 956 963 964 966 969 1037 1038 1041 1042 1045 1046 1057 1058 1061 1064 1065 1068 1069 1072 1073 1082 1083 1091 1094 1095 1098 1099 1102 1105 1106 1109 1110 1113 1114 1115 1118 1119 1127 1137 1138 1141 1142 1144 1145 1146 1151 1152 1158 1165 1166 1169 1170 1197 1200 1203 1204 1207 1208 1209 1212 1213 1216 1217 1220 1221 1224 1225 1228 1229 1232 1233 1236 1237 1239 1240 1241 1245 1248 1249 1251 1252 1253 1256 1257 1260 1261 1264 1265 1268 1269 1272 1273 1274 1277 1278 1279 1280 1283 1284 1285 1286 1288 1289 1290 1293 1294 1296 1297 1298 1301 1302 1303 1304 1305 1307 1308 1309 1310 1313 1314 1315 1316 1319 1320 1321 1324 1325 1326 1327 1330 1331 1332 1333 1334 1338 1339 1342 1343 1344 1345

### Inference consistency check ...
Processing: [====================] 100%
Inference(totl): 1346 frames
Inference(eval): 416 frames - 30%
Inference(skip): 412 frames - 30%
Inference(lost): 519 frames - 38%
Inference(int_min): 2.0 inference/p
Inference(int_max): 2.0 inference/p
Inference(inf_min): 0.00546443600251223 second
Inference(inf_max): 0.025743756999872858 second
Inference(track_min): 0.0006303129994194023 second
Inference(track_max): 0.012549892999231815 second
Inference: frames experience loss
Inference: specific missing 519 frame are 21 24 25 26 27 28 29 30 31 32 33 34 35 36 39 40 41 42 43 46 47 50 51 54 55 58 59 64 65 68 69 72 73 75 79 81 82 83 86 89 90 93 94 106 110 112 126 129 141 142 145 146 151 152 157 160 161 164 165 168 169 172 173 186 187 189 190 193 194 195 198 201 202 203 221 226 244 247 248 254 257 260 261 264 265 268 269 272 273 276 280 283 284 287 288 292 294 295 296 299 300 303 304 307 308 311 315 316 322 323 337 340 343 344 349 363 378 379 382 383 386 389 390 392 393 394 397 398 401 402 405 406 409 410 413 414 417 418 421 422 423 424 427 428 429 433 441 444 445 450 455 456 458 459 460 461 465 466 467 470 471 476 477 479 480 481 484 485 488 489 490 491 499 503 510 511 513 514 515 520 521 524 525 528 529 532 533 535 536 537 540 541 542 543 544 545 546 547 548 549 551 552 553 556 557 560 561 562 565 566 567 569 570 573 574 575 578 579 580 583 584 585 588 589 590 591 594 595 596 598 599 600 601 604 605 607 608 609 612 613 614 618 619 621 622 623 626 627 628 629 631 632 633 634 635 639 640 641 643 644 645 646 649 650 651 652 653 657 658 661 662 663 664 666 667 668 669 670 674 675 676 679 680 681 683 684 687 688 690 694 698 701 702 712 713 714 728 729 734 735 738 739 743 746 747 754 768 769 770 773 774 786 789 790 793 794 797 798 802 807 808 811 812 817 818 822 827 828 831 832 836 839 840 843 844 847 848 851 852 855 856 860 863 864 867 868 874 885 886 890 893 894 897 903 906 907 910 911 914 915 918 919 922 923 927 930 931 935 936 939 940 942 943 944 947 948 951 952 955 956 963 964 966 969 1037 1038 1041 1042 1045 1046 1057 1058 1061 1064 1065 1068 1069 1072 1073 1082 1083 1091 1094 1095 1098 1099 1102 1105 1106 1109 1110 1113 1114 1115 1118 1119 1127 1137 1138 1141 1142 1144 1145 1146 1151 1152 1158 1165 1166 1169 1170 1197 1200 1203 1204 1207 1208 1209 1212 1213 1216 1217 1220 1221 1224 1225 1228 1229 1232 1233 1236 1237 1239 1240 1241 1245 1248 1249 1251 1252 1253 1256 1257 1260 1261 1264 1265 1268 1269 1272 1273 1274 1277 1278 1279 1280 1283 1284 1285 1286 1288 1289 1290 1293 1294 1296 1297 1298 1301 1302 1303 1304 1305 1307 1308 1309 1310 1313 1314 1315 1316 1319 1320 1321 1324 1325 1326 1327 1330 1331 1332 1333 1334 1338 1339 1342 1343 1344 1345
Inference(A_min): 0.000 second
Inference(A_max): 0.003 second
Inference(B_min): 0.000 second
Inference(B_max): 0.012 second
Inference(C_min): 0.000 second
Inference(C_max): 0.190 second
Inference(D_min): 0.000 second
Inference(D_max): 0.010 second
Inference(E_min): 0.001 second
Inference(E_max): 0.014 second
Inference(t_min): 0.001 second
Inference(t_min_id): 197-th frame
Inference(t_max): 0.194 second
Inference(t_max_id): 502-th frame

cars_11n.log

[lida2003]

1. cars.mp4 60FPS source //simulate RTP source
2. 1/2 inference frame ratio, drop 39% frames
3. inference time from CUDA: 9.4 ~ 28.6 ms
4. Deepsort time from CPU: 0.6 ~ 575.6 ms
5. inference time from CPU: max 167 ms???
6. DeepSort average time: 0.6
7. inference avaerage time:18.9

$ git log -n 1
commit 2414bf7473ff8365fd4b290f898edde48a1f55b0 (HEAD -> main, origin/main, origin/HEAD)
Author: Daniel Li <lida_mail@163.com>
Date:   Wed Mar 26 09:22:49 2025 +0800

    yolo: add deepsort&yolo timeing checks

$ ./scripts/tools/analysis_raw_log.sh cars_11n.log
dos2unix: converting file cars_11n.log to Unix format...

### Video log reading ...
Processing: [====================] 100%

### Video frame consistency check ...
Processing: [====================] 100%
Video(totl): 1346 frames
Video: frames capture are continuous
Video: frames forward experience loss 528 frames - 39%
Video: specific missing 528 frame are 21 22 23 26 27 28 29 30 31 32 33 34 35 38 39 40 41 44 45 46 49 50 53 54 57 58 63 64 67 68 71 72 74 79 80 84 87 88 91 121 128 129 130 137 138 141 142 145 146 151 152 154 155 158 159 162 163 167 170 171 174 177 186 187 188 191 192 193 196 197 198 201 202 203 205 206 207 214 225 231 232 248 249 251 252 253 256 257 262 263 266 267 268 271 272 275 276 279 280 283 284 287 288 289 292 293 295 296 297 300 301 302 305 306 309 313 314 321 322 326 327 331 335 338 340 341 342 345 346 357 361 365 376 377 380 381 384 385 387 388 391 392 394 398 399 400 403 404 407 412 413 416 417 418 421 426 430 431 438 439 440 443 446 451 452 453 456 457 458 462 467 468 471 472 477 478 479 483 486 487 489 490 506 507 508 511 512 514 519 520 523 524 527 530 531 534 535 538 539 540 541 542 543 544 545 546 547 550 551 555 559 562 563 566 567 570 572 573 574 577 578 581 582 585 586 589 592 593 595 596 597 600 602 603 604 607 608 611 616 619 620 623 624 625 628 629 630 632 633 634 637 638 639 641 642 643 644 649 650 653 654 655 656 658 659 660 661 664 665 666 667 671 672 673 676 677 678 680 681 687 700 702 710 711 713 714 715 716 725 726 729 736 737 738 741 742 747 748 751 752 760 768 771 772 786 789 790 793 794 798 801 802 807 808 811 816 817 821 829 832 833 836 837 840 841 843 844 845 848 849 852 853 856 858 859 860 863 864 867 868 869 872 873 874 876 884 885 886 887 890 891 892 893 896 897 898 899 906 908 909 910 911 916 917 920 921 924 925 926 929 930 931 936 937 938 941 942 943 944 947 948 952 953 956 957 958 961 962 963 964 973 1001 1016 1024 1027 1028 1029 1036 1037 1040 1041 1044 1045 1049 1058 1059 1062 1063 1066 1067 1070 1078 1081 1082 1091 1092 1093 1096 1097 1101 1104 1105 1113 1116 1117 1120 1121 1125 1131 1132 1133 1134 1137 1138 1141 1142 1148 1149 1150 1158 1161 1168 1169 1185 1194 1195 1196 1199 1200 1203 1204 1207 1208 1211 1212 1215 1216 1219 1220 1222 1223 1224 1227 1228 1231 1232 1235 1236 1240 1243 1244 1245 1248 1249 1252 1253 1254 1255 1258 1259 1260 1263 1264 1268 1271 1272 1273 1276 1277 1280 1281 1285 1287 1288 1289 1292 1293 1296 1297 1301 1302 1303 1304 1307 1308 1309 1310 1311 1315 1316 1318 1319 1320 1323 1324 1327 1328 1329 1330 1333 1334 1335 1337 1338 1339 1341 1342 1343

### Inference consistency check ...
Processing: [====================] 100%
Inference(totl): 1345 frames
Inference(eval): 410 frames - 30%
Inference(skip): 408 frames - 30%
Inference(lost): 528 frames - 39%
Inference(int_min): 2.0 inference/p
Inference(int_max): 2.0 inference/p
Inference(inf_min): 0.009402479001437314 second
Inference(inf_max): 0.02867423000134295 second
Inference(track_min): 0.000643631999992067 second
Inference(track_max): 0.5756136409981991 second
Inference: frames experience loss
Inference: specific missing 528 frame are 21 22 23 26 27 28 29 30 31 32 33 34 35 38 39 40 41 44 45 46 49 50 53 54 57 58 63 64 67 68 71 72 74 79 80 84 87 88 91 121 128 129 130 137 138 141 142 145 146 151 152 154 155 158 159 162 163 167 170 171 174 177 186 187 188 191 192 193 196 197 198 201 202 203 205 206 207 214 225 231 232 248 249 251 252 253 256 257 262 263 266 267 268 271 272 275 276 279 280 283 284 287 288 289 292 293 295 296 297 300 301 302 305 306 309 313 314 321 322 326 327 331 335 338 340 341 342 345 346 357 361 365 376 377 380 381 384 385 387 388 391 392 394 398 399 400 403 404 407 412 413 416 417 418 421 426 430 431 438 439 440 443 446 451 452 453 456 457 458 462 467 468 471 472 477 478 479 483 486 487 489 490 506 507 508 511 512 514 519 520 523 524 527 530 531 534 535 538 539 540 541 542 543 544 545 546 547 550 551 555 559 562 563 566 567 570 572 573 574 577 578 581 582 585 586 589 592 593 595 596 597 600 602 603 604 607 608 611 616 619 620 623 624 625 628 629 630 632 633 634 637 638 639 641 642 643 644 649 650 653 654 655 656 658 659 660 661 664 665 666 667 671 672 673 676 677 678 680 681 687 700 702 710 711 713 714 715 716 725 726 729 736 737 738 741 742 747 748 751 752 760 768 771 772 786 789 790 793 794 798 801 802 807 808 811 816 817 821 829 832 833 836 837 840 841 843 844 845 848 849 852 853 856 858 859 860 863 864 867 868 869 872 873 874 876 884 885 886 887 890 891 892 893 896 897 898 899 906 908 909 910 911 916 917 920 921 924 925 926 929 930 931 936 937 938 941 942 943 944 947 948 952 953 956 957 958 961 962 963 964 973 1001 1016 1024 1027 1028 1029 1036 1037 1040 1041 1044 1045 1049 1058 1059 1062 1063 1066 1067 1070 1078 1081 1082 1091 1092 1093 1096 1097 1101 1104 1105 1113 1116 1117 1120 1121 1125 1131 1132 1133 1134 1137 1138 1141 1142 1148 1149 1150 1158 1161 1168 1169 1185 1194 1195 1196 1199 1200 1203 1204 1207 1208 1211 1212 1215 1216 1219 1220 1222 1223 1224 1227 1228 1231 1232 1235 1236 1240 1243 1244 1245 1248 1249 1252 1253 1254 1255 1258 1259 1260 1263 1264 1268 1271 1272 1273 1276 1277 1280 1281 1285 1287 1288 1289 1292 1293 1296 1297 1301 1302 1303 1304 1307 1308 1309 1310 1311 1315 1316 1318 1319 1320 1323 1324 1327 1328 1329 1330 1333 1334 1335 1337 1338 1339 1341 1342 1343
Inference(A_min): 0.000 second
Inference(A_max): 0.002 second
Inference(B_min): 0.000 second
Inference(B_max): 0.005 second
Inference(C_min): 0.000 second
Inference(C_max): 0.180 second
Inference(D_min): 0.000 second
Inference(D_max): 0.011 second
Inference(E_min): 0.002 second
Inference(E_max): 0.014 second
Inference(t_min): 37-th frame, 0.002 second
Inference(t_max): 502-th frame, 0.184 second
Inference(t_avg): 0.0274337 second
Inference(ds_only_min): 0.000 second
Inference(ds_only_max): 1213-th frame, 0.010 second
Inference(ds_only_avg): 0.000688725
Inference(ds_inf_min): 0.000 second
Inference(ds_inf_max): 2-th frame, 0.167 second
Inference(ds_inf_avg): 0.0189951 second

cars_11n.log

30FPS is OK for jetson orin, or there is a lot of optimization needed here.