Poor performances and getting stuck

[fede72bari]

Dear,

thank you for sharing your code. I made some modifications just to let it run under Tensorflow 2 avoiding some errors on placeholders and importing the slim library. I run it on CartPole with the default settings, I just increased the max number of steps to see how much it could learn. I have to say that the performance is quite poor in terms of speed, but it could be a problem with my SO and HW configuration, but also in terms of results. And at a certain point, it gets stuck. Even if the log shows a few seconds for episodes it takes hours for each. And now, after 10 runs it has done just 61 episodes, with a number of steps that is just a little bit higher than a random run. Have you ever tried to run it increasing the max number of steps to 5000? Which are your results? Thank you.

[tmoer]

Hi, That's weird indeed! It's year ago I ran this code, but I definitely ran more episodes than 61. In the paper I ran 6 million steps. I'm not sure why its so slow now, but I often had issues with memory leakage in Tensorflow. That could be something to check. Performance being bad could have to do with the reward range of Cartpole. You get a +1 for every step you survive, while UCT is taylored for total returns in [0,1]. Increasing your c parameter in UCT, or scaling your reward function down, might help. Hope that helps! Best regards, Thomas T.M. (Thomas) Moerland Post-doctoral researcher Leiden University https://thomasmoerland.nl<http://thomasmoerland.nl/>

…

________________________________ From: fede72bari ***@***.***> Sent: Monday, April 3, 2023 10:06 AM To: tmoer/alphazero_singleplayer ***@***.***> Cc: Subscribed ***@***.***> Subject: [tmoer/alphazero_singleplayer] Poor performances and getting stuck (Issue #2) Dear, thank you for sharing your code. I made some modifications just to let it run under Tensorflow 2 avoiding some errors on placeholders and importing the slim library. I run it on CartPole with the default settings, I just increased the max number of steps to see how much it could learn. I have to say that the performance is quite poor in terms of speed, but it could be a problem with my SO and HW configuration, but also in terms of results. And at a certain point, it gets stuck. Even if the log shows a few seconds for episodes it takes hours for each. And now, after 10 runs it has done just 61 episodes, with a number of steps that is just a little bit higher than a random run. Have you ever tried to run it increasing the max number of steps to 5000? Which are your results? Thank you. [image]<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fuser-images.githubusercontent.com%2F46624075%2F229449138-1d498a08-0239-422a-96ec-b86aac863665.png&data=05%7C01%7Ct.m.moerland%40liacs.leidenuniv.nl%7Cfa78bc27df0d4593791b08db341a5c7e%7Cca2a7f76dbd74ec091086b3d524fb7c8%7C0%7C0%7C638161060056755733%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=VpN7NcIwDNPXcLM3tJDUW9jSD%2FCfVcxixotXeoFKDCU%3D&reserved=0> — Reply to this email directly, view it on GitHub<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftmoer%2Falphazero_singleplayer%2Fissues%2F2&data=05%7C01%7Ct.m.moerland%40liacs.leidenuniv.nl%7Cfa78bc27df0d4593791b08db341a5c7e%7Cca2a7f76dbd74ec091086b3d524fb7c8%7C0%7C0%7C638161060056755733%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=hNlHjnHiLu9oaKj9ep5KDGw%2BSAPgVD7o0rND2I1hy7Y%3D&reserved=0>, or unsubscribe<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAC35J733TZGNY3X2LKZYMJTW7KAJDANCNFSM6AAAAAAWQ6PPRQ&data=05%7C01%7Ct.m.moerland%40liacs.leidenuniv.nl%7Cfa78bc27df0d4593791b08db341a5c7e%7Cca2a7f76dbd74ec091086b3d524fb7c8%7C0%7C0%7C638161060056755733%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=646RKh42P2j29i93ug6Jwe42ngZfjU15ra69u156Rvc%3D&reserved=0>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[fede72bari]

Dear Thomas, thank you again for having shared your work with the international community and for answering to my post. I didn't give up and let the script run to see what could happen further. I copy at the end of this answer the ongoing log, right now it has been running for almost 2 days. As a beginner, I note this 1. when the episode achieves poor results (few steps) the script is relatively fast, seconds or even less 2. when the episode reaches the limit of 5000 steps it takes hours, even 10 hours 3. I noted also a very strange thing, we had two consecutive episodes (112 and 113) reaching the limit of 5000 steps, but one is 10 times faster with respect to the other. At equal results, we have very different execution time. 4. it is very unstable, it could reach the maximum steps, which I assume to be "full learning reached", and then forget everything and drop down to 20 steps. Launching the script I see this warning that could give suggestion about the reason for so slow performance 2023-04-03 01:29:30.414774: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX AVX2 To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2023-04-03 01:29:30.422381: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:357] MLIR V1 optimization pass is not enabled Do you think I can adjust something to let it work in a more predictable (in terms of computational effort), faster and more stable (avoiding catastrofical forget) way? Thank you very much. Federico. Here the log: Finished episode 0, total return: 81.0, total time: 1.7 sec Finished episode 1, total return: 11.0, total time: 0.1 sec Finished episode 2, total return: 9.0, total time: 0.1 sec Finished episode 3, total return: 10.0, total time: 0.1 sec Finished episode 4, total return: 11.0, total time: 0.1 sec Finished episode 5, total return: 11.0, total time: 0.2 sec Finished episode 6, total return: 10.0, total time: 0.1 sec Finished episode 7, total return: 9.0, total time: 0.1 sec Finished episode 8, total return: 8.0, total time: 0.1 sec Finished episode 9, total return: 11.0, total time: 0.1 sec Finished episode 10, total return: 9.0, total time: 0.1 sec Finished episode 11, total return: 86.0, total time: 2.6 sec Finished episode 12, total return: 95.0, total time: 3.0 sec Finished episode 13, total return: 14.0, total time: 0.2 sec Finished episode 14, total return: 115.0, total time: 2.8 sec Finished episode 15, total return: 11.0, total time: 0.1 sec Finished episode 16, total return: 12.0, total time: 0.1 sec Finished episode 17, total return: 23.0, total time: 0.4 sec Finished episode 18, total return: 21.0, total time: 0.4 sec Finished episode 19, total return: 13.0, total time: 0.2 sec Finished episode 20, total return: 23.0, total time: 0.3 sec Finished episode 21, total return: 18.0, total time: 0.2 sec Finished episode 22, total return: 20.0, total time: 0.2 sec Finished episode 23, total return: 32.0, total time: 0.5 sec Finished episode 24, total return: 22.0, total time: 0.3 sec Finished episode 25, total return: 20.0, total time: 0.3 sec Finished episode 26, total return: 21.0, total time: 0.3 sec Finished episode 27, total return: 34.0, total time: 0.7 sec Finished episode 28, total return: 25.0, total time: 0.4 sec Finished episode 29, total return: 22.0, total time: 0.3 sec Finished episode 30, total return: 18.0, total time: 0.3 sec Finished episode 31, total return: 21.0, total time: 0.3 sec Finished episode 32, total return: 24.0, total time: 0.3 sec Finished episode 33, total return: 31.0, total time: 0.6 sec Finished episode 34, total return: 16.0, total time: 0.3 sec Finished episode 35, total return: 39.0, total time: 0.7 sec Finished episode 36, total return: 18.0, total time: 0.3 sec Finished episode 37, total return: 17.0, total time: 0.2 sec Finished episode 38, total return: 21.0, total time: 0.3 sec Finished episode 39, total return: 25.0, total time: 0.3 sec Finished episode 40, total return: 30.0, total time: 0.5 sec Finished episode 41, total return: 26.0, total time: 0.5 sec Finished episode 42, total return: 31.0, total time: 0.5 sec Finished episode 43, total return: 127.0, total time: 2.6 sec Finished episode 44, total return: 221.0, total time: 17.0 sec Finished episode 45, total return: 22.0, total time: 0.4 sec Finished episode 46, total return: 88.0, total time: 1.7 sec Finished episode 47, total return: 15.0, total time: 0.3 sec Finished episode 48, total return: 13.0, total time: 0.2 sec Finished episode 49, total return: 56.0, total time: 0.9 sec Finished episode 50, total return: 19.0, total time: 0.3 sec Finished episode 51, total return: 20.0, total time: 0.4 sec Finished episode 52, total return: 47.0, total time: 0.7 sec Finished episode 53, total return: 21.0, total time: 0.3 sec Finished episode 54, total return: 40.0, total time: 0.6 sec Finished episode 55, total return: 42.0, total time: 0.9 sec Finished episode 56, total return: 13.0, total time: 0.2 sec Finished episode 57, total return: 40.0, total time: 0.6 sec Finished episode 58, total return: 63.0, total time: 1.5 sec Finished episode 59, total return: 80.0, total time: 1.4 sec Finished episode 60, total return: 36.0, total time: 0.9 sec Finished episode 61, total return: 81.0, total time: 1.3 sec Finished episode 62, total return: 5000.0, total time: 36745.2 sec Finished episode 63, total return: 92.0, total time: 3.1 sec Finished episode 64, total return: 34.0, total time: 0.7 sec Finished episode 65, total return: 24.0, total time: 0.4 sec Finished episode 66, total return: 32.0, total time: 0.6 sec Finished episode 67, total return: 103.0, total time: 2.1 sec Finished episode 68, total return: 17.0, total time: 0.2 sec Finished episode 69, total return: 10.0, total time: 0.1 sec Finished episode 70, total return: 17.0, total time: 0.3 sec Finished episode 71, total return: 13.0, total time: 0.2 sec Finished episode 72, total return: 29.0, total time: 0.5 sec Finished episode 73, total return: 52.0, total time: 0.9 sec Finished episode 74, total return: 117.0, total time: 1.9 sec Finished episode 75, total return: 79.0, total time: 1.5 sec Finished episode 76, total return: 122.0, total time: 2.2 sec Finished episode 77, total return: 177.0, total time: 6.5 sec Finished episode 78, total return: 77.0, total time: 1.4 sec Finished episode 79, total return: 71.0, total time: 1.2 sec Finished episode 80, total return: 17.0, total time: 0.2 sec Finished episode 81, total return: 44.0, total time: 0.8 sec Finished episode 82, total return: 28.0, total time: 0.5 sec Finished episode 83, total return: 58.0, total time: 1.0 sec Finished episode 84, total return: 11.0, total time: 0.2 sec Finished episode 85, total return: 14.0, total time: 0.2 sec Finished episode 86, total return: 10.0, total time: 0.2 sec Finished episode 87, total return: 11.0, total time: 0.2 sec Finished episode 88, total return: 29.0, total time: 0.5 sec Finished episode 89, total return: 49.0, total time: 0.9 sec Finished episode 90, total return: 18.0, total time: 0.3 sec Finished episode 91, total return: 21.0, total time: 0.4 sec Finished episode 92, total return: 41.0, total time: 1.0 sec Finished episode 93, total return: 83.0, total time: 1.2 sec Finished episode 94, total return: 305.0, total time: 9.6 sec Finished episode 95, total return: 46.0, total time: 0.9 sec Finished episode 96, total return: 119.0, total time: 4.1 sec Finished episode 97, total return: 11.0, total time: 0.2 sec Finished episode 98, total return: 229.0, total time: 6.9 sec Finished episode 99, total return: 294.0, total time: 25.2 sec Finished episode 100, total return: 43.0, total time: 0.7 sec Finished episode 101, total return: 52.0, total time: 0.8 sec Finished episode 102, total return: 32.0, total time: 0.6 sec Finished episode 103, total return: 12.0, total time: 0.2 sec Finished episode 104, total return: 20.0, total time: 0.4 sec Finished episode 105, total return: 12.0, total time: 0.2 sec Finished episode 106, total return: 112.0, total time: 2.5 sec Finished episode 107, total return: 35.0, total time: 0.6 sec Finished episode 108, total return: 38.0, total time: 0.7 sec Finished episode 109, total return: 70.0, total time: 1.1 sec Finished episode 110, total return: 275.0, total time: 8.1 sec Finished episode 111, total return: 52.0, total time: 1.0 sec Finished episode 112, total return: *5000.0, total time: 2791.7 sec* Finished episode 113, total return: *5000.0, total time: 273.2 sec* Finished episode 114, total return: 144.0, total time: 2.8 sec Finished episode 115, total return: 63.0, total time: 1.0 sec Finished episode 116, total return: 75.0, total time: 1.3 sec Finished episode 117, total return: 35.0, total time: 0.6 sec Finished episode 118, total return: 83.0, total time: 1.4 sec Finished episode 119, total return: 46.0, total time: 0.7 sec Finished episode 120, total return: 98.0, total time: 1.6 sec Finished episode 121, total return: 80.0, total time: 1.3 sec Finished episode 122, total return: 5000.0, total time: 621.0 sec Finished episode 123, total return: 44.0, total time: 0.8 sec Finished episode 124, total return: 30.0, total time: 0.5 sec Finished episode 125, total return: 46.0, total time: 0.8 sec Finished episode 126, total return: 17.0, total time: 0.2 sec Finished episode 127, total return: 234.0, total time: 6.3 sec Finished episode 128, total return: 5000.0, total time: 26232.6 sec Finished episode 129, total return: 261.0, total time: 6.1 sec Finished episode 130, total return: 5000.0, total time: 13568.5 sec Finished episode 131, total return: 5000.0, total time: 17371.6 sec Finished episode 132, total return: 5000.0, total time: 3384.3 sec Finished episode 133, total return: 82.0, total time: 1.3 sec Finished episode 134, total return: 27.0, total time: 0.5 sec Finished episode 135, total return: 46.0, total time: 0.8 sec Finished episode 136, total return: 168.0, total time: 10.9 sec Finished episode 137, total return: 5000.0, total time: 3458.9 sec Finished episode 138, total return: 124.0, total time: 1.9 sec Finished episode 139, total return: 53.0, total time: 0.8 sec Finished episode 140, total return: 4158.0, total time: 3789.5 sec Finished episode 141, total return: 78.0, total time: 1.2 sec Finished episode 142, total return: 56.0, total time: 0.8 sec Finished episode 143, total return: 16.0, total time: 0.2 sec Finished episode 144, total return: 51.0, total time: 0.8 sec Finished episode 145, total return: 49.0, total time: 0.8 sec Finished episode 146, total return: 13.0, total time: 0.2 sec Finished episode 147, total return: 84.0, total time: 1.4 sec Finished episode 148, total return: 1100.0, total time: 52.8 sec Finished episode 149, total return: 44.0, total time: 0.9 sec

[tmoer]

Hi Federico, I'd not run 5000 steps per episode, but 500 or 200 on Cartpole (I guess that's the default?). It's weird that similar episode lengths have such different runtime. I have no clue what that could be. You'd have to log the runtime of each function call to figure out where the issue happens. But the first thing I would look at it memory leakage. Can you check whether your RAM grows full after a while? Best, Thomas T.M. (Thomas) Moerland Post-doctoral researcher Leiden University https://thomasmoerland.nl<http://thomasmoerland.nl/>

________________________________ From: fede72bari ***@***.***> Sent: Tuesday, April 4, 2023 11:05 AM To: tmoer/alphazero_singleplayer ***@***.***> Cc: Moerland, T.M. (Thomas) ***@***.***>; Comment ***@***.***> Subject: Re: [tmoer/alphazero_singleplayer] Poor performances and getting stuck (Issue #2) Dear Thomas, thank you again for having shared your work with the international community and for answering to my post. My email is ***@***.*** I didn't give up and let the script run to see what could happen further. I copy at the end of this answer the on-going log, right now it has been running for almost 2 days. As a beginner I note this 1. when the episode achieves poor results (few steps) the script is relative fast, seconds or even less 2. when the episode reaches the limit of 5000 steps it takes hours, even 10 hours 3. I noted also a very strange thig, we had two consecutive episodes (112 and 113) reaching the limit of 5000 steps, but one is 10 time faster with respect tto he other. At equal result we have very different execution time. 4. it is very unstable, it could reach the maximum steps, that I assume to be "full learning reached", and then forget everything and dropping down to 20 steps. Do you think I can adjust something to let it work in more predictable (in terms of computational effort), faster and stable (avoiding catastrofical forget) way? Thank you very much. Federico.

------------------------------------------------------------- Here the log: Finished episode 0, total return: 81.0, total time: 1.7 sec Finished episode 1, total return: 11.0, total time: 0.1 sec Finished episode 2, total return: 9.0, total time: 0.1 sec Finished episode 3, total return: 10.0, total time: 0.1 sec Finished episode 4, total return: 11.0, total time: 0.1 sec Finished episode 5, total return: 11.0, total time: 0.2 sec Finished episode 6, total return: 10.0, total time: 0.1 sec Finished episode 7, total return: 9.0, total time: 0.1 sec Finished episode 8, total return: 8.0, total time: 0.1 sec Finished episode 9, total return: 11.0, total time: 0.1 sec Finished episode 10, total return: 9.0, total time: 0.1 sec Finished episode 11, total return: 86.0, total time: 2.6 sec Finished episode 12, total return: 95.0, total time: 3.0 sec Finished episode 13, total return: 14.0, total time: 0.2 sec Finished episode 14, total return: 115.0, total time: 2.8 sec Finished episode 15, total return: 11.0, total time: 0.1 sec Finished episode 16, total return: 12.0, total time: 0.1 sec Finished episode 17, total return: 23.0, total time: 0.4 sec Finished episode 18, total return: 21.0, total time: 0.4 sec Finished episode 19, total return: 13.0, total time: 0.2 sec Finished episode 20, total return: 23.0, total time: 0.3 sec Finished episode 21, total return: 18.0, total time: 0.2 sec Finished episode 22, total return: 20.0, total time: 0.2 sec Finished episode 23, total return: 32.0, total time: 0.5 sec Finished episode 24, total return: 22.0, total time: 0.3 sec Finished episode 25, total return: 20.0, total time: 0.3 sec Finished episode 26, total return: 21.0, total time: 0.3 sec Finished episode 27, total return: 34.0, total time: 0.7 sec Finished episode 28, total return: 25.0, total time: 0.4 sec Finished episode 29, total return: 22.0, total time: 0.3 sec Finished episode 30, total return: 18.0, total time: 0.3 sec Finished episode 31, total return: 21.0, total time: 0.3 sec Finished episode 32, total return: 24.0, total time: 0.3 sec Finished episode 33, total return: 31.0, total time: 0.6 sec Finished episode 34, total return: 16.0, total time: 0.3 sec Finished episode 35, total return: 39.0, total time: 0.7 sec Finished episode 36, total return: 18.0, total time: 0.3 sec Finished episode 37, total return: 17.0, total time: 0.2 sec Finished episode 38, total return: 21.0, total time: 0.3 sec Finished episode 39, total return: 25.0, total time: 0.3 sec Finished episode 40, total return: 30.0, total time: 0.5 sec Finished episode 41, total return: 26.0, total time: 0.5 sec Finished episode 42, total return: 31.0, total time: 0.5 sec Finished episode 43, total return: 127.0, total time: 2.6 sec Finished episode 44, total return: 221.0, total time: 17.0 sec Finished episode 45, total return: 22.0, total time: 0.4 sec Finished episode 46, total return: 88.0, total time: 1.7 sec Finished episode 47, total return: 15.0, total time: 0.3 sec Finished episode 48, total return: 13.0, total time: 0.2 sec Finished episode 49, total return: 56.0, total time: 0.9 sec Finished episode 50, total return: 19.0, total time: 0.3 sec Finished episode 51, total return: 20.0, total time: 0.4 sec Finished episode 52, total return: 47.0, total time: 0.7 sec Finished episode 53, total return: 21.0, total time: 0.3 sec Finished episode 54, total return: 40.0, total time: 0.6 sec Finished episode 55, total return: 42.0, total time: 0.9 sec Finished episode 56, total return: 13.0, total time: 0.2 sec Finished episode 57, total return: 40.0, total time: 0.6 sec Finished episode 58, total return: 63.0, total time: 1.5 sec Finished episode 59, total return: 80.0, total time: 1.4 sec Finished episode 60, total return: 36.0, total time: 0.9 sec Finished episode 61, total return: 81.0, total time: 1.3 sec Finished episode 62, total return: 5000.0, total time: 36745.2 sec Finished episode 63, total return: 92.0, total time: 3.1 sec Finished episode 64, total return: 34.0, total time: 0.7 sec Finished episode 65, total return: 24.0, total time: 0.4 sec Finished episode 66, total return: 32.0, total time: 0.6 sec Finished episode 67, total return: 103.0, total time: 2.1 sec Finished episode 68, total return: 17.0, total time: 0.2 sec Finished episode 69, total return: 10.0, total time: 0.1 sec Finished episode 70, total return: 17.0, total time: 0.3 sec Finished episode 71, total return: 13.0, total time: 0.2 sec Finished episode 72, total return: 29.0, total time: 0.5 sec Finished episode 73, total return: 52.0, total time: 0.9 sec Finished episode 74, total return: 117.0, total time: 1.9 sec Finished episode 75, total return: 79.0, total time: 1.5 sec Finished episode 76, total return: 122.0, total time: 2.2 sec Finished episode 77, total return: 177.0, total time: 6.5 sec Finished episode 78, total return: 77.0, total time: 1.4 sec Finished episode 79, total return: 71.0, total time: 1.2 sec Finished episode 80, total return: 17.0, total time: 0.2 sec Finished episode 81, total return: 44.0, total time: 0.8 sec Finished episode 82, total return: 28.0, total time: 0.5 sec Finished episode 83, total return: 58.0, total time: 1.0 sec Finished episode 84, total return: 11.0, total time: 0.2 sec Finished episode 85, total return: 14.0, total time: 0.2 sec Finished episode 86, total return: 10.0, total time: 0.2 sec Finished episode 87, total return: 11.0, total time: 0.2 sec Finished episode 88, total return: 29.0, total time: 0.5 sec Finished episode 89, total return: 49.0, total time: 0.9 sec Finished episode 90, total return: 18.0, total time: 0.3 sec Finished episode 91, total return: 21.0, total time: 0.4 sec Finished episode 92, total return: 41.0, total time: 1.0 sec Finished episode 93, total return: 83.0, total time: 1.2 sec Finished episode 94, total return: 305.0, total time: 9.6 sec Finished episode 95, total return: 46.0, total time: 0.9 sec Finished episode 96, total return: 119.0, total time: 4.1 sec Finished episode 97, total return: 11.0, total time: 0.2 sec Finished episode 98, total return: 229.0, total time: 6.9 sec Finished episode 99, total return: 294.0, total time: 25.2 sec Finished episode 100, total return: 43.0, total time: 0.7 sec Finished episode 101, total return: 52.0, total time: 0.8 sec Finished episode 102, total return: 32.0, total time: 0.6 sec Finished episode 103, total return: 12.0, total time: 0.2 sec Finished episode 104, total return: 20.0, total time: 0.4 sec Finished episode 105, total return: 12.0, total time: 0.2 sec Finished episode 106, total return: 112.0, total time: 2.5 sec Finished episode 107, total return: 35.0, total time: 0.6 sec Finished episode 108, total return: 38.0, total time: 0.7 sec Finished episode 109, total return: 70.0, total time: 1.1 sec Finished episode 110, total return: 275.0, total time: 8.1 sec Finished episode 111, total return: 52.0, total time: 1.0 sec Finished episode 112, total return: *5000.0, total time: 2791.7 sec* Finished episode 113, total return: *5000.0, total time: 273.2 sec* Finished episode 114, total return: 144.0, total time: 2.8 sec Finished episode 115, total return: 63.0, total time: 1.0 sec Finished episode 116, total return: 75.0, total time: 1.3 sec Finished episode 117, total return: 35.0, total time: 0.6 sec Finished episode 118, total return: 83.0, total time: 1.4 sec Finished episode 119, total return: 46.0, total time: 0.7 sec Finished episode 120, total return: 98.0, total time: 1.6 sec Finished episode 121, total return: 80.0, total time: 1.3 sec Finished episode 122, total return: 5000.0, total time: 621.0 sec Finished episode 123, total return: 44.0, total time: 0.8 sec Finished episode 124, total return: 30.0, total time: 0.5 sec Finished episode 125, total return: 46.0, total time: 0.8 sec Finished episode 126, total return: 17.0, total time: 0.2 sec Finished episode 127, total return: 234.0, total time: 6.3 sec Finished episode 128, total return: 5000.0, total time: 26232.6 sec Finished episode 129, total return: 261.0, total time: 6.1 sec Finished episode 130, total return: 5000.0, total time: 13568.5 sec Finished episode 131, total return: 5000.0, total time: 17371.6 sec Finished episode 132, total return: 5000.0, total time: 3384.3 sec Finished episode 133, total return: 82.0, total time: 1.3 sec Finished episode 134, total return: 27.0, total time: 0.5 sec Finished episode 135, total return: 46.0, total time: 0.8 sec Finished episode 136, total return: 168.0, total time: 10.9 sec Finished episode 137, total return: 5000.0, total time: 3458.9 sec Finished episode 138, total return: 124.0, total time: 1.9 sec Finished episode 139, total return: 53.0, total time: 0.8 sec Finished episode 140, total return: 4158.0, total time: 3789.5 sec Finished episode 141, total return: 78.0, total time: 1.2 sec Finished episode 142, total return: 56.0, total time: 0.8 sec Finished episode 143, total return: 16.0, total time: 0.2 sec Finished episode 144, total return: 51.0, total time: 0.8 sec Finished episode 145, total return: 49.0, total time: 0.8 sec Finished episode 146, total return: 13.0, total time: 0.2 sec Finished episode 147, total return: 84.0, total time: 1.4 sec Finished episode 148, total return: 1100.0, total time: 52.8 sec Finished episode 149, total return: 44.0, total time: 0.9 sec Il giorno lun 3 apr 2023 alle ore 15:57 Thomas Moerland < ***@***.***> ha scritto:

Hi, That's weird indeed! It's year ago I ran this code, but I definitely ran more episodes than 61. In the paper I ran 6 million steps. I'm not sure why its so slow now, but I often had issues with memory leakage in Tensorflow. That could be something to check. Performance being bad could have to do with the reward range of Cartpole. You get a +1 for every step you survive, while UCT is taylored for total returns in [0,1]. Increasing your c parameter in UCT, or scaling your reward function down, might help. Hope that helps! Best regards, Thomas T.M. (Thomas) Moerland Post-doctoral researcher Leiden University https://thomasmoerland.nl<http://thomasmoerland.nl/> ________________________________ From: fede72bari ***@***.***> Sent: Monday, April 3, 2023 10:06 AM To: tmoer/alphazero_singleplayer ***@***.***> Cc: Subscribed ***@***.***> Subject: [tmoer/alphazero_singleplayer] Poor performances and getting stuck (Issue #2) Dear, thank you for sharing your code. I made some modifications just to let it run under Tensorflow 2 avoiding some errors on placeholders and importing the slim library. I run it on CartPole with the default settings, I just increased the max number of steps to see how much it could learn. I have to say that the performance is quite poor in terms of speed, but it could be a problem with my SO and HW configuration, but also in terms of results. And at a certain point, it gets stuck. Even if the log shows a few seconds for episodes it takes hours for each. And now, after 10 runs it has done just 61 episodes, with a number of steps that is just a little bit higher than a random run. Have you ever tried to run it increasing the max number of steps to 5000? Which are your results? Thank you. [image]< https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fuser-images.githubusercontent.com%2F46624075%2F229449138-1d498a08-0239-422a-96ec-b86aac863665.png&data=05%7C01%7Ct.m.moerland%40liacs.leidenuniv.nl%7Cfa78bc27df0d4593791b08db341a5c7e%7Cca2a7f76dbd74ec091086b3d524fb7c8%7C0%7C0%7C638161060056755733%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=VpN7NcIwDNPXcLM3tJDUW9jSD%2FCfVcxixotXeoFKDCU%3D&reserved=0 > — Reply to this email directly, view it on GitHub< https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftmoer%2Falphazero_singleplayer%2Fissues%2F2&data=05%7C01%7Ct.m.moerland%40liacs.leidenuniv.nl%7Cfa78bc27df0d4593791b08db341a5c7e%7Cca2a7f76dbd74ec091086b3d524fb7c8%7C0%7C0%7C638161060056755733%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=hNlHjnHiLu9oaKj9ep5KDGw%2BSAPgVD7o0rND2I1hy7Y%3D&reserved=0>, or unsubscribe< https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAC35J733TZGNY3X2LKZYMJTW7KAJDANCNFSM6AAAAAAWQ6PPRQ&data=05%7C01%7Ct.m.moerland%40liacs.leidenuniv.nl%7Cfa78bc27df0d4593791b08db341a5c7e%7Cca2a7f76dbd74ec091086b3d524fb7c8%7C0%7C0%7C638161060056755733%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=646RKh42P2j29i93ug6Jwe42ngZfjU15ra69u156Rvc%3D&reserved=0 >. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***> — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALDW2S3RMSF4PR24FO7GMO3W7LJOBANCNFSM6AAAAAAWQ6PPRQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

— Reply to this email directly, view it on GitHub<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftmoer%2Falphazero_singleplayer%2Fissues%2F2%23issuecomment-1495611278&data=05%7C01%7Ct.m.moerland%40liacs.leidenuniv.nl%7Cb8d64292c0ef46027d7808db34ebc076%7Cca2a7f76dbd74ec091086b3d524fb7c8%7C0%7C0%7C638161959377880013%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=DTK51tHlEQZGK0Y46OpZ5lyBoH%2B80iL0iOrDxkZaLu8%3D&reserved=0>, or unsubscribe<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAC35J7ZO7LFT3YY4R7CZ32TW7PP55ANCNFSM6AAAAAAWQ6PPRQ&data=05%7C01%7Ct.m.moerland%40liacs.leidenuniv.nl%7Cb8d64292c0ef46027d7808db34ebc076%7Cca2a7f76dbd74ec091086b3d524fb7c8%7C0%7C0%7C638161959377880013%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=YxZ5hemyRAee9aL7qqW7VaMKhcfbpfIMXR%2BxfXejj2o%3D&reserved=0>. You are receiving this because you commented.Message ID: ***@***.***>

[fede72bari]

Hi again,

as suggested i tracked the RSS (physical) and VMS (virtual) memory used by the running process at the end of each episode and kept track of the max values for each step of each episode. The long log is at the bottom. This time I run 400 max episodes with max length per episode of 200 steps

python alphazero.py --n_ep 400 --n_mcts 10 --max_ep_len 200

What I can see is that the RSS memory slightly increases, more often and with higher values when there is a longer execution time; but it is not a certain rule. At a certain point, RSS stops to increase. It never decreases. The VMS increase but at a certain point it increases and decreases alternately. I think that the reason for the delay is not memory leakage. Could you share the performance in your system for "max_ep_len 500"? Also, do you have the same warning I see?

    2023-04-03 01:29:30.414774: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX AVX2
    To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
   2023-04-03 01:29:30.422381: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:357] MLIR V1 optimization pass is not enabled

Here the log

Finished episode 0, total return: 124.0, total time: 6.0 sec
        Endof Episode VMS: 276656128, Max VMS this Episode: 276656128, VMS DELTA: 276656128
        End of Episode RSS: 346714112, Max RSS this Episode: 346656768, RSS DELTA: 346714112
Finished episode 1, total return: 86.0, total time: 3.8 sec
        Endof Episode VMS: 284618752, Max VMS this Episode: 284618752, VMS DELTA: 7962624
        End of Episode RSS: 353202176, Max RSS this Episode: 353202176, RSS DELTA: 6488064
Finished episode 2, total return: 94.0, total time: 3.6 sec
        Endof Episode VMS: 284696576, Max VMS this Episode: 284696576, VMS DELTA: 77824
        End of Episode RSS: 353587200, Max RSS this Episode: 353587200, RSS DELTA: 385024
Finished episode 3, total return: 117.0, total time: 5.8 sec
        Endof Episode VMS: 285777920, Max VMS this Episode: 285777920, VMS DELTA: 1081344
        End of Episode RSS: 354570240, Max RSS this Episode: 354570240, RSS DELTA: 983040
Finished episode 4, total return: 23.0, total time: 0.5 sec
        Endof Episode VMS: 285999104, Max VMS this Episode: 285999104, VMS DELTA: 221184
        End of Episode RSS: 354828288, Max RSS this Episode: 354828288, RSS DELTA: 258048
Finished episode 5, total return: 16.0, total time: 0.4 sec
        Endof Episode VMS: 286228480, Max VMS this Episode: 286228480, VMS DELTA: 229376
        End of Episode RSS: 355037184, Max RSS this Episode: 355037184, RSS DELTA: 208896
Finished episode 6, total return: 11.0, total time: 0.2 sec
        Endof Episode VMS: 286228480, Max VMS this Episode: 286228480, VMS DELTA: 0
        End of Episode RSS: 355057664, Max RSS this Episode: 355057664, RSS DELTA: 20480
Finished episode 7, total return: 176.0, total time: 11.4 sec
        Endof Episode VMS: 287346688, Max VMS this Episode: 287346688, VMS DELTA: 1118208
        End of Episode RSS: 355459072, Max RSS this Episode: 355459072, RSS DELTA: 401408
Finished episode 8, total return: 11.0, total time: 0.3 sec
        Endof Episode VMS: 287371264, Max VMS this Episode: 287371264, VMS DELTA: 24576
        End of Episode RSS: 355713024, Max RSS this Episode: 355713024, RSS DELTA: 253952
Finished episode 9, total return: 12.0, total time: 0.3 sec
        Endof Episode VMS: 287547392, Max VMS this Episode: 287547392, VMS DELTA: 176128
        End of Episode RSS: 355893248, Max RSS this Episode: 355893248, RSS DELTA: 180224
Finished episode 10, total return: 11.0, total time: 0.2 sec
        Endof Episode VMS: 287698944, Max VMS this Episode: 287698944, VMS DELTA: 151552
        End of Episode RSS: 356102144, Max RSS this Episode: 356102144, RSS DELTA: 208896
Finished episode 11, total return: 11.0, total time: 0.2 sec
        Endof Episode VMS: 287698944, Max VMS this Episode: 287698944, VMS DELTA: 0
        End of Episode RSS: 356212736, Max RSS this Episode: 356212736, RSS DELTA: 110592
Finished episode 12, total return: 30.0, total time: 0.6 sec
        Endof Episode VMS: 287698944, Max VMS this Episode: 287698944, VMS DELTA: 0
        End of Episode RSS: 356438016, Max RSS this Episode: 356438016, RSS DELTA: 225280
Finished episode 13, total return: 8.0, total time: 0.2 sec
        Endof Episode VMS: 287698944, Max VMS this Episode: 287698944, VMS DELTA: 0
        End of Episode RSS: 356438016, Max RSS this Episode: 356438016, RSS DELTA: 0
Finished episode 14, total return: 13.0, total time: 0.2 sec
        Endof Episode VMS: 287723520, Max VMS this Episode: 287723520, VMS DELTA: 24576
        End of Episode RSS: 356491264, Max RSS this Episode: 356491264, RSS DELTA: 53248
Finished episode 15, total return: 12.0, total time: 0.3 sec
        Endof Episode VMS: 287723520, Max VMS this Episode: 287723520, VMS DELTA: 0
        End of Episode RSS: 356491264, Max RSS this Episode: 356491264, RSS DELTA: 0
Finished episode 16, total return: 9.0, total time: 0.2 sec
        Endof Episode VMS: 287723520, Max VMS this Episode: 287723520, VMS DELTA: 0
        End of Episode RSS: 356511744, Max RSS this Episode: 356511744, RSS DELTA: 20480
Finished episode 17, total return: 57.0, total time: 1.4 sec
        Endof Episode VMS: 288772096, Max VMS this Episode: 288772096, VMS DELTA: 1048576
        End of Episode RSS: 356716544, Max RSS this Episode: 356716544, RSS DELTA: 204800
Finished episode 18, total return: 11.0, total time: 0.2 sec
        Endof Episode VMS: 288870400, Max VMS this Episode: 288870400, VMS DELTA: 98304
        End of Episode RSS: 356876288, Max RSS this Episode: 356876288, RSS DELTA: 159744
Finished episode 19, total return: 89.0, total time: 1.7 sec
        Endof Episode VMS: 288870400, Max VMS this Episode: 288870400, VMS DELTA: 0
        End of Episode RSS: 357072896, Max RSS this Episode: 357072896, RSS DELTA: 196608
Finished episode 20, total return: 31.0, total time: 0.5 sec
        Endof Episode VMS: 289021952, Max VMS this Episode: 289021952, VMS DELTA: 151552
        End of Episode RSS: 357191680, Max RSS this Episode: 357191680, RSS DELTA: 118784
Finished episode 21, total return: 106.0, total time: 2.4 sec
        Endof Episode VMS: 289173504, Max VMS this Episode: 289173504, VMS DELTA: 151552
        End of Episode RSS: 357707776, Max RSS this Episode: 357707776, RSS DELTA: 516096
Finished episode 22, total return: 76.0, total time: 1.6 sec
        Endof Episode VMS: 290226176, Max VMS this Episode: 290226176, VMS DELTA: 1052672
        End of Episode RSS: 358707200, Max RSS this Episode: 358707200, RSS DELTA: 999424
Finished episode 23, total return: 68.0, total time: 2.7 sec
        Endof Episode VMS: 290308096, Max VMS this Episode: 290308096, VMS DELTA: 81920
        End of Episode RSS: 359256064, Max RSS this Episode: 359256064, RSS DELTA: 548864
Finished episode 24, total return: 166.0, total time: 8.6 sec
        Endof Episode VMS: 291430400, Max VMS this Episode: 291540992, VMS DELTA: 1122304
        End of Episode RSS: 360054784, Max RSS this Episode: 360054784, RSS DELTA: 798720
Finished episode 25, total return: 136.0, total time: 6.3 sec
        Endof Episode VMS: 293986304, Max VMS this Episode: 293986304, VMS DELTA: 2555904
        End of Episode RSS: 362360832, Max RSS this Episode: 362356736, RSS DELTA: 2306048
Finished episode 26, total return: 137.0, total time: 7.1 sec
        Endof Episode VMS: 296742912, Max VMS this Episode: 296742912, VMS DELTA: 2756608
        End of Episode RSS: 364916736, Max RSS this Episode: 364916736, RSS DELTA: 2555904
Finished episode 27, total return: 99.0, total time: 2.3 sec
        Endof Episode VMS: 298270720, Max VMS this Episode: 298270720, VMS DELTA: 1527808
        End of Episode RSS: 366764032, Max RSS this Episode: 366764032, RSS DELTA: 1847296
Finished episode 28, total return: 71.0, total time: 2.1 sec
        Endof Episode VMS: 298569728, Max VMS this Episode: 298569728, VMS DELTA: 299008
        End of Episode RSS: 367349760, Max RSS this Episode: 367349760, RSS DELTA: 585728
Finished episode 29, total return: 88.0, total time: 2.5 sec
        Endof Episode VMS: 298594304, Max VMS this Episode: 298594304, VMS DELTA: 24576
        End of Episode RSS: 367624192, Max RSS this Episode: 367624192, RSS DELTA: 274432
Finished episode 30, total return: 54.0, total time: 1.0 sec
        Endof Episode VMS: 299786240, Max VMS this Episode: 299786240, VMS DELTA: 1191936
        End of Episode RSS: 368300032, Max RSS this Episode: 368300032, RSS DELTA: 675840
Finished episode 31, total return: 73.0, total time: 1.9 sec
        Endof Episode VMS: 301084672, Max VMS this Episode: 301084672, VMS DELTA: 1298432
        End of Episode RSS: 369131520, Max RSS this Episode: 369131520, RSS DELTA: 831488
Finished episode 32, total return: 95.0, total time: 2.4 sec
        Endof Episode VMS: 301350912, Max VMS this Episode: 301350912, VMS DELTA: 266240
        End of Episode RSS: 370094080, Max RSS this Episode: 370094080, RSS DELTA: 962560
Finished episode 33, total return: 75.0, total time: 2.2 sec
        Endof Episode VMS: 301621248, Max VMS this Episode: 301621248, VMS DELTA: 270336
        End of Episode RSS: 370434048, Max RSS this Episode: 370434048, RSS DELTA: 339968
Finished episode 34, total return: 47.0, total time: 1.5 sec
        Endof Episode VMS: 302829568, Max VMS this Episode: 302829568, VMS DELTA: 1208320
        End of Episode RSS: 370950144, Max RSS this Episode: 370950144, RSS DELTA: 516096
Finished episode 35, total return: 101.0, total time: 2.4 sec
        Endof Episode VMS: 302915584, Max VMS this Episode: 302915584, VMS DELTA: 86016
        End of Episode RSS: 371388416, Max RSS this Episode: 371388416, RSS DELTA: 438272
Finished episode 36, total return: 103.0, total time: 2.7 sec
        Endof Episode VMS: 302940160, Max VMS this Episode: 302940160, VMS DELTA: 24576
        End of Episode RSS: 371511296, Max RSS this Episode: 371511296, RSS DELTA: 122880
Finished episode 37, total return: 29.0, total time: 0.9 sec
        Endof Episode VMS: 303161344, Max VMS this Episode: 303161344, VMS DELTA: 221184
        End of Episode RSS: 372043776, Max RSS this Episode: 372043776, RSS DELTA: 532480
Finished episode 38, total return: 100.0, total time: 2.4 sec
        Endof Episode VMS: 305557504, Max VMS this Episode: 305557504, VMS DELTA: 2396160
        End of Episode RSS: 373796864, Max RSS this Episode: 373796864, RSS DELTA: 1753088
Finished episode 39, total return: 101.0, total time: 2.7 sec
        Endof Episode VMS: 305897472, Max VMS this Episode: 305897472, VMS DELTA: 339968
        End of Episode RSS: 374272000, Max RSS this Episode: 374272000, RSS DELTA: 475136
Finished episode 40, total return: 97.0, total time: 2.4 sec
        Endof Episode VMS: 307122176, Max VMS this Episode: 307122176, VMS DELTA: 1224704
        End of Episode RSS: 375627776, Max RSS this Episode: 375627776, RSS DELTA: 1355776
Finished episode 41, total return: 100.0, total time: 2.5 sec
        Endof Episode VMS: 308465664, Max VMS this Episode: 308465664, VMS DELTA: 1343488
        End of Episode RSS: 376766464, Max RSS this Episode: 376766464, RSS DELTA: 1138688
Finished episode 42, total return: 101.0, total time: 2.6 sec
        Endof Episode VMS: 309764096, Max VMS this Episode: 309764096, VMS DELTA: 1298432
        End of Episode RSS: 377999360, Max RSS this Episode: 377999360, RSS DELTA: 1232896
Finished episode 43, total return: 104.0, total time: 2.6 sec
        Endof Episode VMS: 309764096, Max VMS this Episode: 309764096, VMS DELTA: 0
        End of Episode RSS: 377999360, Max RSS this Episode: 377999360, RSS DELTA: 0
Finished episode 44, total return: 100.0, total time: 2.2 sec
        Endof Episode VMS: 309764096, Max VMS this Episode: 309764096, VMS DELTA: 0
        End of Episode RSS: 377999360, Max RSS this Episode: 377999360, RSS DELTA: 0
Finished episode 45, total return: 103.0, total time: 2.7 sec
        Endof Episode VMS: 309764096, Max VMS this Episode: 309764096, VMS DELTA: 0
        End of Episode RSS: 377999360, Max RSS this Episode: 377999360, RSS DELTA: 0
Finished episode 46, total return: 106.0, total time: 3.1 sec
        Endof Episode VMS: 309764096, Max VMS this Episode: 309764096, VMS DELTA: 0
        End of Episode RSS: 377999360, Max RSS this Episode: 377999360, RSS DELTA: 0
Finished episode 47, total return: 105.0, total time: 2.9 sec
        Endof Episode VMS: 309764096, Max VMS this Episode: 309764096, VMS DELTA: 0
        End of Episode RSS: 377991168, Max RSS this Episode: 377999360, RSS DELTA: -8192
Finished episode 48, total return: 39.0, total time: 1.2 sec
        Endof Episode VMS: 309764096, Max VMS this Episode: 309764096, VMS DELTA: 0
        End of Episode RSS: 377991168, Max RSS this Episode: 377991168, RSS DELTA: 0
Finished episode 49, total return: 68.0, total time: 2.0 sec
        Endof Episode VMS: 309764096, Max VMS this Episode: 309764096, VMS DELTA: 0
        End of Episode RSS: 377991168, Max RSS this Episode: 377991168, RSS DELTA: 0
Finished episode 50, total return: 36.0, total time: 0.9 sec
        Endof Episode VMS: 309764096, Max VMS this Episode: 309764096, VMS DELTA: 0
        End of Episode RSS: 377991168, Max RSS this Episode: 377991168, RSS DELTA: 0
Finished episode 51, total return: 101.0, total time: 2.4 sec
        Endof Episode VMS: 309764096, Max VMS this Episode: 309764096, VMS DELTA: 0
        End of Episode RSS: 377991168, Max RSS this Episode: 377991168, RSS DELTA: 0
Finished episode 52, total return: 102.0, total time: 2.2 sec
        Endof Episode VMS: 309764096, Max VMS this Episode: 309764096, VMS DELTA: 0
        End of Episode RSS: 377991168, Max RSS this Episode: 377991168, RSS DELTA: 0
Finished episode 53, total return: 104.0, total time: 2.0 sec
        Endof Episode VMS: 309764096, Max VMS this Episode: 309764096, VMS DELTA: 0
        End of Episode RSS: 377991168, Max RSS this Episode: 377991168, RSS DELTA: 0
Finished episode 54, total return: 122.0, total time: 2.5 sec
        Endof Episode VMS: 309764096, Max VMS this Episode: 309764096, VMS DELTA: 0
        End of Episode RSS: 377991168, Max RSS this Episode: 377991168, RSS DELTA: 0
Finished episode 55, total return: 120.0, total time: 2.7 sec
        Endof Episode VMS: 309764096, Max VMS this Episode: 309764096, VMS DELTA: 0
        End of Episode RSS: 377995264, Max RSS this Episode: 377995264, RSS DELTA: 4096
Finished episode 56, total return: 113.0, total time: 2.2 sec
        Endof Episode VMS: 309764096, Max VMS this Episode: 309764096, VMS DELTA: 0
        End of Episode RSS: 377999360, Max RSS this Episode: 378003456, RSS DELTA: 4096
Finished episode 57, total return: 147.0, total time: 3.7 sec
        Endof Episode VMS: 309764096, Max VMS this Episode: 309764096, VMS DELTA: 0
        End of Episode RSS: 377999360, Max RSS this Episode: 377999360, RSS DELTA: 0
Finished episode 58, total return: 117.0, total time: 3.1 sec
        Endof Episode VMS: 309764096, Max VMS this Episode: 309764096, VMS DELTA: 0
        End of Episode RSS: 377999360, Max RSS this Episode: 377999360, RSS DELTA: 0
Finished episode 59, total return: 200.0, total time: 10.3 sec
        Endof Episode VMS: 309764096, Max VMS this Episode: 309764096, VMS DELTA: 0
        End of Episode RSS: 377991168, Max RSS this Episode: 377999360, RSS DELTA: -8192
Finished episode 60, total return: 108.0, total time: 1.8 sec
        Endof Episode VMS: 309764096, Max VMS this Episode: 309764096, VMS DELTA: 0
        End of Episode RSS: 377995264, Max RSS this Episode: 377995264, RSS DELTA: 4096
Finished episode 61, total return: 128.0, total time: 3.2 sec
        Endof Episode VMS: 309764096, Max VMS this Episode: 309764096, VMS DELTA: 0
        End of Episode RSS: 377999360, Max RSS this Episode: 377999360, RSS DELTA: 4096
Finished episode 62, total return: 175.0, total time: 7.1 sec
        Endof Episode VMS: 309764096, Max VMS this Episode: 309764096, VMS DELTA: 0
        End of Episode RSS: 378040320, Max RSS this Episode: 378048512, RSS DELTA: 40960
Finished episode 63, total return: 182.0, total time: 9.2 sec
        Endof Episode VMS: 309800960, Max VMS this Episode: 309800960, VMS DELTA: 36864
        End of Episode RSS: 378077184, Max RSS this Episode: 378077184, RSS DELTA: 36864
Finished episode 64, total return: 194.0, total time: 15.3 sec
        Endof Episode VMS: 310415360, Max VMS this Episode: 310415360, VMS DELTA: 614400
        End of Episode RSS: 379363328, Max RSS this Episode: 379363328, RSS DELTA: 1286144
Finished episode 65, total return: 200.0, total time: 65.6 sec
        Endof Episode VMS: 310415360, Max VMS this Episode: 310415360, VMS DELTA: 0
        End of Episode RSS: 379400192, Max RSS this Episode: 379400192, RSS DELTA: 36864
Finished episode 66, total return: 200.0, total time: 41.4 sec
        Endof Episode VMS: 310415360, Max VMS this Episode: 310415360, VMS DELTA: 0
        End of Episode RSS: 379445248, Max RSS this Episode: 379445248, RSS DELTA: 45056
Finished episode 67, total return: 200.0, total time: 14.5 sec
        Endof Episode VMS: 310415360, Max VMS this Episode: 310415360, VMS DELTA: 0
        End of Episode RSS: 379424768, Max RSS this Episode: 379445248, RSS DELTA: -20480
Finished episode 68, total return: 200.0, total time: 89.0 sec
        Endof Episode VMS: 310415360, Max VMS this Episode: 310415360, VMS DELTA: 0
        End of Episode RSS: 379412480, Max RSS this Episode: 379428864, RSS DELTA: -12288
Finished episode 69, total return: 169.0, total time: 4.1 sec
        Endof Episode VMS: 310415360, Max VMS this Episode: 310415360, VMS DELTA: 0
        End of Episode RSS: 379408384, Max RSS this Episode: 379416576, RSS DELTA: -4096
Finished episode 70, total return: 143.0, total time: 3.6 sec
        Endof Episode VMS: 310722560, Max VMS this Episode: 310722560, VMS DELTA: 307200
        End of Episode RSS: 379699200, Max RSS this Episode: 379699200, RSS DELTA: 290816
Finished episode 71, total return: 200.0, total time: 57.6 sec
        Endof Episode VMS: 311418880, Max VMS this Episode: 311418880, VMS DELTA: 696320
        End of Episode RSS: 380346368, Max RSS this Episode: 380346368, RSS DELTA: 647168
Finished episode 72, total return: 200.0, total time: 27.4 sec
        Endof Episode VMS: 312631296, Max VMS this Episode: 312631296, VMS DELTA: 1212416
        End of Episode RSS: 380768256, Max RSS this Episode: 380768256, RSS DELTA: 421888
Finished episode 73, total return: 200.0, total time: 27.6 sec
        Endof Episode VMS: 312631296, Max VMS this Episode: 312631296, VMS DELTA: 0
        End of Episode RSS: 380768256, Max RSS this Episode: 380768256, RSS DELTA: 0
Finished episode 74, total return: 200.0, total time: 54.3 sec
        Endof Episode VMS: 312631296, Max VMS this Episode: 312631296, VMS DELTA: 0
        End of Episode RSS: 377118720, Max RSS this Episode: 380768256, RSS DELTA: -3649536
Finished episode 75, total return: 200.0, total time: 30.2 sec
       
[I CUT A BIT TO LONG]

        Endof Episode VMS: 312631296, Max VMS this Episode: 312631296, VMS DELTA: 0
        End of Episode RSS: 377147392, Max RSS this Episode: 377147392, RSS DELTA: 0
Finished episode 315, total return: 200.0, total time: 3.7 sec
        Endof Episode VMS: 312631296, Max VMS this Episode: 312631296, VMS DELTA: 0
        End of Episode RSS: 377147392, Max RSS this Episode: 377147392, RSS DELTA: 0
Finished episode 316, total return: 200.0, total time: 37.1 sec
        Endof Episode VMS: 312631296, Max VMS this Episode: 312631296, VMS DELTA: 0
        End of Episode RSS: 377147392, Max RSS this Episode: 377147392, RSS DELTA: 0
Finished episode 317, total return: 200.0, total time: 3.3 sec
        Endof Episode VMS: 312631296, Max VMS this Episode: 312631296, VMS DELTA: 0
        End of Episode RSS: 377147392, Max RSS this Episode: 377147392, RSS DELTA: 0
Finished episode 318, total return: 200.0, total time: 7.6 sec
        Endof Episode VMS: 312631296, Max VMS this Episode: 312631296, VMS DELTA: 0
        End of Episode RSS: 377151488, Max RSS this Episode: 377155584, RSS DELTA: 4096
Finished episode 319, total return: 200.0, total time: 4.3 sec
        Endof Episode VMS: 312631296, Max VMS this Episode: 312631296, VMS DELTA: 0
        End of Episode RSS: 377155584, Max RSS this Episode: 377159680, RSS DELTA: 4096
Finished episode 320, total return: 200.0, total time: 50.0 sec
        Endof Episode VMS: 312631296, Max VMS this Episode: 312631296, VMS DELTA: 0
        End of Episode RSS: 377954304, Max RSS this Episode: 377950208, RSS DELTA: 798720
Finished episode 321, total return: 200.0, total time: 4.4 sec
        Endof Episode VMS: 312631296, Max VMS this Episode: 312631296, VMS DELTA: 0
        End of Episode RSS: 377954304, Max RSS this Episode: 377954304, RSS DELTA: 0
Finished episode 322, total return: 200.0, total time: 27.1 sec
        Endof Episode VMS: 312631296, Max VMS this Episode: 312631296, VMS DELTA: 0
        End of Episode RSS: 377954304, Max RSS this Episode: 377954304, RSS DELTA: 0
Finished episode 323, total return: 200.0, total time: 3.6 sec
        Endof Episode VMS: 312631296, Max VMS this Episode: 312631296, VMS DELTA: 0
        End of Episode RSS: 377950208, Max RSS this Episode: 377954304, RSS DELTA: -4096
Finished episode 324, total return: 200.0, total time: 58.1 sec
        Endof Episode VMS: 312631296, Max VMS this Episode: 312631296, VMS DELTA: 0
        End of Episode RSS: 377929728, Max RSS this Episode: 377950208, RSS DELTA: -20480
Finished episode 325, total return: 200.0, total time: 5.0 sec
        Endof Episode VMS: 312631296, Max VMS this Episode: 312631296, VMS DELTA: 0
        End of Episode RSS: 377929728, Max RSS this Episode: 377937920, RSS DELTA: 0
Finished episode 326, total return: 200.0, total time: 8.2 sec
        Endof Episode VMS: 312631296, Max VMS this Episode: 312631296, VMS DELTA: 0
        End of Episode RSS: 377925632, Max RSS this Episode: 377933824, RSS DELTA: -4096
Finished episode 327, total return: 200.0, total time: 23.6 sec
        Endof Episode VMS: 312721408, Max VMS this Episode: 312721408, VMS DELTA: 90112
        End of Episode RSS: 378048512, Max RSS this Episode: 378048512, RSS DELTA: 122880
Finished episode 328, total return: 200.0, total time: 3.8 sec
        Endof Episode VMS: 312721408, Max VMS this Episode: 312721408, VMS DELTA: 0
        End of Episode RSS: 378068992, Max RSS this Episode: 378068992, RSS DELTA: 20480
Finished episode 329, total return: 200.0, total time: 18.8 sec
        Endof Episode VMS: 312721408, Max VMS this Episode: 312721408, VMS DELTA: 0
        End of Episode RSS: 378068992, Max RSS this Episode: 378068992, RSS DELTA: 0
Finished episode 330, total return: 200.0, total time: 5.2 sec
        Endof Episode VMS: 312721408, Max VMS this Episode: 312721408, VMS DELTA: 0
        End of Episode RSS: 378068992, Max RSS this Episode: 378068992, RSS DELTA: 0
Finished episode 331, total return: 200.0, total time: 49.3 sec
        Endof Episode VMS: 312721408, Max VMS this Episode: 312721408, VMS DELTA: 0
        End of Episode RSS: 378060800, Max RSS this Episode: 378068992, RSS DELTA: -8192
Finished episode 332, total return: 200.0, total time: 7.2 sec
        Endof Episode VMS: 312721408, Max VMS this Episode: 312721408, VMS DELTA: 0
        End of Episode RSS: 378048512, Max RSS this Episode: 378064896, RSS DELTA: -12288
Finished episode 333, total return: 200.0, total time: 16.4 sec
        Endof Episode VMS: 312721408, Max VMS this Episode: 312721408, VMS DELTA: 0
        End of Episode RSS: 378036224, Max RSS this Episode: 378052608, RSS DELTA: -12288
Finished episode 334, total return: 200.0, total time: 5.6 sec
        Endof Episode VMS: 312721408, Max VMS this Episode: 312721408, VMS DELTA: 0
        End of Episode RSS: 378028032, Max RSS this Episode: 378036224, RSS DELTA: -8192
Finished episode 335, total return: 200.0, total time: 6.1 sec
        Endof Episode VMS: 312721408, Max VMS this Episode: 312721408, VMS DELTA: 0
        End of Episode RSS: 378028032, Max RSS this Episode: 378028032, RSS DELTA: 0
Finished episode 336, total return: 200.0, total time: 9.2 sec
        Endof Episode VMS: 312721408, Max VMS this Episode: 312721408, VMS DELTA: 0
        End of Episode RSS: 378032128, Max RSS this Episode: 378032128, RSS DELTA: 4096
Finished episode 337, total return: 200.0, total time: 16.0 sec
        Endof Episode VMS: 312721408, Max VMS this Episode: 312721408, VMS DELTA: 0
        End of Episode RSS: 378032128, Max RSS this Episode: 378032128, RSS DELTA: 0
Finished episode 338, total return: 200.0, total time: 11.3 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 32768
        End of Episode RSS: 378044416, Max RSS this Episode: 378052608, RSS DELTA: 12288
Finished episode 339, total return: 200.0, total time: 6.4 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378036224, Max RSS this Episode: 378048512, RSS DELTA: -8192
Finished episode 340, total return: 200.0, total time: 9.4 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378032128, Max RSS this Episode: 378048512, RSS DELTA: -4096
Finished episode 341, total return: 200.0, total time: 4.2 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378023936, Max RSS this Episode: 378040320, RSS DELTA: -8192
Finished episode 342, total return: 200.0, total time: 28.6 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378036224, Max RSS this Episode: 378036224, RSS DELTA: 12288
Finished episode 343, total return: 200.0, total time: 6.1 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378036224, Max RSS this Episode: 378036224, RSS DELTA: 0
Finished episode 344, total return: 200.0, total time: 43.7 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378036224, Max RSS this Episode: 378036224, RSS DELTA: 0
Finished episode 345, total return: 200.0, total time: 13.5 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378036224, Max RSS this Episode: 378036224, RSS DELTA: 0
Finished episode 346, total return: 200.0, total time: 14.7 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378044416, Max RSS this Episode: 378044416, RSS DELTA: 8192
Finished episode 347, total return: 200.0, total time: 21.2 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378048512, Max RSS this Episode: 378048512, RSS DELTA: 4096
Finished episode 348, total return: 200.0, total time: 25.6 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378040320, Max RSS this Episode: 378052608, RSS DELTA: -8192
Finished episode 349, total return: 200.0, total time: 22.8 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378040320, Max RSS this Episode: 378040320, RSS DELTA: 0
Finished episode 350, total return: 200.0, total time: 3.9 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378040320, Max RSS this Episode: 378040320, RSS DELTA: 0
Finished episode 351, total return: 200.0, total time: 12.7 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378056704, Max RSS this Episode: 378056704, RSS DELTA: 16384
Finished episode 352, total return: 200.0, total time: 21.6 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378064896, Max RSS this Episode: 378064896, RSS DELTA: 8192
Finished episode 353, total return: 200.0, total time: 19.2 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378064896, Max RSS this Episode: 378064896, RSS DELTA: 0
Finished episode 354, total return: 200.0, total time: 56.6 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378032128, Max RSS this Episode: 378081280, RSS DELTA: -32768
Finished episode 355, total return: 200.0, total time: 8.6 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378028032, Max RSS this Episode: 378044416, RSS DELTA: -4096
Finished episode 356, total return: 200.0, total time: 3.7 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378032128, Max RSS this Episode: 378032128, RSS DELTA: 4096
Finished episode 357, total return: 158.0, total time: 2.2 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378032128, Max RSS this Episode: 378032128, RSS DELTA: 0
Finished episode 358, total return: 140.0, total time: 2.6 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378032128, Max RSS this Episode: 378032128, RSS DELTA: 0
Finished episode 359, total return: 200.0, total time: 6.2 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378032128, Max RSS this Episode: 378032128, RSS DELTA: 0
Finished episode 360, total return: 200.0, total time: 28.5 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378015744, Max RSS this Episode: 378036224, RSS DELTA: -16384
Finished episode 361, total return: 190.0, total time: 6.0 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378023936, Max RSS this Episode: 378028032, RSS DELTA: 8192
Finished episode 362, total return: 200.0, total time: 46.5 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 377917440, Max RSS this Episode: 378036224, RSS DELTA: -106496
Finished episode 363, total return: 200.0, total time: 28.6 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 377958400, Max RSS this Episode: 377958400, RSS DELTA: 40960
Finished episode 364, total return: 200.0, total time: 42.3 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 377982976, Max RSS this Episode: 377982976, RSS DELTA: 24576
Finished episode 365, total return: 200.0, total time: 24.7 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 377987072, Max RSS this Episode: 377982976, RSS DELTA: 4096
Finished episode 366, total return: 200.0, total time: 23.3 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 377999360, Max RSS this Episode: 377995264, RSS DELTA: 12288
Finished episode 367, total return: 200.0, total time: 3.8 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378015744, Max RSS this Episode: 378015744, RSS DELTA: 16384
Finished episode 368, total return: 79.0, total time: 1.5 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378019840, Max RSS this Episode: 378028032, RSS DELTA: 4096
Finished episode 369, total return: 200.0, total time: 4.6 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378019840, Max RSS this Episode: 378028032, RSS DELTA: 0
Finished episode 370, total return: 157.0, total time: 5.1 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378023936, Max RSS this Episode: 378032128, RSS DELTA: 4096
Finished episode 371, total return: 200.0, total time: 38.0 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378028032, Max RSS this Episode: 378032128, RSS DELTA: 4096
Finished episode 372, total return: 200.0, total time: 6.1 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378036224, Max RSS this Episode: 378036224, RSS DELTA: 8192
Finished episode 373, total return: 180.0, total time: 5.2 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378040320, Max RSS this Episode: 378040320, RSS DELTA: 4096
Finished episode 374, total return: 134.0, total time: 2.6 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378048512, Max RSS this Episode: 378048512, RSS DELTA: 8192
Finished episode 375, total return: 200.0, total time: 22.2 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378036224, Max RSS this Episode: 378048512, RSS DELTA: -12288
Finished episode 376, total return: 200.0, total time: 16.6 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378032128, Max RSS this Episode: 378036224, RSS DELTA: -4096
Finished episode 377, total return: 200.0, total time: 19.4 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378028032, Max RSS this Episode: 378040320, RSS DELTA: -4096
Finished episode 378, total return: 102.0, total time: 1.7 sec
        Endof Episode VMS: 312754176, Max VMS this Episode: 312754176, VMS DELTA: 0
        End of Episode RSS: 378028032, Max RSS this Episode: 378036224, RSS DELTA: 0
Finished episode 379, total return: 200.0, total time: 18.6 sec
        Endof Episode VMS: 313016320, Max VMS this Episode: 313016320, VMS DELTA: 262144
        End of Episode RSS: 378253312, Max RSS this Episode: 378257408, RSS DELTA: 225280
Finished episode 380, total return: 200.0, total time: 19.3 sec
        Endof Episode VMS: 313016320, Max VMS this Episode: 313016320, VMS DELTA: 0
        End of Episode RSS: 378257408, Max RSS this Episode: 378257408, RSS DELTA: 4096
Finished episode 381, total return: 60.0, total time: 1.2 sec
        Endof Episode VMS: 313016320, Max VMS this Episode: 313016320, VMS DELTA: 0
        End of Episode RSS: 378257408, Max RSS this Episode: 378257408, RSS DELTA: 0
Finished episode 382, total return: 200.0, total time: 37.8 sec
        Endof Episode VMS: 313016320, Max VMS this Episode: 313016320, VMS DELTA: 0
        End of Episode RSS: 378257408, Max RSS this Episode: 378257408, RSS DELTA: 0
Finished episode 383, total return: 90.0, total time: 1.7 sec
        Endof Episode VMS: 313016320, Max VMS this Episode: 313016320, VMS DELTA: 0
        End of Episode RSS: 378257408, Max RSS this Episode: 378257408, RSS DELTA: 0
Finished episode 384, total return: 200.0, total time: 18.5 sec
        Endof Episode VMS: 313016320, Max VMS this Episode: 313016320, VMS DELTA: 0
        End of Episode RSS: 378257408, Max RSS this Episode: 378257408, RSS DELTA: 0
Finished episode 385, total return: 43.0, total time: 0.6 sec
        Endof Episode VMS: 313016320, Max VMS this Episode: 313016320, VMS DELTA: 0
        End of Episode RSS: 378257408, Max RSS this Episode: 378257408, RSS DELTA: 0
Finished episode 386, total return: 41.0, total time: 0.7 sec
        Endof Episode VMS: 313016320, Max VMS this Episode: 313016320, VMS DELTA: 0
        End of Episode RSS: 378257408, Max RSS this Episode: 378257408, RSS DELTA: 0
Finished episode 387, total return: 34.0, total time: 0.5 sec
        Endof Episode VMS: 313016320, Max VMS this Episode: 313016320, VMS DELTA: 0
        End of Episode RSS: 378257408, Max RSS this Episode: 378257408, RSS DELTA: 0
Finished episode 388, total return: 59.0, total time: 0.9 sec
        Endof Episode VMS: 313016320, Max VMS this Episode: 313016320, VMS DELTA: 0
        End of Episode RSS: 378257408, Max RSS this Episode: 378257408, RSS DELTA: 0
Finished episode 389, total return: 38.0, total time: 0.5 sec
        Endof Episode VMS: 313016320, Max VMS this Episode: 313016320, VMS DELTA: 0
        End of Episode RSS: 378257408, Max RSS this Episode: 378257408, RSS DELTA: 0
Finished episode 390, total return: 30.0, total time: 0.5 sec
        Endof Episode VMS: 313016320, Max VMS this Episode: 313016320, VMS DELTA: 0
        End of Episode RSS: 378257408, Max RSS this Episode: 378257408, RSS DELTA: 0
Finished episode 391, total return: 46.0, total time: 0.7 sec
        Endof Episode VMS: 313016320, Max VMS this Episode: 313016320, VMS DELTA: 0
        End of Episode RSS: 378253312, Max RSS this Episode: 378257408, RSS DELTA: -4096
Finished episode 392, total return: 32.0, total time: 0.4 sec
        Endof Episode VMS: 313016320, Max VMS this Episode: 313016320, VMS DELTA: 0
        End of Episode RSS: 378257408, Max RSS this Episode: 378257408, RSS DELTA: 4096
Finished episode 393, total return: 26.0, total time: 0.4 sec
        Endof Episode VMS: 313016320, Max VMS this Episode: 313016320, VMS DELTA: 0
        End of Episode RSS: 378261504, Max RSS this Episode: 378257408, RSS DELTA: 4096
Finished episode 394, total return: 42.0, total time: 0.7 sec
        Endof Episode VMS: 313016320, Max VMS this Episode: 313016320, VMS DELTA: 0
        End of Episode RSS: 378261504, Max RSS this Episode: 378261504, RSS DELTA: 0
Finished episode 395, total return: 89.0, total time: 1.5 sec
        Endof Episode VMS: 313016320, Max VMS this Episode: 313016320, VMS DELTA: 0
        End of Episode RSS: 378261504, Max RSS this Episode: 378261504, RSS DELTA: 0
Finished episode 396, total return: 47.0, total time: 0.7 sec
        Endof Episode VMS: 313016320, Max VMS this Episode: 313016320, VMS DELTA: 0
        End of Episode RSS: 378261504, Max RSS this Episode: 378261504, RSS DELTA: 0
Finished episode 397, total return: 83.0, total time: 1.1 sec
        Endof Episode VMS: 313016320, Max VMS this Episode: 313016320, VMS DELTA: 0
        End of Episode RSS: 378261504, Max RSS this Episode: 378261504, RSS DELTA: 0
Finished episode 398, total return: 200.0, total time: 5.3 sec
        Endof Episode VMS: 313016320, Max VMS this Episode: 313016320, VMS DELTA: 0
        End of Episode RSS: 378245120, Max RSS this Episode: 378261504, RSS DELTA: -16384
Finished episode 399, total return: 200.0, total time: 29.9 sec
        Endof Episode VMS: 313016320, Max VMS this Episode: 313016320, VMS DELTA: 0
        End of Episode RSS: 378232832, Max RSS this Episode: 378249216, RSS DELTA: -12288

[fede72bari]

Hi Thomas, hope everything is fine. Did the last attempt to track memory usage growth could give some suggestions on the trouble fixing according to you? Which is the time performance you can see in your machine environment running the algorithm for a higher number of max steps and episodes? Thank you.