similarity search and connect processes take a lot of time (too much?) #47
Description
The bulk time of simulation is taken by the process that connects similar items with 'similarity' links and weights. The decentralized similarity search even crashes, so the centralized similarity search is currently used (temporarily).
2018-07-16 15:39:52,550 [...] OfferNet.class : method=existsSimilarity : simulationId=SIM07-16-01-53-WuzeNN : [...] similarity_value=-1 : **wallTime_ms=120 msec**.
2018-07-16 15:39:52,554 [...] OfferNet.class : method=connect : simulationId=SIM07-16-01-53-WuzeNN :[...] ; similarity=0.6324555319268611 ; similarityEdge=[...] : wallTime_ms=4 msec.
2018-07-16 15:39:52,555 [...] OfferNet.class : method=connectIfSimilar : simulationId=SIM07-16-01-53-WuzeNN : [...] ; similarityThreshold=0.5 ; added_similarity_edge=[...] : **wallTime_ms=125 msec**.
120 ms method running time is not acceptable when having hundreds of thousands of them running in one simulation... This is not a technical issue, but most probably can be approached by changing gremlin queries as well as not running them on each agent in the graph, but it may be the topic of the next experiment (#17) on learning capabilities of the network. Nevertheless, something to look into for the current one too.