Allowed memory size exhausted

[cryptocoinserver]

The environment
PrestaShop-Version 1.6.1.15
Elasticsearch version: 5.6.2
Products: 1864
PHP-Version 7.1.3
php_memory_limit: 512M
max_execution_time: 20000

This happens, while trying to reindex (both all and missing):

[error] FastCGI sent in stderr: "PHP message: PHP Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 20480 bytes) in /var/www/clients/client1/web2/web/classes/db/DbPDO.php on line 154 PHP message: PHP Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 77824 bytes) in /usr/lib/tideways/Tideways.php on line 354" while reading response header from upstream, client: XXX, server: XXX, request: "POST /adminXXX/index.php?controller=AdminBradSetting&token=XXX HTTP/1.1", upstream: "fastcgi://unix:/var/lib/php5-fpm/web2.sock:", host: "XXX", referrer: "https://XXX/adminXXX/index.php?controller=AdminBradSetting&token=XXX"

• HTTP 500

I tried:

• Reducing bulk request size.
• Increased the refresh interval.
• Set shards and replicas to 1.

Replaced some information with XXX for privacy protection.

Thank you for your time and work!

[sarjon]

hi @mysecondworld, what is size of your bulk request? Do your products have a lot of features or attributes?

[cryptocoinserver]

hi @sarjon
I tried many different sizes. From 10 to the default. Yes they got many features for example flavours.

The shop i work for, decided to try this modul:
https://addons.prestashop.com/en/search-filters/19527-elasticsearchconnector.html
And it works very fast and without error. The settings are great, too.

Thank you for your time.

Brad and elasticsearchconnector does not work the same.

I bought the module elasticsaerch and I prefer brad it does not use sql and has filters 🥇

[veebipoed]

I try to get it to work for a shop with 12000+ active products, in general the purpose is to use elasticsearch for up to 100000 products. Changing batch size to 10 allowed me to index 408 products before I got also out-of-memory error - so in general I can try to run cron for every 10-15 minutes or write a wrapper to check if the script is running or not.

I haven't investigated how Brad works, but is it possible in general to make it work for ca 20th products? I also need the filtering option, this is why I'm trying out Brad.

It seems bottleneck is the amount of features there is in use in our case. But still some things can be improved from the memory point of view.

[nicdnepr]

Any news?
I have shop with 250k goods, run php cli brad cron and get out of memory, I removed this line https://github.com/Invertus/brad/blob/v2/brad.cron.php#L56
How is Bulk request size depends on allocated memory? I try to set Bulk request size much bigger to increase index speed, but not help

[nicdnepr]

PHP Fatal error: Allowed memory size of 6585057280 bytes exhausted (tried to allocate 20480 bytes) in /home/test/public_html/t/classes/db/DbPDO.php on line 154
how can it be?