slow crawl/what can I do to improve #19
Unanswered
Marchewcia
asked this question in
Help & Support
Replies: 2 comments
-
|
What are your specs? also, if you have JS rendering enabled then this would be the biggest bottleneck. otherwise make sure to set crawl delay to 0, concurrency to 50. |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
unfortunately my current only computer available is i3 115G4, 8gb of ram and 512gb nvme, integrated graphics only |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hi, so I'm crawling a website right now and what would be most impactful variable(s) during the scan? I had in peak 0.5 url/s but it dropped down to just 0.34 url/s and it keeps dropping slowly? what could be a bottleneck there?
Beta Was this translation helpful? Give feedback.
All reactions