-
Notifications
You must be signed in to change notification settings - Fork 3
Open
Description
Hi again, after your last fix I thought I'd try it out and whilst I can crawl sites now (which is great) the selector= argument doesn't appear to work and errors out.
I thought I would test it out and see if I could scrape the links form Hackernews with this:
result = safe_crawl('https://news.ycombinator.com', selector='td:nth-child(3) > span:nth-child(1) > a:nth-child(1)', output_dir='./output')
Detected platform: darwin_arm64
Found platform-specific binary at /Users/alexander/Developer/projects/services/.venv/lib/python3.11/site-packages/pathik/bin/darwin_arm64/pathik_bin with correct version 0.3.11
Detected platform: darwin_arm64
Found platform-specific binary at /Users/alexander/Developer/projects/services/.venv/lib/python3.11/site-packages/pathik/bin/darwin_arm64/pathik_bin with correct version 0.3.11
Running command: /Users/alexander/Developer/projects/services/.venv/lib/python3.11/site-packages/pathik/bin/darwin_arm64/pathik_bin -outdir ./output -s td:nth-child(3) > span:nth-child(1) > a:nth-child(1) -crawl https://news.ycombinator.com
Running command: /Users/alexander/Developer/projects/services/.venv/lib/python3.11/site-packages/pathik/bin/darwin_arm64/pathik_bin -outdir ./output -s td:nth-child(3) > span:nth-child(1) > a:nth-child(1) -crawl https://news.ycombinator.com
Command stdout: ...
Command stderr: flag provided but not defined: -s
Usage of /Users/alexander/Developer/projects/services/.venv/lib/python3.11/site-packages/pathik/bin/darwin_arm64/pathik_bin:
-buffer-memory int
...Not sure what the issue is but hopefully it means something to you.
Metadata
Metadata
Assignees
Labels
No labels