Unmaintained, dead, list of alternative projects

[bobpaul]

Since Robobrowser seems to be unmaintained since 2015, here is a list of alternatives that I've found which all have had commits with the list year (ie, during the year 2021). I've not yet started the process of portin any of my scripts over from Robobrowser to any of these, so I can't vouch for how similar any are.

• Scrapy currently 43k stars on github
• Mechanical Soup currently 3.9k stars on github
• PSpider currently 1.3k stars on github
• Feedparser currently 1.3k stars on github
• Feedparser is specifically designed for RSS, ATOM, and other feeds, rather than a generic scraper
• Spidy currently 273 stars on github
  • Spidy is more of a commandline tool than a framework for automation, but there's probably a non-zero number of users who end up looking at Robobrowser but would be happier with a commandline tool.

I included the stars because that helps gauge how popular a project is. Popular projects tend be more less likely to disappear if a maintainer loses interest in the project. RoboBrowser is currently at 3.6k stars. But all of the above projects appear to be currently maintained.