Web Scraping

Frameworks

Listed in editorial order. Click a column header to re-sort the whole list.

Press / to search. Tags open their category. Click a row for details.

Search and filter

Results

Row number Tags
Make websites accessible for AI agents with easy browser automation.
browser-use/github.com/browser-use/browser-use /119,495 stars /
A fast high-level web crawling and scraping framework.
scrapy/github.com/scrapy/scrapy /65,101 stars /
An open-source, LLM-friendly web crawler that provides lightning-fast, structured data extraction specifically designed for AI agents.
unclecode/github.com/unclecode/crawl4ai /86,544 stars /
A fast and token-efficient browser automation SDK to extract data and perform self-healing actions on web pages.
browserbase/github.com/browserbase/stagehand /25,860 stars /
An adaptive scraping framework whose selectors relocate elements after page changes, with stealth fetchers and spiders.
D4Vinci/github.com/D4Vinci/Scrapling /90,322 stars /

Know a project that belongs here?

Tell us what it does and why it stands out.