Web Scraping

Libraries to automate web scraping and extract web content.

Search every project in one place

Press / to search. Tap a tag to filter. Click any row for details.

Filtering for

Row number					Tags
1	browser-use Frameworks	43,437,788	112,291	2026-09-04	Frameworks Web Scraping HTTP & Scraping	→
	Make websites accessible for AI agents with easy browser automation.
	browser-use/github.com/browser-use/browser-use /2026-09-04 /43,437,788 downloads/month
2	feedparser Content Extraction	23,405,630	2,418	2026-07-30	Content Extraction Web Scraping HTTP & Scraping	→
	Universal feed parser.
	kurtmckee/github.com/kurtmckee/feedparser /2026-07-30 /23,405,630 downloads/month
3	html2text Content Extraction	13,692,188	2,167	2025-10-28	Content Extraction Web Scraping HTTP & Scraping	→
	Convert HTML to Markdown-formatted text.
	Alir3z4/github.com/Alir3z4/html2text /2025-10-28 /13,692,188 downloads/month
4	trafilatura Content Extraction	12,737,401	6,765	2026-08-28	Content Extraction Web Scraping HTTP & Scraping	→
	A tool for gathering text and metadata from the web, with built-in content filtering.
	adbar/github.com/adbar/trafilatura /2026-08-28 /12,737,401 downloads/month
5	scrapy Frameworks	4,362,247	64,199	2026-09-03	Frameworks Web Scraping HTTP & Scraping	→
	A fast high-level screen scraping and web crawling framework.
	scrapy/github.com/scrapy/scrapy /2026-09-03 /4,362,247 downloads/month
6	crawl4ai Frameworks	1,432,245	81,299	2026-08-31	Frameworks Web Scraping HTTP & Scraping	→
	An open-source, LLM-friendly web crawler that provides lightning-fast, structured data extraction specifically designed for AI agents.
	unclecode/github.com/unclecode/crawl4ai /2026-08-31 /1,432,245 downloads/month

Contribute

Know a project that belongs here?

Tell us what it does and why it stands out.

Submit a project Star the repository