HTTP & Scraping Python Libraries

Search every project in one place

Press / to search. Tap a tag to filter. Click any row for details.

Filtering for

Row number					Tags
1	urllib3 Clients	1,426,075,169	4,058	2026-09-15	Clients HTTP Clients HTTP & Scraping	→
	A HTTP library with thread-safe connection pooling, file post support, sanity friendly.
	urllib3/github.com/urllib3/urllib3 /2026-09-15 /1,426,075,169 downloads/month
2	requests Clients	1,338,007,483	54,302	2026-09-02	Clients HTTP Clients HTTP & Scraping	→
	HTTP Requests for Humans.
	psf/github.com/psf/requests /2026-09-02 /1,338,007,483 downloads/month
3	httpx Clients	654,841,826	15,481	2026-02-23	Clients HTTP Clients HTTP & Scraping	→
	A next generation HTTP client for Python.
	encode/github.com/encode/httpx /2026-02-23 /654,841,826 downloads/month
4	aiohttp Clients	493,564,496	16,547	2026-09-15	Clients HTTP Clients HTTP & Scraping	→
	Asynchronous HTTP client/server framework for asyncio and Python.
	aio-libs/github.com/aio-libs/aiohttp /2026-09-15 /493,564,496 downloads/month
5	yarl URL Manipulation	485,646,437	1,497	2026-09-15	URL Manipulation HTTP Clients HTTP & Scraping	→
	Yet another URL library.
	aio-libs/github.com/aio-libs/yarl /2026-09-15 /485,646,437 downloads/month
6	httpx2 Clients	200,189,571	1,441	2026-09-14	Clients HTTP Clients HTTP & Scraping	→
	HTTP/1.1 and HTTP/2 client with sync and async APIs, maintained by Pydantic (httpx fork).
	pydantic/github.com/pydantic/httpx2 /2026-09-14 /200,189,571 downloads/month
7	browser-use Frameworks	24,635,245	114,711	2026-09-13	Frameworks Web Scraping HTTP & Scraping	→
	Make websites accessible for AI agents with easy browser automation.
	browser-use/github.com/browser-use/browser-use /2026-09-13 /24,635,245 downloads/month
8	feedparser Content Extraction	20,889,732	2,422	2026-07-30	Content Extraction Web Scraping HTTP & Scraping	→
	Universal feed parser.
	kurtmckee/github.com/kurtmckee/feedparser /2026-07-30 /20,889,732 downloads/month
9	html2text Content Extraction	13,644,323	2,170	2025-10-28	Content Extraction Web Scraping HTTP & Scraping	→
	Convert HTML to Markdown-formatted text.
	Alir3z4/github.com/Alir3z4/html2text /2025-10-28 /13,644,323 downloads/month
10	trafilatura Content Extraction	12,280,956	6,821	2026-09-11	Content Extraction Web Scraping HTTP & Scraping	→
	A tool for gathering text and metadata from the web, with built-in content filtering.
	adbar/github.com/adbar/trafilatura /2026-09-11 /12,280,956 downloads/month
11	scrapy Frameworks	4,651,078	64,358	2026-09-15	Frameworks Web Scraping HTTP & Scraping	→
	A fast high-level screen scraping and web crawling framework.
	scrapy/github.com/scrapy/scrapy /2026-09-15 /4,651,078 downloads/month
12	crawl4ai Frameworks	1,329,723	83,551	2026-08-31	Frameworks Web Scraping HTTP & Scraping	→
	An open-source, LLM-friendly web crawler that provides lightning-fast, structured data extraction specifically designed for AI agents.
	unclecode/github.com/unclecode/crawl4ai /2026-08-31 /1,329,723 downloads/month
13	yagmail HTTP & Scraping	282,236	2,733	2026-05-26	Email HTTP & Scraping	→
	Yet another Gmail/SMTP client.
	kootenpv/github.com/kootenpv/yagmail /2026-05-26 /282,236 downloads/month

Contribute

Know a project that belongs here?

Tell us what it does and why it stands out.

Submit a project Star the repository

Search every project in one place

Search and filter

Results

Know a project that belongs here?