Open-Source Web Scraping Tools: The 42 Most-Starred GitHub Repos, by Category (2026)

The most-starred open-source web scraping tools on GitHub are Firecrawl (199,913 stars), browser-use (120,388), Scrapling (100,136), Playwright (99,642), and Puppeteer (95,816), as of November 9, 2026. Microsoft's MarkItDown (194,504) sits second by stars, but it converts documents to Markdown rather than fetching pages; we keep it in the list because so many AI scraping pipelines use it for that conversion step. We pulled live data from the GitHub API for 52 candidate repos, kept the 42 that met our rules, and sorted them into 8 categories, from AI-ready crawlers to the HTML parsers underneath them. For each one we list stars and license, and for the most-used libraries we compare GitHub stars with last month's PyPI and npm downloads.

The pattern in the data is hard to miss. Tools created in 2024 or later make up 17 of the 42 repos but hold 58% of the stars. Downloads tell a different story, and both numbers are below.

Key Takeaways

• Firecrawl is the most-starred web scraping repo on GitHub, at 199,913 stars. Six of the top ten repos were created in 2024 or later.
• AI-era tools dominate attention: the 17 repos created since January 2024 hold 958,003 of the 1,664,365 stars in this list (58%).
• Stars and downloads disagree. Scrapling has 100,136 stars to Scrapy's 65,286, but Scrapy had 4.30 million PyPI downloads last month against Scrapling's 0.92 million.
• Check the license before you ship. Six of the 42 use copyleft licenses (AGPL-3.0 or MPL-2.0), including Firecrawl and Skyvern.
• None of the self-hosted crawlers, browsers, or agents brings its own IP addresses. Their requests leave from the machine the tool runs on, which is why most of them expose a proxy setting.

How we ranked open-source web scraping tools on GitHub

We started from GitHub's search for the web-scraping topic and for crawler and scraper repos with more than 15,000 stars, then added the browser automation, parsing, and MCP projects that scrapers depend on. We screened out platform-specific tools, such as single-site downloaders and social media scrapers, which gave 52 general-purpose candidates. On November 9, 2026, our script called the GitHub REST API for each one and recorded stars, forks, license, language, creation date, and last push.

A candidate made the list if it met all four rules:

1. At least 5,000 stars.
2. An open-source license detected by GitHub.
3. A push in the last 12 months, so the project is still maintained.
4. Created at least 12 months ago, so the star count reflects more than a launch spike.

Download counts come from pypistats.org for Python packages and the npm registry API for JavaScript ones, both for the last 30 days. Stars measure attention, not usage, so we report both.

AI-ready crawlers and extraction

These tools fetch a page and hand back something a language model can read, usually Markdown or structured JSON. Nearly everything here is new: every repo except Maxun was created in 2024.

Crawl4AI lists Massive as a strategic partner for web access in its README, and we covered that partnership in our Crawl4AI spotlight. If you're choosing a format for model input, our post on how HTML-to-Markdown cuts agent token costs explains why Markdown is the default output here.

Crawling frameworks

Frameworks handle the parts of a crawl that aren't parsing: queues, retries, concurrency, deduplication, and export. Scrapy has done this in Python since 2010. Scrapling, created in October 2024, now has more stars.

Browser automation

When a page builds its content with JavaScript, you need a real browser. These libraries drive one. Playwright and Puppeteer are within 4,000 stars of each other, but on npm Playwright pulled 521 million downloads last month to Puppeteer's 50 million, most likely because test suites also install it in CI.

AI browser agents

Agents put a language model in charge of the browser: it reads the page, decides what to click, and repeats until the task is done. This is the newest category, and its leader, browser-use, has the third-highest star count in the list, behind Firecrawl and MarkItDown. We compared three of these frameworks in depth in browser-use vs Stagehand vs Skyvern.

Stealth and fingerprinting

Sites check more than your IP address. They read the TLS handshake, HTTP/2 settings, and dozens of browser properties to decide whether a client is a person or a script. These projects change what that fingerprint says. The most-starred repo in this space, CloakBrowser, and another stealth browser, camofox-browser, were both created in 2026, so they fail our 12-month age rule and are listed separately below.

A matching fingerprint still arrives from whatever IP address you run on. A browser that looks like Chrome on a home laptop but connects from a cloud data center is a common reason scrapers get blocked, which we covered in why AI agents get blocked on datacenter IPs.

Parsing and content extraction

Once you have the HTML, something has to pull out the parts you want. These libraries sit underneath most of the tools above. One gap in a star ranking: Beautiful Soup, probably the best-known Python parser, is developed on Launchpad rather than GitHub, so it has no star count to compare.

Monitoring and no-code

Not every scraping job is a one-time crawl. These tools watch pages over time or let non-developers build scrapers visually.

MCP servers for scraping

MCP servers expose browsing and scraping as tools that an AI agent can call. Each of the four that passed our rules is a year or more old, and the Playwright one is heavily used: @playwright/mcp had 29.3 million npm downloads last month.

Stars vs downloads: which tools people actually install

A star is a bookmark. A download is closer to a deployment, although download counts also include CI runs, mirrors, and installs pulled in as a dependency of another package. We matched the main Python crawling and agent libraries with their PyPI packages:

Scrapy's more than 15 years of production use show here. It has a third fewer stars than Scrapling and about 4.7 times the installs. Crawlee for Python has the fewest stars in the table and the highest ratio, which suggests a user base that installs more than it stars. The newer tools are growing from a smaller installed base, so the gap is a snapshot, not a verdict. Firecrawl is missing from the table because its PyPI package, firecrawl-py, is the client for the hosted API rather than the self-hosted crawler; it had 5.34 million downloads last month. For browser drivers the scale is different again: Selenium had 28.0 million PyPI downloads and Playwright 97.0 million on PyPI on top of its npm numbers.

New, fast-rising, and left out

Ten candidates missed our rules. Four of them are new repos that are already large, which is worth watching:

• Created in 2026, under 12 months old: CloakBrowser (32,789 stars, a stealth Chromium), Obscura (30,905, a headless browser for agents), moli (23,110, a Rust headless browser), and camofox-browser (12,211). They are too new for us to judge how well they will be maintained.
• No push in the last 12 months: GPT Crawler, undetected-chromedriver, and Postlight Parser.
• Under 5,000 stars: spider-rs, Apify's fingerprint-suite, and Scrapy's Parsel selector library.

We also left out platform-specific tools that would otherwise rank high, such as MediaCrawler (about 70,700 stars), which targets Chinese social platforms, and the video downloader you-get (about 56,900).

Licenses: what you can ship

Most of this list uses permissive licenses (MIT, Apache-2.0, BSD-3-Clause) that let you embed the code in a commercial product. Six repos use copyleft licenses: Firecrawl, Maxun, Skyvern, EasySpider, and Lightpanda are AGPL-3.0, and Camoufox is MPL-2.0. The AGPL's network clause can require you to publish your changes if you run a modified version as a service, which matters if you plan to self-host and modify. Several of these projects also sell a hosted version, which is a separate commercial agreement. Read the license file and ask your legal team before you build on a copyleft repo.

Where proxies fit in an open-source scraping stack

Every crawler, browser driver, and agent above fetches from the machine it runs on (the parsers only process HTML you already have). On a laptop that is a home IP address. On a cloud server it is a data center IP, and many sites rate-limit or block those ranges. The tools solve fetching, parsing, and orchestration, and the network address is left to you.

That's why most of them take a proxy setting. In Scrapy you set proxy in a request's meta, Playwright accepts a proxy option when it launches a browser, and Crawlee has a ProxyConfiguration class with rotation built in. Massive's residential proxies plug into any of these over HTTP, HTTPS, or SOCKS5, from real consumer devices in 195+ countries, with country, state, and city targeting. Every device in the network opted in through the Massive SDK. If you'd rather skip running a browser, the Web Render API renders the page and returns HTML or Markdown.

Picking from the list

If you're starting a new project, the open-source web scraping tools on GitHub sort into a short decision. For a large Python crawl, start with Scrapy or Scrapling. For Node.js, start with Crawlee. When pages need JavaScript, add Playwright. When the output feeds an LLM, Firecrawl and Crawl4AI return Markdown directly, and when an agent has to click through a site, browser-use and Stagehand lead. Whichever you choose, plan the network layer early: the tool decides how you fetch, and the IP address decides whether you get an answer.

Sources

• GitHub REST API, repository metadata for 52 candidate repos, fetched November 9, 2026 (first-party snapshot; script and raw JSON kept with this post)
• pypistats.org, recent downloads API, and npm, download counts API, last-30-day figures fetched November 9, 2026
• Crawl4AI, README, "Current Sponsors", read November 9, 2026
• Beautiful Soup, project on Launchpad
• Scrapy documentation, HttpProxyMiddleware; Playwright documentation, proxy settings; Crawlee documentation, proxy management
• GNU Project, GNU Affero General Public License v3