Is Web Scraping Legal? Myth vs. Law in 2026, Country by Country

Is web scraping legal? In most countries, yes, when you collect publicly available pages without logging in or getting around a technical barrier. What makes a scraping project illegal is almost never the act of reading a page. It is what you collect, what you do with it, and how you got past whatever stood in the way. Personal data falls under privacy law, and copied content falls under copyright. Logged-in access can bring in contract law, and defeating a bot defense can bring in anti-circumvention or computer misuse statutes.

That pattern holds across the nine jurisdictions below, but the details differ a lot. Singapore and Japan have copyright exceptions that cover commercial data analysis, while the UK's covers only non-commercial research. Canadian regulators treat scraped social media data as off-limits without consent. And in July 2026, two U.S. federal courts reached different early-stage rulings on whether bypassing a search engine's bot defenses can violate the DMCA's anti-circumvention rule. This is a summary of the law as of early November 2026, not legal advice. For each claim we read the statute, ruling, or regulator text, or a law firm's report of it, and linked the source inline: 46 external sources across nine jurisdictions, gathered in November 2026.

Key Takeaways

• Reading public pages is generally not hacking. The U.S. Supreme Court's 2021 "gates-up-or-down" test, and similar rules in Germany and Japan, focus on getting around an access barrier.
• Public personal data is still regulated. Data protection authorities said so in joint statements (12 in 2023, 16 in 2024), and the EDPB opened draft scraping guidelines for consultation in July 2026.
• Copyright exceptions for text and data mining vary widely: commercial use is allowed in the EU (unless the owner opts out), Japan, and Singapore, but only non-commercial research in the UK, and Australia ruled out an exception in 2025.
• Bypassing bot defenses is the new legal frontier. In July 2026 two U.S. federal courts reached different early-stage results on DMCA anti-circumvention claims over search-result scraping.
• robots.txt is not a law anywhere we looked, but EU regulators and the EU AI Act code of practice now treat it as a signal you're expected to respect.

The short answer by country

Each column is one of the four areas of law that decide most scraping questions. "Permissive" means public-page scraping is generally allowed under that rule, "conditional" means it depends on how and what you collect, and "restrictive" means the default leans against you.

Myth 1: "Web scraping is hacking"

No. Reading public web pages is not hacking under U.S. federal law. In Van Buren v. United States (2021), the U.S. Supreme Court read the Computer Fraud and Abuse Act narrowly. Liability "stems from a gates-up-or-down inquiry," meaning whether you entered a part of a system that was closed to you (Supreme Court opinion via Cornell LII). The following year, ruling on a preliminary injunction, the Ninth Circuit applied that reasoning in hiQ Labs v. LinkedIn. It concluded that the CFAA's idea of access "without authorization" likely does not apply to public websites (Goodwin). We covered both cases in detail in scraping and hacking aren't the same thing.

In 2026 the Ninth Circuit went a step further. Amazon v. Perplexity concerned an AI browser agent shopping on a user's behalf. On August 4 the court held that the user, not the agent's developer, is the party who "accesses" Amazon's computers. In the court's reading, the statute contemplates access by a person, not an AI tool. It vacated a preliminary injunction against the agent (Jones Day).

Other countries draw a similar line at barriers. Germany's data espionage offense in its Criminal Code is aimed at data specially secured against unauthorized access. In 2014 the Federal Court of Justice held that screen scraping freely accessible flight data was not unfair competition. Ignoring the site's terms of use did not change that, though circumventing a technical protection would have (LTO).

The UK Computer Misuse Act turns on whether access is "unauthorised" (legislation.gov.uk), and we found no UK ruling comparable to Van Buren, so the line is less settled and getting around a block carries more risk.

Myth 2: "If it's public, you can use it"

Public is not the same as free of rules. In August 2023, 12 data protection authorities, including the UK's ICO and Canada's Privacy Commissioner, said in a joint statement that "personal information that is publicly accessible is still subject to data protection and privacy laws in most jurisdictions" (Office of the Privacy Commissioner of Canada). Their October 2024 follow-up, signed by 16 authorities, added that permitted scraping must be "done lawfully and in accordance with strict contractual terms" (OPC).

Australia's regulator is just as direct: "Developers should not assume information posted publicly can be used to train models," and scraping "may constitute a covert and therefore unfair means" of collection (OAIC).

Copyright is the other limit. A page can be public and still protected. The Third Circuit affirmed in September 2026 that copying Westlaw's legal headnotes to build a competing AI search tool was not fair use, an early appeals-court ruling on AI training and fair use. The court distinguished generative AI: "ROSS's AI platform cannot generate original expression" (Ballard Spahr).

Myth 3: "robots.txt is the law"

No statute in the jurisdictions we reviewed makes robots.txt binding on its own, a point we covered in is robots.txt legally binding. But it is gaining legal weight in three places:

1. EU copyright. The text and data mining exception in Article 4 of the EU copyright directive applies unless the rightholder has reserved their rights "in an appropriate manner, such as machine-readable means in the case of content made publicly available online" (EUR-Lex). Many sites use robots.txt for that reservation, though courts have not settled whether a general robots.txt rule qualifies.
2. EU AI rules. Signatories of the EU AI Act's general-purpose AI code of practice commit to using crawlers that follow robots.txt, the Robot Exclusion Protocol (Clifford Chance).
3. GDPR balancing. France's CNIL tells AI developers to exclude sites that object to scraping through robots.txt or CAPTCHAs (Clifford Chance), and the EDPB's draft guidelines treat robots.txt the same way (Herbert Smith Freehills Kramer).

The reverse doesn't hold either: under the EDPB's draft, a site with no robots.txt has not consented to anything (Herbert Smith Freehills Kramer). For the 2026 standards work on AI training opt-outs, see the new robots.txt AI training opt-out.

Myth 4: "Terms of service always bind scrapers"

Only if the scraper agreed to them. A federal court in California ruled for Bright Data in January 2024 on Meta's claims over logged-off scraping of public Facebook and Instagram data. The court found that a logged-off scraper "stands in the same shoes as a visitor to whom the Terms cannot apply as a matter of basic contract law" (Farella Braun + Martel). In May 2024 another judge in the same court held that X's contract claims over scraping and selling public posts were preempted by the Copyright Act (Frankfurt Kurnit).

Two caveats stop this from flipping into the opposite rule. Logged-in scraping, where you clicked "I agree," is still exposed to breach of contract claims. That is how hiQ ultimately lost: in late 2022 the district court held LinkedIn's user agreement enforceable against it. The case ended in a consent judgment that included $500,000 and a permanent injunction (Proskauer). And in the EU, the Court of Justice held in Ryanair v. PR Aviation (2015) that when a database isn't protected by copyright or the database right, the owner can restrict its use by contract (Society for Computers and Law).

Myth 5: "GDPR bans web scraping"

It doesn't. The GDPR requires a legal basis, and EU regulators accept that legitimate interest can be one for scraping, with safeguards. The EDPB's draft Guidelines 03/2026 on web scraping for generative AI list those safeguards: collect only what you need, exclude sites with data about minors or highly sensitive data, and skip sites that clearly object. The consultation opened in July 2026 and closed on October 30, and the final version is pending (EDPB). We broke the draft down in the EDPB web scraping guidelines.

The strictest reading came from the Dutch data protection authority, which said in 2024 that scraping by private individuals and organizations is "almost always a violation" of the GDPR (Pinsent Masons). Treat that as the high-risk end of the range, not the EU rule.

Is web scraping legal in the US?

Yes, for public pages accessed without logging in, as far as federal computer crime law goes. The live risks are elsewhere:

• Copyright. Courts are deciding AI training cases one record at a time. In June 2025 two judges in the Northern District of California found fair use on the records before them in Bartz v. Anthropic and Kadrey v. Meta, while the Third Circuit's 2026 Ross ruling went the other way for a non-generative tool. In Bartz, the court did not treat books downloaded from pirate libraries as fair use, and the case ended in a $1.5 billion class settlement (Bloomberg Law).
• Trespass to chattels. Older U.S. cases used this property theory against scrapers, but California's Supreme Court held in Intel v. Hamidi (2003) that it requires actual harm to the computer system or its operation (Justia). In practice, a light, well-behaved crawler rarely fits it.
• Anti-circumvention (New York). On July 31, 2026, in a case Reddit brought over the scraping of Google search results, a federal court in New York let DMCA section 1201(a) claims proceed against a search-results API provider and Perplexity. At the pleading stage, it held that Google's SearchGuard is a technological measure that "effectively controls access" (Loeb & Loeb).
• Anti-circumvention (California). Eleven days earlier, on July 20, a federal court in California dismissed Google's own DMCA claims against the same API provider. The dismissal was without leave to amend for search results that contain no copyrighted content, and with leave to amend for results that do (FindLaw). Neither decision is a finding of liability, and the law here is unsettled.
• State privacy laws apply to personal data. California's CCPA, for example, excludes "publicly available information" from personal information. It defines that term specifically: government records, and information that a business reasonably believes the consumer or widely distributed media lawfully made public (California Civil Code 1798.140). Other states' definitions differ.

Is web scraping legal in Europe and the EU?

Generally yes, with the GDPR as the main constraint. If you collect personal data, you need a legal basis (usually legitimate interest), a minimization plan, and a way to honor objections. For copyrighted content, Article 4 allows commercial text and data mining of lawfully accessible works unless the rightholder reserved their rights in an appropriate way, such as a machine-readable opt-out.

The EU also has a database right that most other jurisdictions here lack. Under the 1996 Database Directive, the maker of a database built with substantial investment can stop others from extracting or reusing a substantial part of it. In CV-Online Latvia v. Melons (2021), about a job-search engine that indexed another site's listings, the Court of Justice held that this can apply to scraping, but only where the extraction risks the investment the database maker made (EUR-Lex, C-762/19). Scraping a whole listings site is riskier in the EU than scraping a few facts from it.

The copyright side is moving. On November 3, 2026, Advocate General Szpunar advised the Court of Justice in Like Company v. Google on AI chatbots that retrieve and reuse online press articles in real time to answer users. In his view, that may, in certain circumstances, infringe press publishers' rights. He advised the court to set aside the referred questions on model training as hypothetical (Courthouse News). An advocate general's opinion doesn't bind the court, and the judgment is still pending.

Is web scraping legal in Germany?

Yes, provided you don't overcome a technical access barrier and you handle personal data under the GDPR. Germany's copyright act allows text and data mining under section 44b, including for commercial purposes, unless the owner opted out in machine-readable form. Section 60d adds a broader exception for scientific research.

The leading case, Kneschke v. LAION, involves a non-profit dataset builder and has gone LAION's way twice. In December 2025 the Hamburg Higher Regional Court upheld the September 2024 first-instance ruling, applying both exceptions. It held that a photographer's plain-language objection "did not meet the requirement of a machine-readable format" (Morgan Lewis). The Federal Court of Justice heard the appeal on September 3, 2026, with a decision scheduled for December 17 (BFF).

The limit is memorization. The Munich Regional Court held in GEMA v. OpenAI (November 2025) that song lyrics memorized in a model are an unauthorized reproduction that the text and data mining exception does not cover. An appeal is pending at the Higher Regional Court Munich, according to the same Morgan Lewis overview.

Is web scraping legal in the UK?

Collecting public pages is generally lawful, but the UK is stricter than the EU on copyright. Section 29A of the Copyright, Designs and Patents Act allows copying for computational analysis only "for the sole purpose of research for a non-commercial purpose," and contract terms that try to override it are "unenforceable" (legislation.gov.uk). In March 2026 the government said a broad commercial exception with an opt-out is "no longer the government's preferred way forward," without picking a replacement (Osborne Clarke).

For personal data, the UK GDPR applies, and the ICO has said legitimate interests is likely the only lawful basis for collecting data to train AI models (Osborne Clarke). The Upper Tribunal confirmed in October 2025 that the ICO can act against a scraper based outside the UK (11KBW).

Is web scraping legal in India?

It depends on permission, which the courts haven't defined for public pages. Section 43 of the IT Act creates civil liability for anyone who, "without permission of the owner," downloads, copies, or extracts data from a computer, and a separate provision of the act makes such acts a crime when done dishonestly or fraudulently (Centre for Internet and Society).

Two recent developments lean permissive. Section 3(c)(ii) of the Digital Personal Data Protection Act 2023 excludes personal data that the person made public themselves, though the government has told parliament that organizations scraping public data must still comply with consent and other obligations (IAPP). The act's rules were notified in November 2025 with an 18-month rollout (PIB). And on July 24, 2026, the Delhi High Court refused ANI's request for an interim injunction against OpenAI. On a prima facie view, it found that storing ANI's articles to train the models falls under the fair dealing exception in section 52(1)(a) (SpicyIP). The case still goes to trial.

Is web scraping legal in Canada?

Conditionally. Reading public pages isn't treated as a computer crime in any Canadian ruling we found. Canada and Australia are the strictest jurisdictions here for personal data, and Canada's courts have gone furthest. Regulators read the "publicly available" exception in privacy law narrowly, and have applied it to scraped social media data. In February 2026 the British Columbia Court of Appeal found that reading reasonable as applied to a facial recognition company's large-scale scraping (BLG). Federal and provincial commissioners found in May 2026 that "OpenAI did not obtain valid consent for the collection of personal information" in its early model training (OPC). Canada has no text and data mining exception, so copyright questions go through fair dealing.

Is web scraping legal in Australia?

Conditionally. Accessing public pages isn't an offense in any Australian ruling we found, but personal data and copyright both lean restrictive. The privacy regulator, OAIC, warns that scraping can be covert and therefore unfair collection. Its 2021 determination against a facial recognition company still stands after the company withdrew its tribunal challenge in 2023 (OAIC). In October 2025 the Attorney-General ruled out a copyright exception for AI training, saying the government had "no plans to weaken copyright protection when it comes to AI" (iTnews).

Is web scraping legal in Singapore?

Singapore is one of the most permissive places on this list. Section 244 of the Copyright Act 2021 allows computational data analysis, including for commercial purposes, as long as you have lawful access to the material. The official example is a paywall: you can't get around it to claim the exception (Asia IP).

Separately, the Computer Misuse Act 1993 makes unauthorized access to computer material an offense (Singapore Statutes Online). The data protection law has a "publicly available" exception, and the regulator's July 2026 guidance on generative AI says data behind logins or paywalls needs a case-by-case check before you treat it as public (Herbert Smith Freehills Kramer).

Is web scraping legal in Japan?

For copyright, yes: Japan's law is the broadest here. Article 30-4 permits uses that are not for "enjoying" the expression in a work, such as data analysis and AI training, unless they unreasonably prejudice the rightholder. The Agency for Cultural Affairs' 2024 guidance adds that the exception doesn't apply when an enjoyment purpose exists alongside the analysis (WIPO). Personal data falls under the Act on the Protection of Personal Information, overseen by the Personal Information Protection Commission (PPC). The Unauthorized Computer Access Act targets getting around access controls, not reading public pages (Japanese Law Translation).

A practical checklist before you scrape

None of this replaces a lawyer, but these questions catch most of the risk:

1. Are you logged in? Logged-out collection of public pages carries the least contract and computer misuse risk.
2. Are you getting around anything? CAPTCHAs, bot defenses, paywalls, and IP bans are the barriers that anti-circumvention and computer misuse laws care about.
3. Is there personal data? If yes, you need a legal basis, minimization, and a deletion plan, and in Canada and Australia, a much higher bar.
4. What will you do with the content? Price monitoring and indexing are different from republishing articles or training a model on them.
5. Has the site opted out? Check robots.txt, any machine-readable AI reservation, and the site's terms, especially for EU and German sources.
6. Which tools will you use? Several popular open-source scrapers don't check robots.txt by default, so confirm the setting in yours; see open-source web scraping tools for the main options.
7. How is your network sourced? If you use proxies, the provider's sourcing is part of your compliance story. Massive's residential network uses devices whose owners opted in through the Massive SDK, and the company is SOC 2 audited, GDPR compliant, and AppEsteem certified, with an audit trail from source to request. We explain how that sourcing works in are residential proxies legal.

The bottom line

Put the checklist to work on one example. A price-monitoring crawler that stays logged out, never gets around a CAPTCHA or IP ban, skips personal data, keeps only prices, and honors robots.txt clears items 1 to 5. Switch it to a logged-in account to see member prices, and it is back in contract territory under item 1.

Is web scraping legal? In most of the world, reading public pages is, and the real questions are about personal data, copyright, contracts you agreed to, and barriers you got around. Those rules differ most between the UK, Canada, and Australia on one side and Japan and Singapore on the other. Check the law where your data comes from as well as your own.

Sources

• Van Buren v. United States, 593 U.S. 374 (2021), Cornell LII
• Goodwin, "Ninth Circuit: Web Scraping Does Not Violate CFAA", April 2022
• Jones Day, "Ninth Circuit Vacates CFAA Injunction Against Perplexity's Comet AI Agent", September 2026
• Ballard Spahr, "Third Circuit Addresses Fair Use in AI Training", October 2026
• Proskauer, hiQ and LinkedIn settlement, December 2022; California Civil Code section 1798.140
• Loeb & Loeb, Reddit DMCA ruling on search-result scraping, August 2026; FindLaw, July 20, 2026 order in Google's DMCA case
• Justia, Intel Corp. v. Hamidi, 30 Cal. 4th 1342 (2003)
• Farella Braun + Martel, Meta v. Bright Data; Frankfurt Kurnit, X Corp. v. Bright Data
• Bloomberg Law, "Anthropic Authors $1.5 Billion Deal Receives Final Approval"
• Courthouse News, AG opinion in Like Company v. Google, November 2026
• EUR-Lex, CV-Online Latvia v. Melons, C-762/19, June 2021
• EUR-Lex, Directive (EU) 2019/790; Society for Computers and Law, Ryanair v. PR Aviation
• EDPB, Guidelines 03/2026 on web scraping in the context of generative AI; Herbert Smith Freehills Kramer, summary of the guidelines
• Clifford Chance, EU AI Act copyright compliance and CNIL web scraping guidance; Pinsent Masons, Dutch DPA guidance
• LTO, BGH I ZR 224/12; Morgan Lewis, AI and copyright in Germany; BFF, Kneschke v. LAION at the BGH
• legislation.gov.uk, Computer Misuse Act 1990, s.1 and CDPA 1988, s.29A; Osborne Clarke, UK AI and copyright report; Osborne Clarke, ICO views on personal data and generative AI; 11KBW, Upper Tribunal ruling on the ICO's territorial reach
• Office of the Privacy Commissioner of Canada, 2023 joint statement on data scraping, 2024 concluding statement and its announcement, and OpenAI findings; BLG, BC Court of Appeal privacy ruling
• Centre for Internet and Society, IT Act section 43; IAPP, scraping public data in India; PIB, DPDP Rules 2025; SpicyIP, ANI v. OpenAI
• OAIC, guidance on training generative AI and statement on its 2021 facial recognition determination; iTnews, AI training exemption ruled out
• Asia IP, Singapore's computational data analysis exception; Herbert Smith Freehills Kramer, PDPC generative AI guidance
• Singapore Statutes Online, Computer Misuse Act 1993; Japan PPC, laws and policies
• WIPO, General Understanding on AI and Copyright in Japan; Japanese Law Translation, Unauthorized Computer Access Act