AI Search Tracking Checklist: How to Attribute Calls From ChatGPT, Claude, and Perplexity

Download the Checklist
  1. SEJ
  2.  ⋅ 
  3. Facebook

Meta Delays The Terms That Let It Crawl Business Sites

  • Meta pushed back the start date for both of its updated business agreements.
  • Meta says it wants time to explain the terms, including how its crawlers work.
  • The crawling permission stays in the terms while EU and UK regulators review complaints.

Meta delayed its updated business terms amid complaints to EU and UK regulators. The permission to crawl sites tied to its business users hasn't changed.

Meta Delays The Terms That Let It Crawl Business Sites

Meta has pushed back the effective date of its updated Commercial Terms and Self-Serve Ad Terms from Oct. 30 to Nov. 20. The crawling permission that drew complaints from publishers and advertisers is still in both agreements. The Commercial Terms cover any business use, including managing a Facebook Page.

A Meta spokesperson told Nieman Lab the company is extending the date “to allow more time for us to explain to our partners and stakeholders what’s changing and what’s not.” The spokesperson said that includes how Meta uses web crawlers.

I covered the original terms and Meta’s removal of training from the clause earlier this month.

What The Delay Doesn’t Change

The U.S. English previews of the Commercial Terms and Self-Serve Ad Terms list Nov. 20 as the effective date. Nieman Lab reported that as of Oct. 23, Meta hadn’t changed the terms except for the date.

The crawling paragraph still lets Meta use automated software and “create test credentials” to crawl pages associated with a business’s products or services, and to periodically pull updates from them. Neither preview mentions robots.txt.

The current Self-Serve Ad Terms, dated Aug. 22, 2025, already let Meta crawl content on an advertiser’s domains for uses that include “research and training” once an ad order is placed.

As of Oct. 25, the UK English, German and French Commercial Terms previews still listed Oct. 30.

Complaints Reach EU And UK Regulators

The Movement for an Open Web, an advocacy group whose members include advertisers and publishers, has asked the European Commission and the UK’s Competition and Markets Authority (CMA) to block Meta’s terms with interim measures while they investigate. Meta told Reuters it disagreed with the complaint.

After the delay, a CMA spokesperson told Nieman Lab, “We will continue to scrutinize these terms and engage with Meta and others as we consider our next steps.”

The Commission told Reuters that it will be assessing the complaint. It told Nieman Lab on Oct. 23 that it welcomes this postponement and is committed to ensuring compliance by Meta with EU law, including Article 102 TFEU and the Digital Markets Act.

Meta itself announced that there was a delay. None of the regulatory body statements that Reuters and/or Nieman Lab reported ordered a delay or an adjustment to the Terms.

Publishers Want The Clause Changed

WAN-IFRA (World Association of News Publishers) requested that Meta temporarily stop the crawling provision for news publishers, and also make robots.txt binding within its terms. On Oct. 23, WAN-IFRA CEO Stig Ørskov stated, “This postponement is a welcome step, but it does not address our concerns.” The same day, the European Alliance of News Agencies called again on Meta to eliminate the provision altogether and expressed hope that the new deadline would create a meaningful dialogue.

Why This Matters

Meta now says it will implement the revised commercial terms of service effective Nov. 20. As of Oct. 25, however, at least some local previews of Meta’s business terms showed Oct. 30 as the effective date. In addition to other rights and obligations granted under the “Commercial Terms,” a business agrees to all updates to the terms, including crawling permissions, if it keeps using Meta’s business products after receiving notice from Meta.

According to Meta, webmasters may indicate their preferred crawling behaviors through robots.txt in order to influence how Meta crawls those sites. Meta also provides information about different crawlers used for various uses such as advertising, AI model training, and Meta AI search. However, the new terms do not provide information regarding whether a robots.txt setting limits what a business has agreed to grant when they accept the new terms.

Looking Ahead

Meta’s updated terms are set to take effect on Nov. 20. Publisher groups are hoping to see the crawling provision changed before then, while the CMA continues to consider its next steps. In the meantime, take a moment to review the crawling paragraph and check how your robots.txt handles each Meta crawler.

Featured Image: Samuel Boivin/Shutterstock

Category News Facebook
SEJ STAFF Matt G. Southern Senior News Writer at Search Engine Journal

Follow on YouTube. Matt G. Southern is the Senior News Writer at Search Engine Journal, where he’s covered Google, SEO, ...