Firefox Browser Add-ons
  • Extensions
  • Themes
    • for Firefox
    • Dictionaries & Language Packs
Log in
Preview of AI Crawler Access Checker

AI Crawler Access Checker by USA Marketing Pros

Check whether the website you're viewing allows or blocks AI and search crawlers like GPTBot, OAI-SearchBot, ClaudeBot and Googlebot. One click reads its robots.txt. Nothing is stored or sent.

Available on Firefox for Android™Available on Firefox for Android™
0 (0 reviews)0 (0 reviews)
Download Firefox and get the extension
Download file
Scan the QR code to open this extension in Firefox for Android

Extension Metadata

About this extension
AI Crawler Access Checker shows, in one click, which AI and search engine crawlers a website lets in.

Every website can have a robots.txt file that tells bots what they may visit. A single wrong line can keep a business out of ChatGPT search, Google AI Overviews, Perplexity or Bing, often without the owner knowing.

How to use it
1. Open any website.
2. Click the toolbar icon.
3. See which of 17 crawlers the site's robots.txt allows, blocks or partly blocks.

What it checks
- Search and AI answer crawlers: Googlebot, Bingbot, OAI-SearchBot, Claude-SearchBot, PerplexityBot and Applebot. Blocking these can keep a site out of search results and AI answers.
- User-requested agents: ChatGPT-User, Claude-User, Perplexity-User and Meta's fetcher, which open pages when someone asks an AI tool about them.
- AI training crawlers: GPTBot, ClaudeBot, Google-Extended, Applebot-Extended, Meta's training crawler, CCBot and Bytespider. Blocking these is a business choice and doesn't remove a site from search.

Each result includes a plain-language note on what the rule means and which part of robots.txt it came from.

Privacy
The add-on only runs when you click it, reads only the site's public robots.txt file, and never stores or sends anything.

A free web version is available at https://usamarketingpros.com/free-seo-tools/ai-crawler-checker/

Made by USA Marketing Pros, a web design and SEO agency in Arlington, Virginia.
Rated 0 by 0 reviewers
Log in to rate this extension
There are no ratings yet

Star rating saved

5
0
4
0
3
0
2
0
1
0
No reviews yet
Permissions and data

Data collection:

  • The developer says this extension doesn't require data collection.
Learn more
More information
Add-on Links
  • Homepage
  • Support site
  • Support Email
  • Copy add-on ID
Version
1.0.0
Size
20.3 KB
Last updated
7 days ago (Nov 10, 2026)
Related Categories
  • Web Development
License
All Rights Reserved
Version History
  • See all versions
Tags
  • google
  • search
Add to collection
Report this add-on
Go to Mozilla's homepage

Add-ons

  • About
  • Firefox Add-ons Blog
  • Extension Workshop
  • Developer Hub
  • Developer Policies
  • Community Blog
  • Forum
  • Report a bug
  • Review Guide

Download

  • Download Firefox
  • Windows
  • macOS
  • iOS
  • Android
  • Linux
  • All

Latest Builds

  • Nightly
  • Beta

Firefox for Business

  • Enterprise

Community

  • Connect
  • Contribute
  • Developer

Follow

  • Instagram
  • YouTube
  • TikTok
  • Bluesky
  • Podcast
  • Privacy
  • Cookies
  • Legal

Except where otherwise noted, content on this site is licensed under the Creative Commons Attribution Share-Alike License v3.0 or any later version. Android is a trademark of Google LLC.