Περιγραφή
When a known AI crawler (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, and others) requests one of your posts or pages, this plugin returns clean Markdown of the main content on the same URL instead of the full theme HTML — no header, footer, sidebar or scripts. Regular visitors keep seeing your normal site.
It also generates a virtual llms.txt and llms-full.txt — a sitemap-style index of your content for AI tools — served at /llms.txt and /llms-full.txt without creating any physical files.
What it does
- Detects AI crawlers by User-Agent (the list is editable in the settings).
- Serves clean Markdown on the same URL with
Content-Type: text/markdown,Vary: User-AgentandX-Robots-Tag: noindex. - Adds machine-readable Schema.org JSON-LD (title, dates, author, cover image, publisher) to the Markdown front matter.
- Generates virtual
/llms.txtand/llms-full.txt. - Caches the converted Markdown in post meta for fast responses; the cache is cleared automatically when a post is edited.
- A “Hide from AI bots” checkbox on every post to exclude it.
Privacy and external services
This plugin does not connect to any external service, send any data off your site, or load any remote scripts. All processing happens on your own server.
Good to know
- No .htaccess edits and no Composer — a built-in, zero-dependency HTMLMarkdown converter is included.
- Does not interfere with SEO plugins, caches, or WooCommerce shop/cart/checkout pages.
- Private, password-protected and unpublished content is never served to bots — access rights are not bypassed.
Pro version
A Pro version is available with bot IP verification (anti-spoofing), bot-visit analytics with CSV export, a one-click self-test, and priority support. See the plugin’s settings page for details. The free version is fully functional on its own and is not time-limited.
Εγκατάσταση
- Upload the
adg-markdown-for-ai-crawlersfolder to/wp-content/plugins/, or install the plugin through the WordPress “Plugins” screen. - Activate the plugin.
- Go to Settings AI Bots & Markdown to configure it.
- On Nginx, if
/llms.txtreturns a 404, make sure your config falls through toindex.php(the standardtry_files $uri $uri/ /index.php?$args;block covers this). On Apache it works out of the box.
Συχνές Ερωτήσεις
-
Will private or draft pages be exposed to bots?
-
No. Markdown is served only for published, non-password-protected, non-private posts of the allowed types. Access rights are never bypassed.
-
Does it send my content to any external server?
-
No. There are no outbound requests at all — detection is by User-Agent, and all conversion happens locally.
-
Does it work with page builders like Elementor?
-
Yes. Content is taken through
the_content, so shortcodes and builder output are rendered before conversion. Testing on your specific setup is recommended. -
A bot still receives HTML instead of Markdown. Why?
-
Most often the request is blocked at the server/CDN level (HTTP 403) or served from a page cache before the plugin runs. Make sure your cache honors
Vary: User-Agentor excludes bot user-agents.
Κριτικές
Δεν υπάρχουν αξιολογήσεις για αυτό το πρόσθετο.
Συνεισφέροντες & Προγραμματιστές
“ADG Markdown for AI Crawlers” είναι λογισμικό ανοιχτού κώδικα. Οι παρακάτω έχουν συνεισφέρει στη δημιουργία του.
Συντελεστές
Μεταφράστε το “ADG Markdown for AI Crawlers” στην γλώσσα σας.
Ενδιαφέρεστε για την ανάπτυξη;
Περιηγηθείτε στον κώδικα, ανατρέξτε στο αποθετήριο SVN ή εγγραφείτε στο αρχείο καταγραφής αλλαγών ανάπτυξης μέσω RSS .
Σύνοψη αλλαγών
1.0.0
- Initial release: content negotiation (Markdown for AI bots on the same URL), Schema.org/JSON-LD front matter, post-level Markdown cache, virtual llms.txt / llms-full.txt, and a “Hide from AI bots” metabox.