=== Sali LLMs.txt Generator ===
Contributors: ghasali
Tags: llms.txt, llm, ai seo
Requires at least: 5.6
Tested up to: 7.1
Stable tag: 1.0.0
Requires PHP: 7.4
License: GPLv2 or later
License URI: https://www.gnu.org/licenses/gpl-2.0.html

Automatically generate and serve structured llms.txt and llms-full.txt files to make your content readable and discoverable by AI models & LLMs.

== Description ==

Sali LLMs.txt Generator is a lightweight, high-performance plugin that automatically compiles and serves structured markdown index directories for LLM agents, scrapers, and web crawlers.

By providing a standardized `/llms.txt` directory index at your root domain, search crawlers can easily list, parse, and consume your content. The plugin also builds `/llms-full.txt` which concatenates full page markdown for accelerated ingestion.

Unlike generic text generators, Sali LLMs.txt Generator features native deep integration with **WooCommerce**. It reads product specifications and converts catalog information (SKU, price, stock status, categories, and descriptions) into clean, standard-compliant markdown for seamless AI catalog indexing.

=== Engineered for Performance & Compatibility ===

This plugin is built from the ground up with a focus on e-commerce catalog complexity, resource efficiency, and compatibility, ensuring it uses minimal server resources:

* **Transient Database Caching:** Directory and content pages are cached for 12 hours using WordPress Transients. When an AI crawler requests the directory, the cached markdown is served instantly without querying the database, resulting in near-zero CPU and RAM overhead.
* **Smart Cache Purging:** Caches are automatically flushed whenever you create, edit, trash, or delete posts, pages, or WooCommerce products, keeping your indexes updated without manual work.
* **Size Protection Limits:** Caps query counts per post type to prevent memory exhaustion on larger websites.
* **Conflict-Free Architecture:** All classes, functions, options, and hooks use isolated namespace prefixes (like `sali_llms_`), ensuring complete compatibility with core WordPress features, SEO plugins, and third-party tools.

== Key Features ==

* **WooCommerce Catalog Integration:** Translates products, prices, SKUs, stock status, and descriptions into clean markdown.
* **Dynamic Search Retrieval API:** Real-time search query endpoint (`/llms-search.txt?q=...`) to allow conversational AI agents to search your catalog and posts instantly.
* **Selective AI Bot Blocker:** A configuration grid to block specific crawlers with a `403 Forbidden` header and inject custom `Disallow` directives into the dynamic `robots.txt` output.
* **AI Crawler Analytics Tracker:** An admin dashboard tracking cumulative and daily crawling logs of major AI agents.
* **Server-Agnostic File Generation:** A single-click "Generate llms" button that writes physical files to the site root using standard WordPress Filesystem APIs, resolving Nginx 404 issues without server edits.
* **Robots.txt Auto-Integration:** Automatically appends the directory link to your site's dynamic `robots.txt` file for native AI discovery.
* **Dynamic Custom Post Types:** Scans all registered public post types (Posts, Pages, WooCommerce Products, Portfolios, etc.) dynamically for index inclusion.
* **Granular Exclusions:** Exclude specific items by Post IDs or entire post/product categories by comma-separated slugs.
* **Custom AI Instructions:** Prepend a system prompt to guide crawlers on how to consume your site's content.

== Installation ==

1. Upload the `sali-llms-txt-generator` folder to the `/wp-content/plugins/` directory, or search for "Sali LLMs.txt Generator" in your WordPress plugins panel and install.
2. Activate the plugin through the 'Plugins' menu in WordPress.
3. Access configuration and documentation at the **Sali LLMs** menu in your WordPress dashboard.
4. Visit your settings tab and click **Generate llms** to write physical files, or visit `yourdomain.com/llms.txt` to view your dynamic directory index.

== Frequently Asked Questions ==

= How do I configure which content is indexed? =
You can select which Custom Post Types to include in the settings dashboard. You can also exclude specific IDs or category slugs.

= How does it protect server CPU and RAM? =
All generation queries are cached for 12 hours using WordPress Transients. This serves crawls instantly from memory/cache without reloading the database or bloating server RAM. The cache is automatically cleared when content changes.

= Why am I getting a 404 error when visiting /llms.txt? =
On Apache/LiteSpeed servers, navigate to **Settings > Permalinks** and click **Save Changes** to flush rules. On Nginx servers, simply go to our plugin settings tab and click **Generate llms** to write physical files directly to the root folder, allowing Nginx to serve them statically without configuration changes.

= What is the AI Bot Blocker? =
It allows you to selectively restrict specific AI crawlers (like GPTBot, ClaudeBot, Perplexity, etc.) from indexing your directory. When blocked, the crawlers receive a `403 Forbidden` header, and the plugin automatically injects `Disallow` directives for those bots into your site's `robots.txt`.

= What is the AI Search API? =
The `/llms-search.txt?q=your-query` endpoint allows Custom GPT Actions or other conversational bots to search your WooCommerce catalog and articles in real-time, returning matches in clean markdown format.

= What does the Analytics Dashboard show? =
It logs cumulative hits, last active times, and daily crawler requests (for the last 30 days) of major AI crawlers, helping you monitor which engines are indexing your website.

== Changelog ==

= 1.0.0 =
* Initial release of Sali LLMs.txt Generator.
* Implemented standard routing for /llms.txt, /llms-full.txt, and /llms-search.txt.
* Implemented WooCommerce product metadata compilation (Price, SKU, Stock, Categories).
* Created admin settings dashboard with interactive tabbed layout.
* Added "Generate llms" manual file writer to bypass Nginx 404 issues out-of-the-box.
* Added AI Bot Blocker, AI Crawler Analytics Tracker, and dynamic robots.txt injection.
* Added 12-hour transient caching and automatic purge hooks on post/product modifications.
