Documentation Index
Fetch the complete documentation index at: https://docs.crawleo.dev/llms.txt
Use this file to discover all available pages before exploring further.
What is Crawleo?
Crawleo is a privacy-first, real-time web search and crawling API built specifically for:- AI developers
- LLM-powered applications
- Retrieval-Augmented Generation (RAG) pipelines
- AI agents and automation workflows
Get Started
Set up your first API call in under 5 minutes.
Core Features
Bing Search API
LLM-optimized web search with auto-crawling and content extraction. Best for RAG pipelines and AI agents.
Google Search API
Structured Google SERP data with knowledge graphs, news, images, places, and shopping. Best for SEO and lead generation.
Crawler API
Direct URL crawling with JavaScript rendering support.
Headful Browser API
Premium headed browser with advanced anti-bot evasion. For sites protected by Cloudflare, Akamai, and DataDome.
MCP Integration
Native Model Context Protocol support for AI assistants.
Why Crawleo?
Privacy-First
Privacy-First
No selling user data, no training models on your data, no persistent storage of your queries or crawled content.
LLM-Optimized Outputs
LLM-Optimized Outputs
Multiple output formats (HTML, Markdown, plain text) designed for reduced token usage and minimal post-processing.
Real-Time Data
Real-Time Data
Live web access, not cached datasets. Get the freshest data available.
Developer-First
Developer-First
Simple APIs, fast setup, predictable subscription-based pricing.
Integration Options
REST API
Direct HTTP integration. Perfect for custom applications, backends, and scripts.
MCP Protocol
Connect AI assistants directly to Crawleo. Works with Claude, Cursor, GitHub Copilot, and more.
Quick Links
API Reference
Complete API documentation
Pricing
View subscription plans
GitHub
Examples and integrations