Crawleo Documentation

Base URL

All API requests should be made to:

https://api.crawleo.dev

Authentication

All endpoints require authentication using your API key. You can provide it in one of two ways: Option 1: Authorization Header

Authorization: Bearer YOUR_API_KEY

Option 2: X-API-Key Header

x-api-key: YOUR_API_KEY

Get Your API Key

Available Endpoints

Crawleo provides four API endpoints:

Bing Search API

GET /searchBing-powered web search with auto-crawling and content extraction. Optimized for LLM and RAG pipelines.

Google Search API

GET /google-searchGoogle SERP data with knowledge graphs, news, images, places, and shopping. Ideal for SEO and lead generation.

Crawler API

GET /crawlDirect URL crawling with JavaScript rendering support.

Headful Browser API

GET /headful-browserPremium headed browser crawling with advanced anti-bot evasion. For sites that block standard crawlers.

Which Search API should I use? Use the Bing Search API for LLM/RAG workflows — it supports auto-crawling and returns page content in Markdown. Use the Google Search API for SEO monitoring, lead generation, and structured SERP analysis.

Output Formats

Both APIs support multiple output formats optimized for different use cases:

Format	Crawler Parameter	Description	Best For
Raw HTML	`raw_html=true`	Original page source	Full content preservation
Enhanced HTML	`enhanced_html=true` (default)	Clean HTML (no ads, scripts)	Cleaner processing
Markdown	`markdown=true` (default)	Structured markdown	RAG pipelines, LLMs

Use Markdown output (enabled by default) for AI/LLM applications to minimize token usage.

Response Format

All responses are returned as JSON. Successful responses include the requested data, while errors return an error object:

{
  "error": {
    "code": "ERROR_CODE",
    "message": "Human-readable error message"
  }
}

HTTP Status Codes

Code	Description
`200`	Success
`400`	Bad Request - Invalid parameters
`401`	Unauthorized - Invalid or missing API key
`403`	Forbidden - Insufficient permissions
`429`	Credits exhausted or concurrent request limit reached
`500`	Internal Server Error

Credits System

API usage is tracked using credits. Each API call deducts credits from your monthly allowance based on your subscription plan.

Credit Costs by Endpoint

API Endpoint	Credits Cost
Bing Search API	10 credits per page of results
Google Search API	10 credits per request
Crawl API (HTTP request)	1 credit per URL
Crawl API (browser rendering)	10 credits per URL

Subscription Plans

Plan	Credits/Month	Concurrent Requests	Price
Free	10,000	5	$0/month
Developer	100,000	20	$20/month
Pro	1,000,000	50	$100/month
Scale	2,500,000	100	$200/month

Credits & Pricing

View detailed pricing, usage examples, and optimization tips.

Concurrent Request Limits

Each plan has a maximum number of concurrent requests. If you exceed this limit, you’ll receive a 429 response with the message:

Concurrent request limit reached ({limit}). Please wait for other requests to finish.

MCP Integration

For AI assistants and IDEs, Crawleo also provides a Model Context Protocol (MCP) endpoint:

https://api.crawleo.dev/mcp

MCP Integration

Learn how to connect AI assistants directly to Crawleo.

Getting Started

API Overview

Search APIs

Crawling APIs

Core Concepts

MCP Integration

Integrations

API Introduction

Base URL

Authentication

Get Your API Key

Available Endpoints

Bing Search API

Google Search API

Crawler API

Headful Browser API

Output Formats

Response Format

HTTP Status Codes

Credits System

Credit Costs by Endpoint

Subscription Plans

Credits & Pricing

Concurrent Request Limits

MCP Integration

MCP Integration

​Base URL

​Authentication

Get Your API Key

​Available Endpoints

Bing Search API

Google Search API

Crawler API

Headful Browser API

​Output Formats

​Response Format

​HTTP Status Codes

​Credits System

​Credit Costs by Endpoint

​Subscription Plans

Credits & Pricing

​Concurrent Request Limits

​MCP Integration

MCP Integration

Base URL

Authentication

Available Endpoints

Output Formats

Response Format

HTTP Status Codes

Credits System

Credit Costs by Endpoint

Subscription Plans

Concurrent Request Limits

MCP Integration