Crawleo Documentation

Crawler API

curl --request GET \
  --url https://api.crawleo.dev/crawl \
  --header 'x-api-key: <x-api-key>'

import requests

url = "https://api.crawleo.dev/crawl"

headers = {"x-api-key": "<x-api-key>"}

response = requests.get(url, headers=headers)

print(response.text)

const options = {method: 'GET', headers: {'x-api-key': '<x-api-key>'}};

fetch('https://api.crawleo.dev/crawl', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

{
  "results": [
    {
      "url": "<string>",
      "status_code": 123,
      "raw_html": "<string>",
      "enhanced_html": "<string>",
      "markdown": "<string>",
      "page_text": "<string>",
      "screenshot": "<string>",
      "error": "<string>"
    }
  ],
  "credits": 123,
  "successful_pages": 123
}

GET

https://api.crawleo.dev

crawl

Crawler API

curl --request GET \
  --url https://api.crawleo.dev/crawl \
  --header 'x-api-key: <x-api-key>'

import requests

url = "https://api.crawleo.dev/crawl"

headers = {"x-api-key": "<x-api-key>"}

response = requests.get(url, headers=headers)

print(response.text)

const options = {method: 'GET', headers: {'x-api-key': '<x-api-key>'}};

fetch('https://api.crawleo.dev/crawl', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

{
  "results": [
    {
      "url": "<string>",
      "status_code": 123,
      "raw_html": "<string>",
      "enhanced_html": "<string>",
      "markdown": "<string>",
      "page_text": "<string>",
      "screenshot": "<string>",
      "error": "<string>"
    }
  ],
  "credits": 123,
  "successful_pages": 123
}

Overview

The Crawler API performs direct crawling of specified URLs with JavaScript rendering support. Ideal for extracting content from single pages or multiple URLs in a single request.

Endpoint

GET https://api.crawleo.dev/crawl

Parameters

Required Headers

x-api-key

string

required

Your Crawleo API key for authentication. (Alternatively, use the Authorization: Bearer YOUR_API_KEY header.)Example: x-api-key: YOUR_API_KEY or Authorization: Bearer YOUR_API_KEY

Required Parameters

urls

string

required

URL(s) to crawl. Can be a single URL or comma-separated list.Example: https://example.com or https://example.com,https://example.org

Rendering Options

render_js

boolean

default:"false"

Enable browser rendering for JavaScript-heavy sites.

true - Browser rendering (10 credits/URL)
false - HTTP request (1 credit/URL)

geolocation

string

ISO 3166-1 alpha-2 country code for geolocation (e.g., us, gb, de).Example: geolocation=us

Output Format Parameters

raw_html

boolean

default:"false"

Return the raw HTML as-is from the page source.

enhanced_html

boolean

default:"true"

Return cleaned/sanitized HTML with ads, scripts, and tracking removed.

page_text

boolean

default:"false"

Return plain-text extraction of the page content.

markdown

boolean

default:"true"

Return markdown conversion (recommended for RAG/LLM applications).

Screenshot Options

screenshot

boolean

default:"false"

Capture a screenshot of the page. Only available when render_js=true.

screenshot_full_page

boolean

default:"false"

Capture full-page screenshot vs viewport only.

Screenshots are only available when render_js=true. Setting screenshot=true without JavaScript rendering will be ignored.

Example Requests

Basic Crawl with Markdown Output

curl -X GET "https://api.crawleo.dev/crawl?urls=https://example.com&markdown=true" \
  -H "x-api-key: YOUR_API_KEY"

import requests

response = requests.get(
    "https://api.crawleo.dev/crawl",
    params={
        "urls": "https://example.com",
        "markdown": True
    },
    headers={"x-api-key": "YOUR_API_KEY"}
)

data = response.json()

const response = await fetch(
  "https://api.crawleo.dev/crawl?urls=https://example.com&markdown=true",
  {
    headers: { "x-api-key": "YOUR_API_KEY" }
  }
);

const data = await response.json();

Crawl Multiple URLs

curl -X GET "https://api.crawleo.dev/crawl?urls=https://example.com,https://example.org&markdown=true" \
  -H "x-api-key: YOUR_API_KEY"

import requests

urls = [
    "https://example.com",
    "https://example.org"
]

response = requests.get(
    "https://api.crawleo.dev/crawl",
    params={
        "urls": ",".join(urls),
        "markdown": True
    },
    headers={"x-api-key": "YOUR_API_KEY"}
)

data = response.json()

const urls = [
  "https://example.com",
  "https://example.org"
].join(",");

const response = await fetch(
  `https://api.crawleo.dev/crawl?urls=${encodeURIComponent(urls)}&markdown=true`,
  {
    headers: { "x-api-key": "YOUR_API_KEY" }
  }
);

const data = await response.json();

Crawl with JavaScript Rendering

curl -X GET "https://api.crawleo.dev/crawl?urls=https://example.com&render_js=true&markdown=true" \
  -H "x-api-key: YOUR_API_KEY"

import requests

response = requests.get(
    "https://api.crawleo.dev/crawl",
    params={
        "urls": "https://example.com",
        "render_js": True,
        "markdown": True
    },
    headers={"x-api-key": "YOUR_API_KEY"}
)

data = response.json()

Crawl with Screenshot and Geolocation

curl -X GET "https://api.crawleo.dev/crawl?urls=https://example.com&render_js=true&screenshot=true&screenshot_full_page=true&geolocation=gb" \
  -H "x-api-key: YOUR_API_KEY"

import requests

response = requests.get(
    "https://api.crawleo.dev/crawl",
    params={
        "urls": "https://example.com",
        "render_js": True,
        "screenshot": True,
        "screenshot_full_page": True,
        "geolocation": "gb"
    },
    headers={"x-api-key": "YOUR_API_KEY"}
)

data = response.json()

Get Multiple Output Formats

curl -X GET "https://api.crawleo.dev/crawl?urls=https://example.com&raw_html=true&enhanced_html=true&markdown=true&page_text=true" \
  -H "x-api-key: YOUR_API_KEY"

import requests

response = requests.get(
    "https://api.crawleo.dev/crawl",
    params={
        "urls": "https://example.com",
        "raw_html": True,
        "enhanced_html": True,
        "markdown": True,
        "page_text": True
    },
    headers={"x-api-key": "YOUR_API_KEY"}
)

data = response.json()

Response

A successful response returns crawled content for each URL:

{
  "results": [
    {
      "url": "https://example.com",
      "status_code": 200,
      "markdown": "# Example Domain\n\nThis domain is for use in illustrative examples...",
      "enhanced_html": "<html>...</html>"
    }
  ],
  "credits": 1,
  "successful_pages": 1
}

results

array

Array of crawl result objects.

Show Result object properties

url

string

The crawled URL.

status_code

integer

HTTP status code of the crawled page.

raw_html

string

Full HTML source (if raw_html=true).

enhanced_html

string

Cleaned/sanitized HTML (if enhanced_html=true).

markdown

string

Markdown content (if markdown=true).

page_text

string

Plain-text extraction (if page_text=true).

screenshot

string

Base64-encoded screenshot (if screenshot=true).

error

string

Error message if the crawl failed for this URL.

credits

integer

Number of credits consumed by this request. Varies based on rendering:

HTTP request (render_js=false): 1 credit per URL
Browser rendering (render_js=true): 10 credits per URL
Failed requests: 0 credits

successful_pages

integer

Number of URLs that were successfully crawled.

Use Cases

RAG Data Ingestion

Crawl documentation pages or knowledge bases and convert to Markdown for vector database ingestion.

# Example: Crawl docs for RAG
response = requests.get(
    "https://api.crawleo.dev/crawl",
    params={
        "urls": "https://docs.example.com/guide,https://docs.example.com/api",
        "markdown": True
    },
    headers={"x-api-key": "YOUR_API_KEY"}
)

for result in response.json()["results"]:
    # Add to vector database
    vector_db.add(result["markdown"], metadata={"url": result["url"]})

Content Extraction

Extract clean content from web pages for analysis or processing.

Web Scraping

Scrape multiple pages in a single API call with JavaScript rendering support.

AI Agent Tools

Provide AI agents with the ability to read and understand web pages.

Tips

For LLM applications, always use markdown=true to get clean, structured content that minimizes token usage. The markdown format is enabled by default.

Ensure you have permission to crawl the target URLs. Respect robots.txt and website terms of service.

Last modified on April 21, 2026

Google Maps API Headful Browser API

Getting Started

API Overview

Search APIs

Crawling APIs

Core Concepts

MCP Integration

Integrations

Crawler API

Overview

Endpoint

Parameters

Required Headers

Required Parameters

Rendering Options

Output Format Parameters

Screenshot Options

Example Requests

Basic Crawl with Markdown Output

Crawl Multiple URLs

Crawl with JavaScript Rendering

Crawl with Screenshot and Geolocation

Get Multiple Output Formats

Response

Use Cases

Tips

​Overview

​Endpoint

​Parameters

​Required Headers

​Required Parameters

​Rendering Options

​Output Format Parameters

​Screenshot Options

​Example Requests

​Basic Crawl with Markdown Output

​Crawl Multiple URLs

​Crawl with JavaScript Rendering

​Crawl with Screenshot and Geolocation

​Get Multiple Output Formats

​Response

​Use Cases

​Tips

Overview

Endpoint

Parameters

Required Headers

Required Parameters

Rendering Options

Output Format Parameters

Screenshot Options

Example Requests

Basic Crawl with Markdown Output

Crawl Multiple URLs

Crawl with JavaScript Rendering

Crawl with Screenshot and Geolocation

Get Multiple Output Formats

Response

Use Cases

Tips