POST/v1/map

Discover URLs

Fast URL discovery without Playwright. Uses a 3-tier strategy: (1) parse robots.txt for Sitemap directives, (2) parse sitemap XML recursively one level deep, (3) fall back to homepage link extraction with Cheerio. Returns deduplicated, same-domain URLs sorted by path.

Request body

Name	Type	Required	Description
url	string	required	Site root URL
limit	number	optionaldefault: 100	Maximum URLs to return
includeSubdomains	boolean	optionaldefault: false	Include links to subdomains of the target host
includeExternal	boolean	optionaldefault: false	Include off-domain links (capped at 50 results)

Example

bash

curl -X POST http://localhost:3000/v1/map \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com","limit":500}'

Response

bash

{
  "success": true,
  "data": {
    "links": [
      "https://example.com/",
      "https://example.com/about",
      "https://example.com/docs"
    ],
    "total": 3,
    "source": "sitemap"
  }
}

Try it

Live preview only works when the API is running on your machine.

Run docker compose up and the Try-It panel will become interactive. Until then, copy the curl example below.

Attach a webhook Structured extraction