Preview — auto-detect scraper configs

Two convenience endpoints used by the scraper-builder UI’s “Simple Mode”. Hand over a target URL (or a curl invocation) and the service runs every applicable detection strategy in parallel — SINGLE_PRODUCT, MULTIPLE_PRODUCTS, APPLICATION_LD_JSON, plus the Amazon / Google specialisations — and returns the candidate scraper configs that successfully produced data, along with their sample rows.

Nothing persists. Pick a candidate, then save it via PUT /api/scraper.

Preview fetches a page, so it is billable — charged per page delivered against the prepaid wallet, from €0.00015 for a plain page. With no balance and no free requests left the call returns 402 with the code WALLET_INSUFFICIENT_BALANCE; see Wallet & billing.

Endpoint summary

Preview from URL — POST /api/scraper-simple/url-based

POST /api/scraper-simple/url-based?tryWithHiddenData=false&useCache=false
Authorization: Bearer <key>
Content-Type: application/json
 
{
  "id": null,
  "name": "competitor-prices",
  "groupId": "grp_xyz",
  "url": "https://competitor.com/products",
  "itemsConfig": null
}

Request body (SimpleModeWithUrlDTO)

Query params

Response (200) — Set<ScraperSampleDataDTO>. One entry per detection strategy that produced data:

[
  {
    "scraperDTO": {
      "name": "competitor-prices",
      "groupRef": "grp_xyz",
      "type": "MULTIPLE_PRODUCTS",
      "sourceConfig": {
        "url": "https://competitor.com/products",
        "pagination": { "type": "URL_PAGE_PARAM", "param": "page", "max": 50 }
      },
      "itemsConfig": [
        { "name": "title",    "selector": "h2.product-title",   "kind": "TEXT" },
        { "name": "price",    "selector": ".price",             "kind": "TEXT" },
        { "name": "imageUrl", "selector": "img.product-image",  "kind": "ATTR", "attr": "src" }
      ]
    },
    "sampleData": [
      { "title": "Premium Widget XL", "price": "29.99", "imageUrl": "https://..." },
      { "title": "Standard Widget",   "price": "19.99", "imageUrl": "https://..." }
    ],
    "executionTimeSec": 2.4
  }
]

Empty array is a normal success state — no detection strategy succeeded for this URL; treat it as “this URL is unsupported by the available detectors”.

Errors — 400 (validation failed) / 401 / 402 (wallet out of balance — WALLET_INSUFFICIENT_BALANCE) / 403 (N/A) / 404 (N/A) / 429 / 500 (correlationId returned if a detector or downstream proxy crashes).

Node.js

const res = await fetch(
  'https://portal.scrapewise.ai/api/scraper-api/api/scraper-simple/url-based',
  {
    method: 'POST',
    headers: {
      Authorization: `Bearer ${process.env.SCRAPEWISE_KEY}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      name: 'competitor-prices',
      groupId: 'grp_xyz',
      url: 'https://competitor.com/products',
    }),
  }
);
const candidates = await res.json();
const pick = candidates[0];          // pick whichever shape matches your need
// → then persist via PUT /api/scraper with `pick.scraperDTO`

Preview from curl — POST /api/scraper-simple/curl-based

Same idea but the input is a full curl invocation. Use when the target endpoint needs custom request shape — cookies, headers, an auth bearer, or a POST body — that wouldn’t survive a plain GET. Scrapewise parses the curl, replays the exact request, then runs detection on the response.

Requires the API plan feature.

POST /api/scraper-simple/curl-based
Authorization: Bearer <key>
Content-Type: application/json
 
{
  "name": "vendor-graphql-products",
  "groupId": "grp_xyz",
  "curl": "curl 'https://vendor.example/graphql' -H 'Cookie: session=...' -H 'Content-Type: application/json' --data-raw '{\"query\":\"{ products { id title price } }\"}'"
}

Request body (SimpleModeWithCurlDTO)

Response (200) — same Set<ScraperSampleDataDTO> shape as /url-based.

Errors — 400 (curl string malformed) / 401 / 402 (wallet out of balance — WALLET_INSUFFICIENT_BALANCE; or, on accounts still on legacy plan gating, the plan lacks the API feature — switch on errors[0].code to tell them apart) / 403 (N/A) / 404 (N/A) / 429 / 500.

Dedicated endpoints

For 18 targets Scrapewise recognises the URL and calls a dedicated upstream endpoint instead of scraping the page HTML. Nothing to configure — post the ordinary public URL and the routing happens server-side; the candidate comes back tagged with the endpoint that produced it.

Amazon deals, best sellers and seller profile are new. Every Amazon plugin is priced at €0.00015 per delivered page; Google search is €0.0015.

Field coverage. 16 of the 18 return the complete field set the upstream response contains, so the preview’s itemsConfig candidates cover every field that is really there instead of a curated subset — tick the ones you want and the saved scraper delivers exactly those. Google Trends and Gemini are the two exceptions: their rows feed a chart view and a question-and-answer view, so they keep their fixed shape.

What to do with a candidate

The response is a list. Each entry’s scraperDTO is a ready-to-persist payload. To save the picked candidate as a real scraper:

curl -X PUT \
  -H "Authorization: Bearer $KEY" \
  -H "Idempotency-Key: $(uuidgen)" \
  -H "Content-Type: application/json" \
  -d "$(cat candidate-from-preview.json)" \
  "https://portal.scrapewise.ai/api/scraper-api/api/scraper"

That returns the persisted scraper with its id. Then trigger runs via GET /api/scraper/{id}/run.

When to use Preview vs a real scraper

Preview is the “test drive.” For production, save the picked candidate as a real scraper.

See also

• Scrapers — Create or update — save a picked candidate
• Schemas — for AI_CONF scrapers; preview won’t generate one, you bring your own
• Wallet & billing — what a preview costs, the 402 WALLET_INSUFFICIENT_BALANCE contract, top-ups
• Scrape prices — per-page prices, plus the legacy GET /api/plan-features catalogue that lists the API feature