Scrapers

Runs and errors

A run is one go of a scraper over its links. A run has a state (PENDING, RUNNING, COMPLETED, WARNING, FAILED, STOPPED), a start and end time, the number of good and failed links, and its cost. Each page that could not be read is listed with the reason. You only pay for pages that were delivered, so failed pages are not charged.

To run one scraper, press Run scrapper on its box. To run every scraper of a group, press Run group on the group box. Upload scrapers are skipped.
If the wallet is empty, Not enough balance for this run opens. Nothing was started and nothing was charged. Press Top up balance, or Not now.
Watch the box. It shows the state while the run goes. A box marked Broken – needs fix has a problem; open its run to see why.
To stop, press Stop scrapper on the box or Stop group on the group box. A waiting run ends at once. A running one ends after the page it is reading. Rows already saved are kept.
Open the group’s data page and pick History. Every run is a row with its state, time, links and cost. Open a row to see the links that failed and the reason.
To delete the data of a run, tick its row and press Delete Selected, or use the bin icon of one row. Confirm in Delete Scraper Job or Delete Selected Jobs. Deleting removes the rows. The run line stays in History, because it counts for billing. Switch on Show deleted jobs to see them.
A run that is still going cannot be deleted. The Cannot Delete Running Scrapper box says so. Press OK, stop the run, then delete.

Base address https://portal.scrapewise.ai/api/scraper-api, header Authorization: Bearer <key>. Which key: a Read key looks, a Full key also changes things, see Read or Full.
With a Full key, start a scraper: GET /api/scraper/{id}/run. The answer has the jobId. Add an Idempotency-Key so a retry does not start a second run. Starting a scraper that already runs gives 400 scraper_already_running.
With a Full key, start every scraper of a group: GET /api/scraper/group/{id}/run. The answer is 204. Add mergeData=true to merge the results into the group’s enriched copy when all runs end, see Groups.
With a Full key, follow a run live: GET /api/scraper-job-status/{id}/stream is a server-sent event stream with the progress.
With a Read key, list the runs: GET /api/scraper/load-history. Filter with groupId, scraperId, id or filters (for example {"state":"FAILED"}). Add withDeleted=true for deleted runs.
With a Read key, find the failed links of a run: GET /api/scraper/load-history/group/{groupId}/job/{jobId}/errors. An empty list means every link worked.
With a Full key, stop: GET /api/scraper/{id}/stop for one scraper or GET /api/scraper/group/{id}/stop for the group. Send an Idempotency-Key. Rows already saved are kept.
With a Full key, delete the rows of one run in two calls. Call 1 shows what goes (read it first). Call 2 deletes. Over REST, call 2 does not need the token that call 1 returns: you decide after reading the preview. An assistant over MCP passes the token to its delete tool itself.

bash

curl "https://portal.scrapewise.ai/api/scraper-api/api/scraper/[SCRAPER ID]/run" \
  -H "Authorization: Bearer $SCRAPEWISE_FULL_KEY" \
  -H "Idempotency-Key: [NEW UUID]"

bash

curl "https://portal.scrapewise.ai/api/scraper-api/api/scraper/load-history?scraperId=[SCRAPER ID]&size=5" \
  -H "Authorization: Bearer $SCRAPEWISE_READ_KEY"

bash

curl -X POST "https://portal.scrapewise.ai/api/scraper-api/api/scraper/data/preview-delete?scraperJobStatusId=[RUN ID]" \
  -H "Authorization: Bearer $SCRAPEWISE_FULL_KEY"

curl -X DELETE "https://portal.scrapewise.ai/api/scraper-api/api/scraper/data?scraperJobStatusId=[RUN ID]" \
  -H "Authorization: Bearer $SCRAPEWISE_FULL_KEY" \
  -H "Idempotency-Key: [NEW UUID]"

A history row carries id, scraperId, groupId, state, started, finished, successLinks, errorLinks, costMicros (1,000,000 is 1 EUR) and totalRequests. A run has no time limit. It ends when the list is done, the wallet is empty, or you stop it. A run that makes no progress for 30 minutes is stopped with stoppedReason: "NO_PROGRESS".

Connect your assistant with a Full key. See Connect Claude.
Ask it to run one scraper: scrapewise_run_scraper (Full). Run the first scrape of each scraper alone, then check the rows before you run more.
To run the whole group once all scrapers pass: scrapewise_run_scraper_group (Full).
Ask it to wait for the end and read the result: scrapewise_get_scraper_load_history (Read) with scraperId. It shows the state and costMicros.
Ask it why links failed: scrapewise_get_scraper_job_errors (Read) with groupId and jobId.
To stop: scrapewise_stop_scraper or scrapewise_stop_scraper_group (Full).
To delete the rows of a run: scrapewise_delete_scraper_data_preview (Full), show you the result, then after your yes scrapewise_delete_scraper_data (Full).

json

scrapewise_get_scraper_load_history
{ "scraperId": "[SCRAPER ID]", "size": 5 }

Errors you may see

402 WALLET_INSUFFICIENT_BALANCE

The wallet and the free requests are used up. Nothing was started. Top up, see Wallet and billing.

400 scraper_already_running

The scraper already has a run. Wait for it or stop it.

400 start_urls_unreachable

None of the start addresses answered. Check the address and the shop.

409 SCRAPER_RUN_ACTIVE

You tried to delete a run that is still going. Stop it first.

409 RUN_STILL_ENDING

The run was stopped and is still ending. Wait 5 seconds and try again. Nothing was deleted.

Status 503, 502 or Timeout on a link

The shop did not answer. The link is listed in the errors of the run and is not charged. Run the scraper again later.

Questions

Are failed pages charged?

No. Only pages that were delivered are charged. The run shows the number of good and failed links.

Why did my run give fewer rows than links?

Open the errors of the run. Typical causes are a removed product (404), a shop that blocked the page, or a page layout that changed. With AI control on, Scrapewise checks and repairs such scrapers, see AI control.

Does stopping delete what was already read?

No. Rows already saved are kept. Delete them with the delete steps above if you do not want them.

Can I run a scraper on a timer?

Yes, see Schedules.

Everything on this screen

Scraper box states and buttons:

Broken – needs fix: the box is marked broken when the last run found a problem with the page. Open Edit Automation to repair it.
Run scrapper, Stop scrapper and Upload new file: the round button on the scraper box changes with its state. It starts a run, stops one, or sends a new file for an upload scraper.
Run group and Stop group: the same on the group box, for all its scrapers.

Last updated 4 November 2026