Groups, rules and exports
AI control
Websites change. A scraper that worked last week can return empty or wrong rows today. AI control watches the scrapers of one group for you. You switch it on for each group.
When it is on, AI looks at the newest run of the scrapers in the group on the rhythm you choose. It looks for failed runs, empty runs and values that went missing. When it finds a problem it proposes a fix and tests the fix on live pages before it saves it. It undoes a fix when the next real run is worse.
What it can repair:
- Scrapers that read the page with selectors, JSON-LD scrapers and API scrapers. Each kind of repair is switched on by Scrapewise, one by one.
- AI-schema scrapers are checked and you get an alert. They are not repaired yet.
- Supplier endpoints and uploaded files are not checked.
AI control is in a test phase. Some problems are only recorded for now. Each check looks at up to a set number of scrapers, the ones checked longest ago first, and the rest come next time. The number is shown in the dialog. Switching on needs your first wallet top-up. Switching off is always allowed.
- Open the 3-dots menu of the group and pick AI control.
- Switch on Let AI check and fix the scrapers in this group.
- Pick How often: Every day, Twice a week or Once a week. The first choice is Once a week.
- Press Save. The dialog says “Saved.” The first check usually starts within a few minutes.
- Read the status under the switch. It shows where the check stands, when the next check is, and the result of the last one. Up to N scrapers per check shows how many scrapers one check covers.
- Press Refresh to read the status again. Close leaves the dialog.
- If it says “AI control starts after your first wallet top-up.”, press Top up wallet, then save again.
To switch it off, switch off the first toggle and press Save.
- Base address
https://portal.scrapewise.ai/api/scraper-api, headerAuthorization: Bearer <key>. Which key: a Read key looks, a Full key also changes things, see Read or Full. - With a Full key, switch it on or off:
PUT /api/scraper/group/{id}/ai-controlwithenabledandcadence, and a newIdempotency-Key.cadenceisDAILY,TWICE_WEEKLYorWEEKLY. Leave it out to keep the current choice. - With a Read key, read the status:
GET /api/scraper/group/{id}/ai-control/status. - Do not ask again and again (a Read key is enough to read the status). The answer gives the next check time in
nextCheckAt.
curl -X PUT "https://portal.scrapewise.ai/api/scraper-api/api/scraper/group/[GROUP ID]/ai-control" \
-H "Authorization: Bearer $SCRAPEWISE_FULL_KEY" \
-H "Idempotency-Key: [NEW UUID]" \
-H "Content-Type: application/json" \
-d '{ "enabled": true, "cadence": "WEEKLY" }'
curl "https://portal.scrapewise.ai/api/scraper-api/api/scraper/group/[GROUP ID]/ai-control/status" \
-H "Authorization: Bearer $SCRAPEWISE_READ_KEY"
The status answer has enabled, cadence, phase, nextCheckAt, lastCheckAt, lastResultAt, nothingNewSinceResult, results, checkedCount and scrapersPerCheck.
phase | Meaning |
|---|---|
OFF | AI control is off for this group |
PAUSED | Checks are paused on our side. Your choice is kept and checks go on by themselves |
RUNNING | A check is running now |
QUEUED | A first check is queued and usually starts within minutes |
DUE | The next check is due and starts soon |
SCHEDULED | The next check is planned at nextCheckAt |
Each line in results has a kind and a count of scrapers. Read kind and ignore reasons, which are internal codes.
kind | Meaning |
|---|---|
NOTHING_TO_REPAIR | Checked, nothing to repair |
REPAIR_APPLIED | A repair was made. The next real run confirms it |
REPAIRED | A repair was made and the next real run confirmed it |
REPAIR_UNDONE | The repair was undone because the next real run was worse |
REPAIR_KEPT_WORSE | The repair was kept, but the next run looked worse |
FIX_NOT_SAFE | A fix did not pass the safety test, so nothing was changed |
REPAIR_PAUSED | A problem keeps coming back, so repairs are paused for that scraper |
PROBLEM_NOT_FIXABLE | AI cannot fix this kind of scraper yet |
PROBLEM_RECORDED | A problem was recorded and is not repaired in the test phase |
BLOCKED | The scraper could not be checked this time. It is tried again next check |
ACCEPTED_CHANGE | A column stayed empty and is now treated as normal |
IN_PROGRESS | A repair is being made |
CHECK_FAILED | The check could not finish. It is tried again next time |
A scraper is BLOCKED when the daily limit for AI checks was reached or the AI service was busy (this is not your wallet), when the scraper was busy or changed at the same time, when the website failed, when the test gave no clear answer, or when that kind of repair is switched off.
- Connect your assistant with a Full key to change AI control, or a Read key to read it. See Connect Claude.
- Ask it to switch it on:
scrapewise_update_scraper_group_ai_control(Full) withid, a newidempotency_keyand abodywithenabledandcadence. - Ask it for the status:
scrapewise_get_scraper_group_ai_control_status(Read). It tells you the phase, the next check and the last result.
scrapewise_update_scraper_group_ai_control
{ "id": "[GROUP ID]", "idempotency_key": "[NEW UUID]", "body": { "enabled": true, "cadence": "WEEKLY" } }
Errors you may see
402. Switching off is never refused.cadence is not one of the three values.Questions
Why does it ask for a top-up?
AI control starts after your first wallet top-up. Switching it off is always allowed.
Will it change my scraper without telling me?
A repair is tested first and the next real run confirms it. A repair that makes the next run worse is undone. The result shows in the status.
Why are some of my scrapers not checked?
One check covers a set number of scrapers, the ones checked longest ago first. Uploaded files and supplier endpoints are not checked.
Is it the same as a schedule?
No. A schedule runs the scrapers. AI control checks the runs and repairs broken scrapers.
Everything on this screen
AI control dialog:
- The line at the top says: when on, AI checks this group’s scrapers on the schedule you pick and fixes broken selectors. The switch, How often, Save, Refresh, Close and Top up wallet are described in the steps above.
Last updated 4 November 2026