Groups, rules and exports
Groups
A group holds the scrapers that belong together. Every scraper is in exactly one group. The rows of all its scrapers land in one data table. A group also carries what the scrapers share: the schedule, the data retention, the rules for all scrapers, AI control, the delivery webhook and the matching setup.
Make one group for each job. For a competitor price comparison that is one group with one scraper for each shop and one upload scraper for your own list.
- On the home page press Add Group.
- Type a Name. A Description is optional. Press Save.
- Press + on the new group box to add a scraper, see Read a website. Use Upload in the group menu to add a file, see Upload a file.
- To change the name later, open the 3-dots menu of the group and pick Edit info.
- To see the rows, pick Data in the same menu. The group page opens on Table Data. The other tabs are Matching, History and Match quality.
- To keep the data small, pick Data retention in the group menu. Leave No limit (keep every run) on to keep all runs, or switch it off and type Runs to keep. Each scraper then keeps only its newest runs. Older runs lose their rows after the next run, and the run line stays in History. The group value wins over a scraper’s own value.
- To tidy a big canvas, pick Move to desktop and choose a desktop or Create new desktop with a New desktop name, then press Move. A desktop is a tab on the home page. Use its menu to Rename or delete it (Delete desktop).
- To delete a group, open its 3-dots menu and pick Delete. The Delete the scraper Group dialog lists what goes. The scrapers inside go with it. Switch on Delete associated data? only if the rows should go too. Without it the rows are kept.
- To share the group with another person, use Share on the group page, see Shared groups.
Merge in the group menu joins data of several scrapers of the group on a key column into one enriched table. The result is a separate table next to the group data. A saved recipe is drawn on the match canvas.
- Base address
https://portal.scrapewise.ai/api/scraper-api, headerAuthorization: Bearer <key>. Which key: a Read key looks, a Full key also changes things, see Read or Full. - With a Read key, list your groups:
GET /api/scraper/group/list. Groups shared with you are inGET /api/scraper/shared/group/list. - With a Full key, make a group:
PUT /api/scraper/groupwithname(anddescription) and a newIdempotency-Key. Sendidto change an existing group. - With a Full key, keep fewer runs:
PUT /api/scraper/group/{id}/retentionwith{ "retentionCount": 3 }, ornullto clear. For one scraper usePUT /api/scraper/{id}/retention. The group value wins. - With a Full key, set the currency the Analytics page shows prices in:
PUT /api/scraper/group/{id}/currencywith{ "displayCurrency": "EUR" }, ornullto clear. - With a Full key, move a group to a desktop:
PUT /api/scraper/group/{id}/desktop?desktopId=.... LeavedesktopIdout for the default desktop. Make a desktop withPUT /api/scraper/desktop(name).GET /api/scraper/desktop/listlists them (Read).PUT /api/scraper/desktop/reordertakes the ids in the new order.DELETE /api/scraper/desktop/{id}removes one. - With a Full key, delete a group in two calls, see below.
curl -X PUT "https://portal.scrapewise.ai/api/scraper-api/api/scraper/group" \
-H "Authorization: Bearer $SCRAPEWISE_FULL_KEY" \
-H "Idempotency-Key: [NEW UUID]" \
-H "Content-Type: application/json" \
-d '{ "name": "competitor-prices", "description": "Yarn shops, weekly" }'
Delete a group in two calls. Call 1 shows what will go (read it first). Call 2 deletes. Over REST, call 2 does not need the token that call 1 returns. An assistant over MCP passes the token to its delete tool itself. Without withData=true the rows are kept. With it the whole data table is dropped and cannot be brought back.
curl -X POST "https://portal.scrapewise.ai/api/scraper-api/api/scraper/group/[GROUP ID]/preview-delete?withData=false" \
-H "Authorization: Bearer $SCRAPEWISE_FULL_KEY"
curl -X DELETE "https://portal.scrapewise.ai/api/scraper-api/api/scraper/group/[GROUP ID]?withData=false" \
-H "Authorization: Bearer $SCRAPEWISE_FULL_KEY" \
-H "Idempotency-Key: [NEW UUID]"
Group rules go with the group when it is deleted.
Merge scrapers. With a Full key, PUT /api/scraper/data/group/{id}/merge joins the latest completed run of several scrapers into one enriched table. Send a recipe with primaryScraperId, nodes (the scrapers) and edges (which column of one scraper joins which column of another). GET /api/scraper/data/group/{id}/enrichable-scrapers (Read) lists the scrapers with their latest run. GET /api/scraper/data/group/{id}/enrichment (Read) shows where the enriched table came from, and DELETE /api/scraper/data/group/{id}/enrichment (Full) removes the saved recipe. The answer of a merge has mergedItems, primaryRows, droppedEmptyKeys and matched.
Columns of a run. To pick the join columns, read the column names of a run first. GET /api/scraper/data/group/{id}/job/{jobId}/columns (Read) gives the sorted names of one run, url and number columns included. POST /api/scraper/data/group/{id}/columns/batch (Read) does it for up to 200 runs of one group in one call: send { "jobIds": ["..."] }. Ids it cannot read come back in omittedIds, and you can ask for them one by one.
Data quality. PUT /api/scraper/group/{id}/data-quality (Full) switches single behaviours of the engine on or off for this group, for example one saved row per product page. Each field is true, false or null to follow the global setting. The portal shows this to Scrapewise staff only.
- Connect your assistant with a Full key. See Connect Claude.
- Before it makes a group, ask it to list the groups:
scrapewise_get_scraper_group_list(Read). Reuse a group that already fits. - Ask it to make one:
scrapewise_create_scraper_group(Full) with a newidempotency_keyand abodywithname. - To keep fewer runs:
scrapewise_update_scraper_group_retentionorscrapewise_update_scraper_retention(Full) withretentionCount. - To set the display currency:
scrapewise_update_scraper_group_currency(Full). - To merge scrapers:
scrapewise_list_enrichable_scrapers(Read), thenscrapewise_run_scraper_data_group(Full) with the recipe.scrapewise_get_enrichment(Read) andscrapewise_delete_enrichment(Full) read and remove it. To pick the join columns first,scrapewise_get_run_columns(Read) lists the columns of one run andscrapewise_get_run_columns_batch(Read) lists them for up to 200 runs. - To delete a group:
scrapewise_delete_scraper_group_preview(Full), show you the result, and only after your yesscrapewise_delete_scraper_group(Full).
scrapewise_create_scraper_group
{ "idempotency_key": "[NEW UUID]", "body": { "name": "competitor-prices", "description": "Yarn shops, weekly" } }
scrapewise_update_scraper_group_data_quality (Full) sets the data quality overrides described on the API tab.
Errors you may see
Questions
How many scrapers should a group have?
As many as you have shops, one scraper for each shop and data source. There is no limit on shops in a group. Different product kinds are labels and rules on the group, not extra scrapers.
Does deleting a group delete my data?
Not by default. The rows are kept unless you switch on Delete associated data? or send withData=true.
Where is the group schedule?
See Schedules. Data retention is separate from the schedule.
What happens to old runs when I set Runs to keep?
After the next run, older runs lose their rows. The run line stays in History because it counts for billing.
Everything on this screen
Home page, group boxes and desktops:
- Empty home page: the hint says to make a group first, then add automations under it with the + button.
- Add desktop: the + at the end of the desktop tabs. It asks for a name and makes a new desktop. Rename desktop uses the same dialog to change a name.
- Desktop tab menu: the small menu on a desktop tab. It holds Rename and Delete desktop.
- Move group to desktop: the dialog behind Move to desktop. See the steps above.
-
- on a group box: adds a scraper to the group. In the add dialog, Add saves it.
- 3-dots menu on a group box: Schedule, Data retention, Rules for all scrapers, AI control, Data, Delivery status, Edit info, Move to desktop, Upload, Find emails, Merge and Delete.
- 3-dots menu on a scraper box: Schedule, Data retention, Edit Automation, View Automation, Manage Links, Find emails, Enable or Disable, and Delete.
- Edit Automation: change how the scraper reads its pages. View Automation: look at it without changing anything.
- A row in a list: click it to open it.
- Plan Limit Reached: shows when your plan has no room for one more group or scraper. Upgrade Plan opens the plans.
Several scrapers at once:
- Select multiple: lets you tick scrapers. Exit selection mode ends it.
- Delete All: deletes every ticked scraper. Confirm Bulk Delete asks once more and shows what goes. Delete associated data? decides whether the rows go too.
- Delete the Scrapper: the same check for one scraper.
Switching a scraper on or off:
- Enable Scraper and Disable Scraper: the dialog that asks before you switch it. Enable and Disable confirm.
Data retention messages:
- When a scraper’s retention belongs to its group, its dialog says so and tells you to change it in Data retention of the group.
- The group dialog explains that each scraper keeps only its newest runs and that older runs lose their rows.
Enrichment and run history:
- Enrich Data: with runs ticked, joins them to your list. The number is how many runs are ticked. It reads Enriching while it works.
- Show deleted jobs: also lists jobs you deleted.
- Delete jobs: tick jobs, press the bin icon, then confirm. The button counts the jobs. Cancel closes the window.
- Clear enrichment? and Clear: removes the saved enrichment of this group after you confirm.
Last updated 4 November 2026