Scrapers

Upload a file

An upload scraper gets its data from a file you send, not from a website. Scrapewise reads the file and writes every row into the group, like the rows of a normal run. The rows then work like any scraped data: you can see them, export them and use them in matching.

An upload scraper never fetches a page and is never scheduled. Run group skips it. It only gets new data when you send a new file. Uploading is free and works with an empty wallet.

On the home page open the 3-dots menu of the group box and pick Upload.
Type a Scraper name.
Press Choose file and pick a .csv, .xlsx or .jsonl file. Press Upload.
A new scraper box appears under the group. It shows Running while the file is read, then Completed.
To send a new file later, press the Upload new file button on the box. The same dialog opens.
If the file is used in matching, pick how the file is applied. Add rows adds new rows and fills empty cells, and never changes what is there. Sync whole catalogue treats the file as the full list: changed rows are updated and rows missing from the file are set inactive.
If the Key column list shows, pick the column that is unique for every row, for example the EAN.
If Scrapewise warns that many rows would become inactive, check the file. Press Apply anyway only when the file is right.

An upload scraper keeps only the latest file by default. When a new file is read in full, the rows of the old file are removed. Change this in Data retention, see Groups.

Base address https://portal.scrapewise.ai/api/scraper-api, header Authorization: Bearer <key>. Which key: a Read key looks, a Full key also changes things, see Read or Full. Use a Full key for every call on this tab.
Optional: check first (Full key). Send the same call as in step 3 with check=true and an empty body. The answer 204 means the upload would be accepted. A check makes nothing.
With a Full key, make the scraper and import the first file: POST /api/scraper/group/{groupId}/file-scraper?name=...&format=csv. The file is the request body. Send it in one request.
To send a new file to the same scraper (Full key): POST /api/scraper/{scraperId}/file?format=csv.
Follow the import. The answer has a jobId. Read it with GET /api/scraper/load-history or the job stream, see Runs and errors.
Read the rows like other data, see Exports and delivery.

bash

curl -X POST -T 'products.csv' \
  -H 'Authorization: Bearer <YOUR_FULL_KEY>' \
  -H 'Content-Type: text/csv' \
  -H 'Expect:' \
  'https://portal.scrapewise.ai/api/scraper-api/api/scraper/group/[GROUP ID]/file-scraper?name=Customer+feed&format=csv'

bash

curl -X POST -T 'products.jsonl' \
  -H 'Authorization: Bearer <YOUR_FULL_KEY>' \
  -H 'Content-Type: application/x-ndjson' \
  -H 'Expect:' \
  'https://portal.scrapewise.ai/api/scraper-api/api/scraper/group/[GROUP ID]/file-scraper?name=Customer+feed&format=jsonl'

bash

curl -X POST -T 'products.xlsx' \
  -H 'Authorization: Bearer <YOUR_FULL_KEY>' \
  -H 'Content-Type: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet' \
  -H 'Expect:' \
  'https://portal.scrapewise.ai/api/scraper-api/api/scraper/group/[GROUP ID]/file-scraper?name=Customer+feed&format=xlsx'

Use -T, not --data-binary, so the file streams from disk. Keep the empty Expect: header. On Windows use curl.exe and double quotes.

`format`	`Content-Type`	Limit
`csv`	`text/csv`	200 MB, comma or semicolon, UTF-8
`jsonl`	`application/x-ndjson`	200 MB, one JSON object per line
`xlsx`	`application/vnd.openxmlformats-officedocument.spreadsheetml.sheet`	36 MB, first sheet only. For a bigger Excel file save it as CSV

Read your account’s limits first with GET /api/scraper/config/parameters (Read). The limits can be lower than the table.

The first row gives the column names. All values are stored as text, so an EAN keeps its leading zeros.
At most 500 columns, a name of at most 256 characters, a cell of at most 32 KB, a row of at most 1 MB.
A small file (up to 2 MB) can go as JSON: { "name": "...", "format": "csv", "content": "ean;name\n0012;Mug\n" }. For xlsx the content is base64.

When the file is used in matching, the scraper is protected. A new file never deletes anything. Rows are matched by a key column. A new key adds a row. A known key fills empty cells and adds new columns, and never changes a value. Send keyColumn when the first protected upload cannot find one. Add mode=sync to treat the file as the whole catalogue: rows missing from the file become inactive and come back when the key returns. If more than 20 percent of the rows would drop, the import ends FAILED with SYNC_DROP_GUARD. Send the same file again with confirmDrop=true if the file is right. PUT /api/scraper/{scraperId}/file-sync-mode with { "fileSyncMode": true } makes sync the default for that scraper.

A file import is all or nothing. It ends COMPLETED with every row stored, or FAILED and the previous file stays the latest data. A file without data rows fails with “The file has no data rows”.

Connect your assistant with a Full key. See Connect Claude.
Ask it to upload a small file: scrapewise_create_file_scraper (Full) with groupId, a new idempotency_key and a body that holds name, format and content. The file goes as text, so use this for files up to 2 MB. For a bigger file use the API tab.
To send a new file to the same scraper: scrapewise_update_file_scraper_file (Full) with scraperId and a body that holds format and content. Add mode (sync or add), confirmDrop and keyColumn when the file is used in matching.
Ask it to follow the import: scrapewise_get_scraper_load_history (Read), see Runs and errors.

json

scrapewise_create_file_scraper
{
  "groupId": "[GROUP ID]",
  "idempotency_key": "[NEW UUID]",
  "body": {
    "name": "Customer feed",
    "format": "csv",
    "content": "ean;name;price\n0012345678905;Blue mug;4.99\n"
  }
}

Errors you may see

409 NAME_TAKEN

The group already has a scraper with this name. The answer names it. Send the file to that scraper instead.

409 ALREADY_RUNNING

This scraper is still reading a file. Wait, or press Stop on the box, wait a minute and upload again.

400 KEY_COLUMN_REQUIRED

The file is used in matching and no key column is known. Send keyColumn, or pick Key column in the portal.

400 BAD_FORMAT

format is missing, does not match the Content-Type, or the base64 is bad.

413

The file is over the size limit, or has no Content-Length. Use curl -T. For Excel over 36 MB save as CSV.

408

The upload was too slow or took too long. Send it again from a faster connection.

429 INGEST_CAPACITY

Too many uploads at once. Wait a moment and send again.

FAILED with SYNC_KEY_INVALID, SYNC_DROP_GUARD or SYNC_MATCHER_BUSY

A sync needs a key on every row, each key once. See the sync part above. Nothing was changed.

Questions

Is uploading free?

Yes. An import costs nothing and works with an empty wallet.

Does matching run again after an upload?

No. Run matching again to refresh the matched results, see Run a match.

Can I upload my product list and my competitors in one file?

Keep them apart. Upload your own list as one upload scraper and keep each shop as its own scraper, so the group rules can treat them the same.

Should I clean the file by hand first?

No. Upload the raw file. A value such as a percent sign can change when it is cleaned by hand. The group rules clean uploaded rows too.

Everything on this screen

Upload dialog:

The message “Keeps the latest file only” tells you that old files are dropped when a new one is read. Change it in Data retention, see Groups.
The file box behind Choose file takes the file you pick.

Upload window:

Upload: Sends the file. The button shows Uploading while the file goes.
The file box behind Choose file.
Scraper name: The name of the upload scraper.
Add rows: Adds new rows and fills empty cells.
Sync whole catalogue: Treats the file as the full list.
Key column: The column that is unique for every row.
Choose file: Picks the file to send.
Apply anyway: Applies the file although many rows would become inactive.

Last updated 4 November 2026