Skip to content

Groups, rules and exports

Rules for all scrapers

A scraped price is rarely comparable as it is. One shop shows 9,99 €, another £8.50, a third 12.49 USD. One lists 500 ml, another 0.5 l. The cheaper number can be the more expensive product. Rules fix this after the scrape and before matching. You write them once on the group and they run on every scraper in it, also on uploaded files.

Four things usually need a rule:

  • One currency. Convert every price to EUR with the daily ECB rate. The rate and its date are written next to the price.
  • One unit. Divide the price by the pack size to get a price per 50 g, per 100 g, per litre or per piece.
  • One label. Map In stock, Available and 1 to one word.
  • One number. Take the pack size out of text such as 2 x 250 ml.

The rules add new columns. They never change what you scraped, unless you tick Replace the extracted value in on a rule (see the screen list below). When a rule cannot work, for example the pack size is missing, the cell stays empty and the row gets a marker. A rule never writes a value it had to guess.

  1. Open the 3-dots menu of the group and pick Rules for all scrapers. The dialog has three parts: Rules, Scrapers and Labels.
  2. In Rules press Add a rule and pick a kind.
  3. Convert currency: under Take the value from pick the price column. Pick Fixed currency and a From currency, or From a column and name the column that holds the currency code (Currency code is in this column). Pick the target in To, for example EUR. Name the new column in Save the result as, for example price_eur.
  4. Price per unit: pick the price column. Under Pack size is in this column pick the column with the pack size, for example pack_grams. Or use Fixed number and Divide by this fixed number. Type the size to compare at in Rescale to this size, for example 50. The buttons Cents → units (÷100), kg → g (×1000), oz → g and lb → g fill the numbers for common cases. Name the new column, for example unit_price.
  5. Take a number from text: reads a number and a unit out of text such as Ball 100 g. Use Add a unit or Edit as a list to say which unit words count.
  6. Replace values from a list: When scraped value is one thing, Save as another. Add another row for more. Use If no row matches, save this instead for the rest.
  7. Clean up text: type Which characters to keep? and pick Letter case.
  8. To test, type a real value in Paste or type a real value from your data (and a Sample pack size for a unit rule) and press Try it. Nothing is saved.
  9. Press Add to list. To change a rule later use Update rule. Keep editing and Close without saving protect unsaved work.
  10. Use Add a condition when a rule should run only on some rows, for example only where product_type is yarn. A rule can have up to 3 conditions and all must be true.
  11. When one calculation has several versions, put its rules in a Choice group with one Choice each, for example per 50 g, per 100 g and per piece. Add choice, Edit choice and Delete choice manage them. A row must fit exactly one choice.
  12. Press Save new rules to save the whole list.
  13. In Labels press Add label, type a Label name and give each scraper its value, for example product_type is yarn for one shop. Labels only steer conditions. They are not written into the rows.
  14. In Scrapers search with Search scrapers. see Problems and see Own rules filter the list. Show all problems opens every problem. Reload reads the rules again.

To run the current rules again on the rows of a finished run, open History on the group’s data page and press Re-apply rules on the run. Look at the numbers and press Re-apply. Stop re-apply rules ends it. Rows already changed keep their new values.

A price-compare layout that works

  • Two rules outside any choice: Convert currency price to price_eur and sale_price to sale_price_eur, both with the currency from the currency column.
  • One choice group basis, with one choice for each unit, for example per 50 g, per 100 g and per piece. Each choice writes the same columns: unit_price, sale_unit_price, unit_price_eur, sale_unit_price_eur and unit. The unit is a fixed word such as 50 g, written with Replace values from a list from the price column.
  • Each scraper gets a label, for example product_type is yarn, accessory or mixed. The conditions use it together with the pack size column.
  • Never scrape a column called unit, unit_price, sale_unit_price or any *_eur. A scraped column with a rule’s output name blocks the rule.

Note

The platform writes eur_rate and eur_rate_date on every converted row. Do not name a column that.

Errors you may see

409 CONFLICT on save
Someone changed the rules after you read them. Read again, merge, and save with the new version.
400 on save
A limit was broken or a rule is not valid, for example two rules write the same column outside one choice group. The message names the rule.
Rules for all scrapers are not available on this server yet.
The server is older than the portal. Try again later.
Row marker RULE_NO_BRANCH
No choice fitted the row, usually because the pack size is missing. The unit price stays empty. This is expected, not an error.
Row marker RULE_MULTI_BRANCH
Two choices fitted the row. Change the conditions so that they exclude each other.
Busy, try again on re-apply
Too many re-applies at once. Wait and try again.

Questions

Why on the group and not on each scraper?

With twelve scrapers you would copy the same rule twelve times and the copies drift. On the group it exists once. A scraper can still override one rule, and you can see which scrapers do.

Does a rule change the price I scraped?

No. A rule only adds columns. If the row already has a column with the rule’s output name, the rule is skipped and your column stays.

When do changes apply?

To runs and samples that start after you save. Use Re-apply rules to update the rows of a finished run.

Do rules run on uploaded files?

Yes, on a replace import and on the new rows of an add import. Rows that an add import only fills in get no rules.

Why is a price converted at a different rate than today's?

The rate belongs to the day the row was scraped. eur_rate_date shows which day.

Everything on this screen

The dialog:

  • The title reads Rules for all scrapers and the group name.
  • Rules, Scrapers and Labels: the three parts. Each shows a count, for example Rules (3).
  • Save: saves the list. It says Saving while it works. close leaves the dialog.
  • Rules run when the scraper runs. Click a chip on the left to edit a rule, or add more on the right.

Inside a rule:

  • Remove this row: deletes one row of a Replace values from a list rule.
  • Condition 1 value: the value a column must have for the rule to run. Each condition has a column box, an operator box (is, is not and so on) and this value. Remove condition 1 deletes the condition.
  • Unit word 1: a unit word to look for in the text, for example g or kg. The number next to it says how many base units one word is worth. Remove unit 1 deletes it.
  • Replace the extracted value in: tick it when the new value is more reliable than what the page gave. The old value in that column is then replaced.

Choice groups:

  • Choice name: the name of one choice, for example per 50 g.

Labels part:

  • Rename label opens a New name for label box. Save name of label keeps the new name. Delete label removes it.

Scrapers part:

  • Details: shows more about a scraper and its rules or problems.
  • The filter chips above the list switch between all scrapers, scrapers with problems and scrapers with their own rules.
  • Rows per page, Prev and Page 1: move through a long list.

Last updated 4 November 2026