USE CASE

Product Data Cleaning: From Messy Shop Rows to Comparable Prices

Every shop writes prices, sizes and stock in its own way. Scrapewise group rules turn them into one clean format: the price as a number, the pack size in grams or millilitres, price per kg or per litre, prices in EUR, and one row per product page. Then the rows are ready to match and compare.

THE PROBLEM

Why Scraped Product Data Is Not Ready to Compare

  1. 01

    Prices are text, not numbers

    "1,29 €", "EUR 1.29" and "1.29 EUR/tk" are the same price written three ways. A spreadsheet sorts them as text, and a chart cannot read them at all.

  2. 02

    Pack sizes hide the real price

    A 500 g bag at EUR 1.29 is cheaper per kilo than a 1 kg bag at EUR 2.79. Without price per kg the comparison rewards the smallest pack.

  3. 03

    Shops sell in different currencies

    A Polish, Swedish or Swiss shop shows its own currency. Until every price is in EUR, the cheapest offer is a guess.

  4. 04

    Duplicates and noise in titles

    The same product page shows up twice in a link list, and titles carry "-20%!" or "NEW" that break matching. Both need cleaning before any comparison.

5

Rule types: number from text, price per unit, currency, map values, clean text

3

Conditions per rule, so one group can hold groceries, drinks and multipacks

5

Free requests on every new account, no card needed

Where Cleaning Fits: Scrape, Clean, Match, Ready Analysis

Cleaning is step 2 of 4. It runs on every scraped row, so the matcher and your price report work on the same clean columns from every shop.

  1. Scrape

    Collect the shops you compare against with a ready data API or an AI scraper for any site. Use a category, a link list, a sitemap or an uploaded file, on demand or on a schedule.

  2. Clean

    Group rules run on the rows of every scraper in the group: take the number out of a text, compute price per kg, litre or piece, convert prices to EUR, map stock words to yes or no, and trim noise from titles.

  3. Match

    The clean rows are linked to your own product list: same EAN or ID first, then text columns such as title, brand and size, then product pictures. Unsure pairs wait for review.

  4. Ready competitor analysis

    Your product next to each competitor, with the same price per unit and the same currency. Export as CSV or Excel, read it by REST API, or let an AI agent read it over MCP.

The Five Cleaning Rules, in Plain Words

Group rules write new clean columns next to the scraped ones. The scraped value is never overwritten, so you can always check where a clean value came from.

Take a number from text

Reads the number written right before a unit and converts it: "Oat flakes 0.5 kg" gives 500 grams, "Juice 1 l" gives 1000 ml. Ready unit lists cover grams, kilos, ounces and pounds, millilitres to litres, and pieces.

  • pack_g
  • pack_ml
  • pieces

Price per unit

Divides the price by the pack size and scales it to the unit you compare on: per kg, per 100 g, per 50 g, per litre or per piece. You pick the reference number, so one rule fits any product type.

  • price_per_kg
  • price_per_100g
  • price_per_l

Currency conversion

Converts prices in other currencies into one currency, such as EUR, with current exchange rates, so offers from shops in different countries compare on one scale.

  • price_eur

Map values

Turns shop-specific words into one value: "Laos", "In stock" and "Saadaval" become true, "Out of stock" becomes false. Useful for stock, condition and category names.

  • in_stock
  • condition

Clean text with a pattern

Keeps only the part of a text you need, for example the title without "-20%!" or the model code from a long product name, so titles are ready for matching.

  • title_clean
  • model

Clean Data Means No Guessing

A wrong unit price is worse than an empty cell, because it looks real in every report. The rules are built to leave a cell empty rather than guess.

  • Unclear sizes are skipped

    Multipacks like "3 x 100 g", ranges like "50-100 g" and unclear numbers like "1.000 g" give no value instead of a wrong one. An empty cell is easy to filter for; a wrong number is not.

  • Rules can depend on the product type

    Each group rule can have up to 3 conditions, such as "product type is yarn" or "pack size is not empty". So yarn is compared per 50 g and coffee per kg in the same group.

  • One row per product page

    If the same product page appears twice in one run, it is kept once and the dropped address is reported, so a price is never counted twice. The same product in different shops is linked by matching.

What Goes In, What Comes Out

Any scraped or uploaded rows go in. Clean, typed columns come out, next to the original values.

You give
  • Scraped or uploaded rowsrequired

    Rows from any scraper in the group, or a CSV / Excel file you upload as a source.

    KAERAHELBED 500G -20%!, 1,29 €, Laos
  • Compare perrequired
    • per kg
    • per litre
    • per 100 g
    • per piece

    The reference unit for the price per unit column.

  • One currencyoptional

    Prices in other currencies are converted to it.

    EUR
You get

One clean row per product page, 8 columns each

  • title
  • title_clean
  • price
  • pack_g
  • price_per_kg
  • price_eur
  • in_stock
  • url

CSV or Excel export, REST API, or MCP for AI agents.

Before and After: One Grocery Row, Cleaned

Example rows in the style of a grocery price comparison (illustrative data, not a customer's). The left column is what the shop shows; the right is what the group rules write.

ColumnAs scraped (before)After cleaningRule
Price1,29 €1.29Take a number from text
Pack sizeKAERAHELBED 500G -20%!500 (grams)Take a number from text, grams
Price per kgnot shown by the shop2.58Price per unit: 1.29 ÷ 500 × 1000
Price in EURPLN 5,49 (shop in Poland)EUR value at the current rateCurrency conversion
In stockLaostrueMap values
TitleKAERAHELBED 500G -20%!KAERAHELBED 500GClean text with a pattern
Multipack3 x 100 gleft empty, not guessedNo guessing
Same page twice2 rows, same address1 row, the drop is reportedOne row per product page
RESULT

What Clean Data Gives You

Fair price comparisons

Fair price comparisons

Price per kg, per litre or per piece across every pack size, so the cheapest offer is really the cheapest.

One currency

One currency

Offers from shops in different countries in one currency, ready for one chart or one report.

Rows ready to match

Rows ready to match

Clean titles, sizes and numbers make matching to your product list more reliable, and one row per product page keeps counts honest.

THE SHORT ANSWER

Product Data Cleaning: The Short Answers

How do I clean messy product data?

How do I clean messy product data?

Write the cleaning once as group rules instead of fixing cells by hand. In Scrapewise a rule takes the number out of a price text, reads the pack size, computes price per unit, converts the currency and trims title noise. The same rules run on every shop and every new run.

How do I normalise product data from different shops?

How do I normalise product data from different shops?

Put the shops in one group and add rules that write the same clean columns for all of them: price, pack size, price per unit, currency and stock. Each scraped value stays next to its clean copy, so every shop ends up in one format you can compare and check.

How do I compare prices across pack sizes?

How do I compare prices across pack sizes?

Compare price per unit, not the shelf price. First take the pack size out of the title, for example 500 g, then divide the price by it and scale to per kg, per litre or per piece. Unclear sizes such as multipacks are left empty instead of guessed.

How do I remove duplicate products?

How do I remove duplicate products?

There are two kinds. The same product page twice in one run is kept once, and the dropped address is reported. The same product sold by different shops is not a duplicate to delete: product matching links those rows to one product in your own list.

Turn Your Competitor Rows Into Comparable Prices

Scrape one shop, add a price per kg rule, and see the clean column next to the raw one. Every new account gets 5 free requests; after that you pay as you go from a prepaid balance. bebo.ee, a grocery price comparison, runs on the same scrape, clean per kg and match flow. bebo.ee compares 12 grocery e-shops: more than 130,000 products, and about 36,000 of them are matched across shops so their prices can be compared.

FAQ

Product Data Cleaning FAQs

What the cleaning rules do, what they refuse to do, and how they fit with matching.

Product data cleaning turns the text a shop shows into values you can compute with: a price as a number, a pack size in grams or millilitres, a price per kg or per litre, one currency and one word for stock. In Scrapewise you set it up once as group rules, and every scraper in the group follows them on every run.