What data is really needed?

How to conduct a successful pricing POC?

Short answer: five data categories, three levels of requirements

The dataset for a pricing proof of concept (POC) consists of all the data extracts—documented and validated by the retailer—that enable a solution to reconstruct past business activity within a given scope and calculate pricing recommendations for that scope. It comprises five categories: the product catalog, sales, inventory, purchasing terms, and the history of prices and promotions. Market data (competitor analysis, market share, web traffic) is used to enrich the dataset.

For each family, there are three distinct levels. What is essential at the outset—without which the test cannot reach a conclusion. What is useful and improves the quality of the recommendations without being a prerequisite for launch. What can wait until deployment. Confusing these levels leads to the two symmetrical failures of pricing projects: waiting for perfect data that never arrives, or launching a test based on data that no one has verified. This article addresses each category in order, then describes the consistency checks that validate the entire set.

Essential, Useful, Superfluous: The Complete List

The table below is the checklist we use to review a brand's samples before a test. It is intentionally strict in the first column and lenient in the last.

This checklist complements—rather than replaces—our article on ensuring data reliability before implementing a pricing tool, which explains why data quality determines a project’s success. Here, the focus is narrower: what data is needed, at what level of detail, and how to verify that it is accurate.

The product catalog: the key to everything else

Every other data set is linked to a product. An incomplete or unstable master data repository therefore affects sales, inventory, and prices simultaneously. The minimum requirements are a unique, stable identifier, a description, an EAN code, a multi-level product hierarchy (universe, family, subfamily), the brand, and a status that distinguishes between active products, new products, and end-of-life products.

The Case of New Products

This is the most common gap in pricing models—and the most costly one. A product launched six weeks ago has no sales history, and therefore no measurable price elasticity. A pricing solution can still handle it, provided it has attributes that link it to known products: format, size, product line, brand, and, if possible, the product it replaces. These attributes are not listed in the “essential” column, but a retailer whose product assortment changes rapidly (high-tech, fashion, baby products) must treat them as such. A proof of concept (POC) for these categories without “new product” attributes will only test the products that are always in stock.

Duplicates and code changes

If the same product exists under two different identifiers, or if its code has changed during the year, this splits its history in two and skews any analysis. The process of deduplicating and chaining successive codes is described in our article on improving the reliability of the internal product catalog; for a proof of concept (POC), it is sufficient to document known cases and provide the mapping table when it exists.

Sales: Mesh Size and Depth

The minimum data unit is the quantity sold and sales revenue per product, per day, per retail location, and per channel. Monthly or national-level aggregation does not allow for measuring price elasticity, creating a control group, or distinguishing between a downtown store and a suburban store. The receipt line item provides valuable additional information (it reveals products purchased together and checkout discounts) but is not required to get started.

The expected time horizon is 24 months, to account for two full seasons and two occurrences of each peak period. Twelve months allow for testing on categories with little seasonal variation. Our article on the sales history required for accurate forecasting details this issue of time horizon by category type.

Promotional sales must be clearly identifiable

A sale made during a promotional campaign does not represent the same thing as a sale at the regular price. If the sales data does not include a promotional indicator, the promotional calendar (next section) can be used to reconstruct it, provided that the data is specific to the product and the store. Without either of these, the calculated elasticities will be skewed by promotional volumes attributed to the regular price.

Supply shortages mask actual demand

A product that is out of stock does not sell, and its sales history shows “zero” where demand actually existed. This bias, known as censored demand, distorts forecasts and underestimates the effect of price. It is corrected using inventory data (the next family), which explains why the two families go together. Our article on the link between stockouts and sales forecasting describes how this works.

This lack of confidence is not inevitable, but it has a practical implication for a POC: until the retailer has verified the data extracts itself, it cannot distinguish between a flaw in the solution being tested and a flaw in its own data. The checks described below serve this purpose.

Inventory: The Most Difficult Category to Restock

Management systems track current inventory but rarely maintain its history: a product’s inventory level in a warehouse fourteen months ago is not stored anywhere, unless a regular snapshot has been archived. This is the most common shortcoming encountered when preparing a POC, and one of the reasons people mistakenly give up.

There are two options. If weekly or monthly snapshots have been archived (in a data warehouse, in management control reports, or in a replenishment tool), those are sufficient. Otherwise, historical inventory is reconstructed retroactively based on current inventory and transactions (receipts, sales, returns, inventory counts, markdowns), with accuracy decreasing as you go further back in time. A 12-month reconstruction is generally sufficient for a proof of concept (POC); it allows you to identify stockouts, correct censored demand, and test markdown functions, the entire logic of which is based on remaining inventory and the sales rate.

What can wait: daily inventory, inventory in transit, and reservations. What cannot wait: knowing, for each product and each store, whether it was available for sale each week during the period.

Purchase terms and valuation: a decision to be made in consultation with the finance department

Without purchase costs, there is no margin; and without a margin, there is no pricing. The minimum data requirement is a net purchase price per product and per period—at least quarterly—that takes into account invoice discounts. Conditional and year-end discounts, logistics costs, and back-end margins are additional details that refine the margin but do not prevent a test from running, provided that their absence is documented and consistent across the entire scope.

Weighted average purchase price, last price, or standard price?

This is the most hotly debated issue among pricing, procurement, and management control teams, and it must be resolved before the POC, not during it. The weighted average purchase price (WAP) smooths out price increases and reflects the actual value of the inventory sold; it is generally the figure used in accounting. The most recent purchase price reflects the replacement cost and carries more weight in pricing decisions during periods of price increases. The standard price simplifies matters but deviates from reality. None of these rules is wrong, but only one should be applied—the same one used in management accounting—otherwise the margins calculated in the solution will never match those of the finance department, and the POC will spend all its time explaining variances. Our guide on calculating the gross margin provides the relevant definitions.

Reconstructing the historical context

To measure price elasticity or margin gain based on historical data, you need the purchase cost from that time, not today’s cost. When the history of supplier rates is not tracked at the product level, it is reconstructed based on invoices or, failing that, based on a margin coefficient per product family approved by the finance department. This approximation is acceptable for a proof of concept (POC) if it is explicitly stated; it is not acceptable if it is implied.

Price and Promotion History: The Actual Price Charged

The pitfall here is providing the theoretical price (the national list price) instead of the price actually charged at each retail location and through each sales channel. In between, there are local adjustments, price alignments with competitors, pricing errors, and checkout discounts. A pricing solution that learns from the theoretical price will measure price elasticities based on prices that customers have never seen.

The essential data is therefore the actual unit sales price collected—by product, store, and period—which is derived from sales revenue and sales quantities when it is not directly available. Added to this is the promotional calendar, including the mechanics of each promotion (immediate discount, bundle, strikethrough price, loyalty card benefit), their dates, and the stores covered. Displayed crossed-out prices, channel-specific prices, and competitor prices collected are supplementary data. The history of rules from the old tool, however, can wait: it is relevant for deployment, not for testing.

Enhancements: Useful, but never a hindrance

Three sources significantly improve the quality of recommendations without requiring the launch of a POC. Competitor price data, which the solution can collect online itself or integrate from in-store surveys and panelist data provided by the retailer (our article on the differences between web scraping, field surveys, and panelists explains what each contributes). Market shares by category, derived from retailer panels, which position a business unit’s performance relative to its market and help avoid confusing a decline in volume with a market downturn. Web metrics (traffic, conversion rates by product) for retailers where the online channel plays a significant role.

These data sources are often already purchased by the retailer and used within a single department. A proof of concept (POC) is a good opportunity to make them available to the pricing team, provided that their availability is not made a prerequisite: their absence delays a test, but their presence does not save it if the five core data sets are flawed. The respective roles of internal and external data in pricing decisions are the subject of a separate article.

Consistency Checks in Management Accounting

This is the step that distinguishes a data extraction from a data foundation. It involves recalculating, based on the provided files, the aggregates that management control already publishes, and verifying that they match. It takes one to three weeks and avoids months of debate over the reliability of the POC results.

Three comparisons, month by month and category by category

First, revenue: the total sales figures extracted—by month and by product category—must match the revenue figures in the management report. Next, volumes, using the same breakdown. Finally, the margin, calculated using the selected valuation method, compared to the management margin. A variance of less than 1% is normal (rounding, closing dates, returns). A discrepancy of several percentage points for a given month or product category indicates a specific problem: a missed sales channel, unaccounted-for promotions, returns counted twice, or incorrectly dated purchase costs. Each discrepancy must be documented and corrected before moving on to the next step.

The Rejection Rate as a Quality Indicator

During the load, some of the records do not map to anything: sales for a product not in the catalog, purchase prices without a product, or inventory at a closed store. This rejection rate, measured by data family, is the first indicator of the quality of the data foundation. A rejection rate of a few percent can be addressed during the POC; a rate of 15 or 20% on sales means that the master data is not ready and that the test must be postponed. It is precisely this distinction that allows us to answer the question “Is our data good enough to start?” based on more than just a hunch.

At Booper, this alignment with management accounting is part of the data integration process: the platform loads the available data extracts, tracks rejections, and the test does not begin until the revenue, volumes, and margins within the scope match those of the finance department. The page dedicated to data integration and flow monitoring describes how this works, and our article on integrating pricing data without an IT project demonstrates that no connectors are required at the outset.

What Really Holds Up a Test—and What Can Be Fixed Along the Way

Four issues prevent a conclusion and justify postponing a POC: a data model without a stable identifier, aggregated sales data that goes beyond a single day or point of sale, a complete lack of purchase costs, and revenue figures that do not closely match those in the production system. Everything else (stock to be replenished, partial new-product attributes, promotions to be linked, and approximate historical conditions) can be addressed during the test, provided it is documented and consistent across both the test and control scopes. Waiting for a perfect foundation means never testing; testing without addressing these four points means measuring nothing.

A common framework for all publishers surveyed

When a retailer compares several solutions, the dataset is what ensures a fair comparison. Once prepared and validated by management control, it is provided in identical form to each vendor along with its documentation: file descriptions, grid size, depth, valuation rules, known rejection rates, and documented defects. Any discrepancies in the results then stem from the solutions themselves, not from the data they received.

This foundation has a second benefit: it outlasts the POC. Whether or not a solution is selected, the retailer has a documented and reconciled pricing dataset—something it generally never had before. Many retailers discover at this stage that the problem they attributed to their existing tool stemmed in part from the data feeds that powered it. A software vendor can assist with this preparation without waiting to be selected: Booper offers a complimentary review of the planned data extracts to verify that they cover the criteria listed above before internal teams begin the extraction process. The scope of the test itself (scope, baseline, metrics, duration) is covered in our article on how to verify that a pricing POC creates value.

The Dataset Acceptance Checklist

The twelve checks listed below, which the store manager checks off before submitting their data, summarize the article. They fit on one page and are shared with the management control department and the IT department.

• Reference System: a unique and stable identifier for each product, an EAN code, a hierarchy with at least three levels, and a status of "active," "new," or "end-of-life."
• New Features: Attributes (format, line, brand, replaced product) for products less than six months old.
• Duplicates: Code changes and known duplicates are listed, along with their mapping table.
• Sales: volume and revenue by product, day, retail location, and channel, over 24 months (at least 12).
• Promotions: Each promotional sale can be identified either by a flag in the sales data or by a specific schedule tied to the product and store.
• Inventory: a weekly inventory level by product and store location for the period, either archived or reconstructed based on inventory movements.
• Purchase Costs: A net purchase price per product and per period, using a valuation method (PMP, last price, standard) selected in consultation with the finance department and applied consistently across the board.
• Price: The actual selling price received per product, point of sale, and period, not the list price.
• Revenue reconciliation: a variance of less than 1% compared to management reporting, by month and by product family.
• Reconciliation of volumes and margins: the same requirement applies, based on the selected valuation method.
• Rejection rate: measured by data set, less than 5% of sales, with causes documented.
• Documentation: one description per file (mesh, depth, source, known defects) provided in identical format to each editor.

Retailers that address these twelve points before a proof of concept (POC) reap benefits that go beyond the test itself: pricing data that the procurement, category management, and finance teams interpret in the same way. What the data does not include—the business context (commercial constraints, supplier commitments, branding policies)—is the subject of our article on the business context beyond the data; it is conveyed during scoping workshops, not in files.

FAQ: Data from a Pricing Proof of Concept (POC)