Upload Evaluation
We offer several metrics that enable retrospective evaluation on our predicted sales, profit and revenue. In short, you can see if our past predictions match what occurred in reality.
Live Share
The live share is the percentage of products in the optimization for which the recommended price is actually online/active. Please keep in mind that the older the optimization, the more irrelevant/outdated this share becomes, so be sure to refer to the latest optimizations if you want to know the live share.
Profit Impact
The profit impact refers to the difference between the actual (realized) profit and what the profit would have been without the optimization. This is in the context of the time period and products covered by the optimization. For example, if you uploaded an optimization 7 days ago, we calculate the actual profit generated by the products that were part of this optimization for the past 7 days and compare it to the "potential" profit these products would have generated had you not done the optimization. This "potential" profit is the one derived from the status quo point (green point in the preview curve), which just predicts the profit if you "do nothing" and keep the prices unchanged. Note that the profit metric here is calculated for all sales before returns.
Revenue Impact
The revenue impact is analogous to the profit impact, with the exception that the metric we are now measuring is the revenue.
Understanding Impact
In order to better understand how we actually calculate the impact, it's best to illustrate with an example. Let's say that two weeks ago, we have created an optimization, "X", which covers a period of two weeks and the steering strategy was "Max Profit". For this optimization, today we say that the profit impact is 2,000 EUR. In order to come up with this value, we look at two things:
- The realized (actual) profit generated by the products which were part of Optimization X during these two weeks.
- The predicted profit of Optimization X for the scenario where we would not have done the optimization (Status Quo or “Do Nothing”).
Let's assume that this predicted profit for “Do Nothing” is 10,000 EUR. Also, let's assume that the actual profit is 12,000 EUR. In this case, the profit impact would be 12,000 - 10,000 = 2,000 EUR. So, because we steered for Max Profit rather than doing nothing, we generated an extra 2,000 EUR, which are attributed to the steering strategy.
Please keep in mind that this impact is an incremental (relative) impact. Incremental in the sense that we always compare our tool against itself, because the “Do Nothing” scenario of the current optimization is nothing more than the “Max Profit” scenario of the previous optimization. So, the current prices (“Do Nothing”) are always the suggested prices of the previous upload (“Max Profit”). This calculation only makes sense in the context of a continuous use of the tool.
The "absolute" impact, which answers the question: "What would the profit have looked like, had we not used the tool at all?" is not shown here and is not easily quantifiable. The only reasonable way to answer this question would be to perform an A/B test. It compares results with and without the tool under similar conditions, isolating our tool's overall effect.
Sales Accuracy
The sales accuracy is of predicted sales before returns in the optimization divided by the sum of the actual sales before returns over the same period of time. For example, if we predict 100 sales before returns in the next two weeks, and in reality, only 90 sales were realized, then this value would be 90%. Values close to 100% indicate good predictability. Values above or below that indicate overprediction or underprediction respectively.