Price discount management: models for quantifying the effect of gas stations as an example

We continue to publish reports delivered at the RAIF 2019 (Russian Artificial Intelligence Forum). This time, Vadim Abbakumov, PhD in Physics and Mathematics, chief expert analyst at Gazprom Neft, shares his experience. We give him the word:

At the RAIF 2019 artificial intelligence systems forum (hosted by Jet Infosystems), my colleagues talked more about technological progress, and I made a presentation on a breakthrough in business processes.

To begin with - about measuring the effect of promotions. We sold the goods at the same price, then for a month he went at a discount, and then we returned to the old price. Now you need to understand whether the company has remained in the black, how much sales have increased.

On the one hand, the authors of textbooks argue that in 60% of cases holding discounted promotions does not bring the desired effect, such actions are unprofitable. On the other hand, the volume and frequency of promotions are growing. For example, sales of discount coffee are 69%.

Why it happens? In fact, almost no company has data on which stock was profitable and which was unprofitable. There is no data on the magnitude of losses. As a result, there are no process control tools.

We have long known how to measure the effectiveness of promotions, but for a huge number of companies this is still a problem - they constantly turn to me for consultations: how to do it competently? So I decided to talk about it.

For example, take our company. You may ask: what does Gazprom Neft have to do with it? They drill wells, produce oil. This is true, but not all. Gazprom Neft is one of the leaders in the sale of ... coffee. Each gas station (and this is 800 points) has its own store, and in each we arrange promotions, trying to stimulate purchases.

First, we discuss popular but non-optimal methods.

Option 1. So measure the effect of promotions of 80% of companies. If the discounts were in February, then compare with the February sales of the previous year. The difference is the result of the promotion. If there was an overlay (for example, last February there was also some kind of promotion and it is impossible to compare using this scheme), then we take the arithmetic average of sales in January and March! This option calculates perfectly in Excel, but is actually not optimal. The fact is that this ignores either a change in trend or a change in seasonal components.

Option 2A / B-test - holding a promotion only in part of the chain stores in order to compare sales figures in them and in stores where there was no promotion. This is a powerful, cool tool, but not quite simple. Our experts say that logistics is becoming very complicated (it is difficult to organize an action at one gas station and not hold it at another), moreover, this already makes testing expensive. Secondly, the lawyers of the company warn of possible problems if the promotion is held on the part of gas stations. Thus, the A / B test is theoretically possible and good, but in practice it is too difficult to apply, too many organizational problems.

Option 3 (worst).Take the time series of sales and compare the values ​​in February of the previous and current year, using the Student t-test. It seems that this will be the same A / B test, but half-knowledge is worse than ignorance: observations of the time series depend on each other, as a result, the t-criterion cannot be applied.

Option 4 (optimal): Consider the most simple scheme in which we have two rows of sales. Red color indicates sales without a stock, blue indicates a modified series, in which from 4 to 6 observation there were discounts and, accordingly, increased sales. We need to measure how much the blue line is above the red.

We construct a pair of ordinary linear regression models.

For the red line, the model is obvious:

$$

$$=a +b\cdot t$$

For the blue line, add the predictor :xt

$$

$$=a +b\cdot t+c\cdot x_{t}$$

As for the red line, we will describe the trend as a straight line. The variable is equal to one on the days of promotions and zero on the days of their absence (if sales for each day are saved). The variable is multiplied by the variable , which is an indicator of increased sales. If the result is negative, then sales have fallen, and they have decreased by units. This is the basic scheme. View variables are called indicators or dummy variables. Such variables are used in different situations, for example, in one-hot-encoding. In our case, it is an intervention, that is, an event that temporarily or permanently changes the nature of the series. Despite the fact that the basic scheme is obvious, problems begin at the stage of refinement of the model. The trend.xtxt

xtxt



The trend is most often non-linear, so care must be taken that it does not include ups and downs associated with stocks. Conversely, care must be taken that the effect of the promotion does not include ups and downs, which should be described by the trend. To solve this problem, the Prophet procedure (aka fbprophet) proved itself well. In it, the trend is described by a piecewise linear function, the segments flexibly describe the local trend.

Seasonality A series may have one or more seasonal components. For example, at a gas station there are three seasons: intraday (at night at a gas station there are fewer people than during the day), intraday (on Friday there are more customers at a gas station than on Tuesday) and annual (in summer there are more cars than in winter). Moreover, seasonality is multiplicative or additive. In sales, seasonality is usually multiplicative.

Additional predictors. The model will inevitably include many additional predictors. I will give two examples. If the air temperature is below 24 ^on C, then we have prosyadut gasoline sales, what discount nor offer gasoline is not needed, because many simply started the engine. At -24 ^on With people often use public transport rather than go to his car. Therefore, a multiplier should be built into the model that will reduce sales at low temperatures.

Second example. Perhaps this is purely St. Petersburg phenomenon, but even if the -30 on the street ^ofC, people even buy ice cream, but if it started to rain, then the buyers disappear. From the beginning to the end of the rain, sales simply stop, but why this happens is completely incomprehensible. And it doesn’t matter, you only need to integrate the multiplier into the model, which will reduce sales in those hours when it rains.

We must add additional external variables correctly and accurately, using common sense and understanding of business processes. In data analysis, this is called feature engineering.

The model already has the following form:

$$

$$=+++...+c\cdot x_{t}+d\cdot z_{t}$$

where is the same as before, and is the vector of additional predictors. Further improvements include the abandonment of ordinary linear regression models. How else can I improve the description of the promotion? For the version of the variable discussed above, we have the following graph of the change in time:xtzt

xt

If the company raised the price not “temporarily”, but “forever”, this will be an intervention, the influence of which remains. The chart will look like this:

Here, sales have risen (and more often - have fallen), and all this lasts indefinitely.

I recommend the following flexible description of the intervention:

Not too complicated, but not too simple. First, there is a rise, then the effect of the action slowly fades. In this case, the beginning and end of the “action” must be selected manually. For example, Python programmers adore grid-search, with which you can determine the beginning and end of a process.

We digress to discuss an example of an intervention that is not a promotion. A colleague worked in the "Tape", where in front of one of the shops began to build a roundabout. Getting to this supermarket was very inconvenient, as a result - the flow of customers fell. The effect of this intervention can be measured as described above. It is necessary to estimate how many customers the store lost during the entire construction period. Plus, when the denouement was completed, buyers had to get used to the idea that it was convenient to get to this supermarket. Thus, the effect of the construction quietly subsided, but persisted for some time, and this had to be taken into account.

Now we turn to the real example of evaluating the effect of a stock. Suppose we sell soft drinks. In the graph below, yellow indicates the size of the discount, and blue indicates the volume of sales.

A few observations.

In September 2018, the discount led to an increase in sales. Everything is logical - the model allows us to evaluate such growth.

In November 2017, there was a maximum discount, but it left sales at the same low level. What stunted growth? We assume the influence of an unaccounted factor and carefully select additional characteristics.

In June 2017, a small discount sharply increased sales. Maybe it's not a discount at all, but a summer heat?

In December 2019, the manufacturer arranged a master class. They came to gas stations and laid out drinks at the entrance, decorated them. As a result, simply due to the calculation of the goods, sales have quadrupled. It seems that everyone has a lot to learn from those who have worked out the business process.

Findings:

Sometimes everything goes the way it should go, sometimes the opposite.

If the model works, all is well.

But even a bad model is better than no model. In forecasting the same. A bad model at least makes us think about the effectiveness of our marketing activities.

Author: Vadim Abbakumov, Ph.D. in Physics and Mathematics, chief expert analyst at Gazprom Neft.