This project went through a couple of time series forecasting technique to forecast the sales on each of the Google Ads (PPC) accounts. One application is to detect performance abnormalities when actual sales are off from forecasting amount so that we can go ahead to investigate that account and find out the root causes, it also helps to back up our actions when it's the right time to adjust keywords' bids, e.g., to bid up the entire account when sales are outperforming the forecast.
We have tested out 2 of the most talked about forecasting models, ARIMA and Prophet, to predict the sales for the next day in each one of the accounts. The length of the training set is fixed, and it would roll forward each day and predict the next day until the end of the time series. Then we will compare the average RMSE of the predictions from different models.
Before jump into the results, let's briefly go through what's under the hood of these 2 models,
-
ARIMA: A linear combination of an autoregressive model(AR) and a moving average models(MA). AR is a regression model using past values of itself as variables, and MA is a regression-like model using past forecast error as variables, can also be considered as a weighted moving average of the past few forecast error. A model with p order of AR and q order to MA can be expressed like this:
is the first order of difference series, the purpose is to transform the time series to stationary which AR and MA can fit.
In this project we useforecastpackage developed by Rob Hyndman. More details on ARIMA are explained clearly here. -
Prophet: An open source project built by Facebook, originally they use it to tackle capacity planning and performance measurement related issues within the organization. Prophet is an additive regression model with four main components, growth curve trend, yearly and weekly seasonality modeled with Fourier series, user-provided special holidays using dummy variables, and changepoints where trend changes suddenly and abruptly.
Prophet is designed to be intuitively customizable even for non-experts. It doesn't have complex mathematics behind the parameters. Instead, they provided clear instructions of how to tweak forecasts with each parameter.
In the end of the project we see that ARIMA model performs slightly better than Prophet with ~7% lower in RMSE. And we further test an ensemble model which averages out the predictions generated from these two models, it reduces RMSE for additional 4%. The result is not so surprising as ensemble method often has better predictions than the best individual model.
ARIMA remains powerful especially when dealing with the series has clear seasonality and it's in short-term, in this case 7 days. In addition, with the built-in auto.arima function from the package, it's highly scalable in modeling multiple time series when their patterns are not exactly the same. On the other hand, Prophet is handy when the series has multiple seasonality, e.g., weekly, quarterly and yearly. Prophet also has a strength of easy to tune the hyperparameters. By adjusting growth function, seasonality scale and frequency, you can easily shape the forecast as you see fit. However, for a very short forecasting window like this example, its strength can't be brought into play.