Forecast combination based forecast reconciliation: insights and extensions (original) (raw)

Cross-temporal forecast reconciliation: Optimal combination method and heuristic alternatives

International Journal of Forecasting

Forecast reconciliation is a post-forecasting process aimed to improve the quality of the base forecasts for a system of hierarchical/grouped time series (Hyndman et al., 2011). Contemporaneous (cross-sectional) and temporal hierarchies have been considered in the literature, but-except for Kourentzes and Athanasopoulos (2019)-generally these two features have not fully considered together. Adopting a notation able to simultaneously deal with both forecast reconciliation dimensions, the paper shows two new results: (i) an iterative cross-temporal forecast reconciliation procedure which extends, and overcomes some weaknesses of, the two-step procedure by Kourentzes and Athanasopoulos (2019), and (ii) the closed-form expression of the optimal (in least squares sense) point forecasts which fulfill both contemporaneous and temporal constraints. The feasibility of the proposed procedures, along with first evaluations of their performance as compared to the most performing 'single dimension' (either cross-sectional or temporal) forecast reconciliation procedures, is studied through a forecasting experiment on the 95 quarterly time series of the Australian GDP from Income and Expenditure sides considered by Athanasopoulos et al. (2019).

Probabilistic forecast reconciliation: Properties, evaluation and score optimisation

European Journal of Operational Research

We develop a framework for prediction of multivariate data that follow some known linear constraints, such as the example where some variables are aggregates of others. This is particularly common when forecasting time series (predicting the future), but also arises in other types of prediction. For point prediction, an increasingly popular technique is reconciliation, whereby predictions are made for all series (so-called 'base' predictions) and subsequently adjusted to ensure coherence with the constraints. This paper extends reconciliation from the setting of point prediction to probabilistic prediction. A novel definition of reconciliation is developed and used to construct densities and draw samples from a reconciled probabilistic prediction. In the elliptical case, it is proven that the true predictive distribution can be recovered from reconciliation even when the location and scale matrix of the base prediction are chosen arbitrarily. To find reconciliation weights, an objective function based on scoring rules is optimised. The energy and variogram scores are considered since the log score is improper in the context of comparing unreconciled to reconciled predictions, a result also proved in this paper. To account for the stochastic nature of the energy and variogram scores, optimisation is achieved using stochastic gradient descent. This method is shown to improve base predictions in simulation studies and in an empirical application, particularly when the base prediction models are severely misspecified. When misspecification is not too severe, extending popular reconciliation methods for point prediction can result in a similar performance to score optimisation via stochastic gradient descent. The methods described here are implemented in the ProbReco package for R.

Forecast reconciliation: A geometric view with new insights on bias correction

International Journal of Forecasting, 2021

A geometric interpretation is developed for so-called reconciliation methodologies used to forecast time series that adhere to known linear constraints. In particular, a general framework is established nesting many existing popular reconciliation methods within the class of projections. This interpretation facilitates the derivation of novel results that explain why and how reconciliation via projection is guaranteed to improve forecast accuracy with respect to a specific class of loss functions. The result is also demonstrated empirically. The geometric interpretation is further used to provide a new proof that forecast reconciliation results in unbiased forecasts provided the initial base forecasts are also unbiased. Approaches for dealing with biased base forecasts are proposed and explored in an extensive empirical study on Australian tourism flows. Overall, the method of bias-correcting before carrying out reconciliation is shown to outperform alternatives that only bias-correct or only reconcile forecasts.

Probabilisitic forecasts in hierarchical time series

2018

Forecast reconciliation involves adjusting forecasts to ensure coherence with aggregation constraints. We extend this concept from point forecasts to probabilistic forecasts by redefining forecast reconciliation in terms of linear functions in general, and projections more specifically. New theorems establish that the true predictive distribution can be recovered in the elliptical case by linear reconciliation, and general conditions are derived for when this is a projection. A geometric interpretation is also used to prove two new theoretical results for point forecasting; that reconciliation via projection both preserves unbiasedness and dominates unreconciled forecasts in a mean squared error sense. Strategies for forecast evaluation based on scoring rules are discussed, and it is shown that the popular log score is an improper scoring rule with respect to the class of unreconciled forecasts when the true predictive distribution coheres with aggregation constraints. Finally, evid...

Hierarchical Forecasting and Reconciliation in The Context of Temporal Hierarchy

IRJET, 2022

The purpose of this study is to find a suitable forecast aggregation strategy for forecasting temporally aggregated hierarchical data series when the base level data exhibits a seasonal pattern. The study employs 10-year monthly data of foreign tourists visited in Kerala. Forecasting is essential for the four levels of hierarchy; the monthly, quarterly, half yearly and annual foreign tourist visit data. The forecasting strategies deliberated in the project are; bottomup approach, top-down approach, and the optimal combination approach with Ordinary least square (OLS) for reconciliation. The performance of different strategies is compared using the Mean Absolute Percentage Error (MAPE). The exponential smoothing techniques; single exponential smoothing, double exponential smoothing and triple exponential smoothing are used for forecasting individual series. The study concludes that the suitable forecast aggregation strategy for forecasting temporally aggregated hierarchical data series when the base level data exhibits a seasonal pattern is bottom-up approach. Bottom-up approach outperform all top-down approaches and optimal combination approaches on average and across all levels.

Forecast aggregation via recalibration

Machine Learning, 2013

It is known that the average of many forecasts about a future event tends to outperform the individual assessments. With the goal of further improving forecast performance, this paper develops and compares a number of models for calibrating and aggregating forecasts that exploit the well-known fact that individuals exhibit systematic biases during judgment and elicitation. All of the models recalibrate judgments or mean judgments via a twoparameter calibration function, and differ in terms of whether (1) the calibration function is applied before or after the averaging, (2) averaging is done in probability or log-odds space, and (3) individual differences are captured via hierarchical modeling. Of the nonhierarchical models, the one that first recalibrates the individual judgments and then averages them in log-odds is the best relative to simple averaging, with 26.7 % improvement in Brier score and better performance on 86 % of the individual problems. The hierarchical version of this model does slightly better in terms of mean Brier score (28.2 %) and slightly worse in terms of individual problems (85 %).

Combination of multi level forecasts

The Journal of VLSI Signal Processing, 2007

This paper provides a discussion of the effects of different multi-level learning approaches on the resulting out of sample forecast errors in the case of difficult real-world forecasting problems with large noise terms in the training data, frequently occurring structural breaks and quickly changing environments. In order to benefit from the advantages of learning on different aggregation levels and to reduce the risks of high noise terms on low level predictions and overgeneralization on higher levels, various approaches of using information at different levels are analysed in relation to their effects on the bias, variance and Bayes error components proposed by James and Hastie. We provide an extension of this decomposition for the multi-level case. An extensive analysis is also carried out answering the question of why the combination of predictions using information learned at different levels constitutes a significantly better approach in comparison to using only the predictions generated at one of the levels or other multi-level approaches. Additionally we argue why multilevel combinations should be used in addition to thick modelling and the use of different function spaces. Significant forecast improvements have been obtained when using the proposed multi-level combination approaches.

Combining forecasts: The end of the beginning or the beginning of the end

International Journal of Forecasting, 1989

Research from over 200 studies demonstrates that combining forecasts produces consistent but modest gains in accuracy. However, this research does not define well the conditions under which combining is most effective nor how methods should be combined in each situation. Rule-based forecasting can be used to define these conditions and to specify more effective combinations.

Combining forecasts given different types of objectives

European Journal of Operational Research, 1991

In this paper, two fundamentally different types of objectives -accuracy and direction of change are incorporated into a multiple objective framework for combining forecasts. Most previous attempts at combining forecasts have focused on a single objective, usually some measure of forecast accuracy. While accuracy measures in terms of deviations between forecasted and actual values are more common, being able to predict correctly the direction of change, either increasing or decreasing, in a variable of interest from one period to the next can be an important consideration also. Direction of change measures can be utilized to provide important additional information for decision making. Models including both types of objectives are developed, and solution strategies are discussed and illustrated. These enhanced models provide additional flexibility for decision makers without significantly increasing overall computational requirements.

Analysis of Weighting Strategies for Improving the Accuracy of Combined Forecasts

Mathematics

This paper deals with the weighted combination of forecasting methods using intelligent strategies for achieving accurate forecasts. In an effort to improve forecasting accuracy, we develop an algorithm that optimizes both the methods used in the combination and the weights assigned to the individual forecasts, COmbEB. The performance of our procedure can be enhanced by analyzing separately seasonal and non-seasonal time series. We study the relationships between prediction errors in the validation set and those of ex-post forecasts for different planning horizons. This study reveals the importance of setting the size of the validation set in a proper way. The performance of the proposed strategy is compared with that of the best prediction strategy in the analysis of each of the 100,000 series included in the M4 Competition.