Forecasting using judgement is commonn, and may be the only option if there is a lack of historical data.
There are also situations where data are incomplete, or there is a delay in their acquisition. An exampleis central banks forecasting the current level of economic activity. This is known as nowcasting.
There are three general settings:
- No available data.
- Data available and forecasts generated, which are then adjusted using judgement.
- Data available, then statistical and judgemental forecasts are generated independently and combined.
Limitations
Judgemental forecasts are subjective, and therefore are not free of bias or limitations. Recent events may have a larger affect on cognition, and more momentous events from the past may be deemed less important.
Judgement can be clouded by political agendas - ‘beware the enthusiasm of marketing and sales’.
Another property is anchoring, where forecasts tend to converge or be close to an initial familiar reference point - e.g. the last observed value.
Key Principles
A systemic and structured approach helps to reduce the adverse effects and limitations of judgemental forecasting.
Clear and Concise
Set teh forecasting task clearly and concisely. All definitions should be clear and comprehensive, avoiding ambiguous and vague expressions.
Systemic Approach
Use a checklist of categories of information which are relevant to the forecasting task. Identify what information is important and how is it weighted. What factors should be accounted for, and how should they be accounted?
Put together decision rules that will lead to the best possible systematic approach.
Document and Justify
Formalise and document the decision rules and assumptions implemented in your systematic approach.
Systematically Evaluate Forecasts
Keep records of forecasts and use them to obtain feedback when the corresponding data becomes available.
Segregate Forecasters and Users
Forecast accuracy can be impeded if the task is carried out by users of the forecasts. As an examplem, sales orecasts of a new product may differ significantly from what management want the sales to be.
Explain and clarify the process with users so that they have cionfidence in them.
The Delphi Method
This method was invented in the 1950s by the Rand corporation to address a specific military problem. It relies on the assumption that forecasts from a group are generally more accurate than forecasts from and individual.
The aim is to develop consensus forecasts from a group of experts in an iterative manner. The stages are generally:
- Assemble a panel of experts
- Distribute forecasting tasks/challenges to these experts.
- Compile returned initial forecasts.
- Provide feedback, with the experts revewing in light of this feedback.
- This step may be iterated.
- Construct final forecasts by aggregating the experts’ forecasts.
Experts and Anonymity
The suggestion is that between 5 and 20 experts with diverse expertise is required. A key feature is that these experts remain anonymous all the time. This is so they cannot be influenced by political and social forces. They are all given and equal say, and held accontable for their forecasts.
Setting the Task
In can be useful to conduct a preliminary round of information gathering from the experts before setting the forecasting tasks.
Feedback
Feedback to the experts should include:
- Summary statistics of the forecasts.
- Outlines of qualitative justifications.
Feedback is controlled by the facilitator, who may direct attention to areas where it is most required. E.g. it may be directed to responses that fall outside the interquartile range, and the justification for such forecasts.
Iteration
The process of submission, feedback and review is repeated until a satisfactory level of consensus is reached. Thi does not men complete convergence, but that the variability of the responses has decreased to a satisfactory level.
Generally two or three rounds is enough.
Final Forecasts
Final forecasts are generally constructed by giving equal weight to all of the experts’ forecasts. The facilitator should keep in mind the possibility of extreme values.
Limitations and Variations
Applying this method can be time consuming. In a group meeting, forecasts can be reached in hours or minutes, something that is not possible with this method. The panel may lose interest or cohesiveness.
In a group setting, personal interactions can lead to quicker and better clarifications. A variation of the Delphi method is the ‘estimate-talk-estimate’ method, where experts can interact between iterations, although the forecast submissions can still remain anonymous. The disadvantage is that the loudest person can exert undue influence.
The Facilitator
The facilitator is of the utmost importance. They are responsible for the design and administraton of the process. They are also responsible for providing feedback and generating the final forecasts.
They need to be experienced to recognise the areas that need more attention, and to direct experts’ attention to these.
Forecasting By Analogy
An example of forecasting by analogy is the pricing of a house. An appraiser estimates the market value of a house by comparing it to similar properties tht have sold in the areas. They take into account size, number of bedrooms and bathrooms, etc.
Just thinking and discussing analogous products or situations can generate useful - and crucial - information.
Scenario Forecasting
A different approach to judgemental forecasting is scenario-based forecasting. THe aim is to generate forecasts based on plausible scenarios. In contrast to Delphi and analogy where the forecast is a likley outcome, each scenario-based forecast may have a low probability.
It is usual to present “best”, “middle” and “worst” case scenarios. Contingency planning can be based on these.
New Product Forecasting
Judgemental forecasting is usually the only method available as there is a lack of historical data. There are other methods which are more specific to the situation. These methods are less structured and potentially more biased.
Sales Force Composite
Forecasts for each outlet/branch/store are generated by salespeople, the aggregated. These people are usually closest to the interaction, and develop intuition about customer purchasing patterns. However this violates the key principle of segregating forecasters from users.
Sales people will also have little to no training in forecasting.
Executive Opinion
This contrasts against sales force composite in that it is the top of the hierarchy that is making the forecasts. This carries all of the disadvantages of a group meeting setting. The executives need to be held accountable in order to reduce biases.
Customer Intentions
Questionnaires are filled in by customers on their intentions to buy the product. Survey challenges, including an appropriate sample size, and applying a cost and time effective method, and dealing with non-responses need to be addresses.
Also, purchase intention versus purchase behaviour do not necessarily line up. The strength of the correlation between intention and behaviour vary substantially.
Judgemental Adjustments
These adjustments are used in a situation where historical data is available and statistical forecasts are generated, then judgements applied to these forecasts.
This could be because of promotions, public holidays, or recent events not reflected in the data.
Use Sparingly
Practicioners adjust much more than they should. By adjusting, users feel ownership over the models. They don’t often appreciate or understand the mechanisms that have generated the model.
These adjustments should not attempt to correct for a systematic pattern in the data that is thought to have been missed. They are most effective when there is significant information at hand or strong evidence of the need for adjustment. Small adjustments, especially in the positive direction, have been found to hinder accuracy.
Apply a Structured Approach
Using a structured and systematic approach will improve the accuracy:
- Documentation of approach.
- Justification of adjustments.