Production forecasting frequently asked questions and examples
The examples below are motivated by a set of frequently asked questions (FAQs), in turn highlighting common errors seen in forecasting, and are summarized by learning points that demonstrate why a consistent forecast definition is a pre-requisite for a lean forecasting process, applicable to resource estimation, business planning and decision making.
It is not a requirement to use these definitions or the proposed forecasting principles but it is considered best practice; the examples will show that, the closer a company applies these definitions and principles, the leaner the overall forecasting, resource estimation and business planning process will be. Lean in this context means “getting it right the first time” and avoiding waste and unnecessary re-work.
Does my forecast always have to result in a high (P10)/best (P50)/low (P90) estimate of the ultimate recovery?
There are many situations, where the model objectives dictate another objective function than ultimate recovery; however, the forecaster should always plan for making a P10/P50/P90 forecast that is consistent with the resource estimates in addition to the primary objectives of the study. This should be done whether the customer asks for it or not.
A reservoir engineer was requested to provide forecasting support for an exploration lease sale. A number of offshore blocks were on offer and he made a Monte Carlo simulation, based on seismically derived volumes, reservoir property trends, range of well count, development/operating costs and infrastructure requirements to point of sale. The objective functions were NPV and EMV for a significant number of prospects in these offshore blocks. This was exactly the information the exploration department had requested to determine the optimal bid value of these block.
Unfortunately, the exploration department also kept a database of all the prospects where they stored the so-derived prospect volumes as low (P90), best (P50), high (P10) estimates, consistent with PRMS definitions for prospective volumes (assuming the company would win the lease sale and get these block). The reservoir engineer’s volumetric ranges were input into this database.
Subsequently, an exploration review was carried out on this database and the reviewers discovered inconsistencies in the volume estimates: ranges were not wide enough, the best (P50) EUR was bigger than the high (P10) EUR and the best (P50) EUR was smaller than the low (P90) EUR in some prospects Fig 1. The reservoir engineer was not involved in this review, and it took several days and lots of rework to discover the root cause of the inconsistent EUR volume estimate, which was, of course, due to choosing the inappropriate objective function in the Monte Carlo analysis. Ie. For a given prospect, the realization that results in the P50 NPV will generally NOT result in the P50 EUR.
INSERT Figure 1 Mapping of NPV vs. EUR as objective function (Pending permission approval)
Valuable time was lost in the lease sale, good staff work was discredited by a misinterpretation of the results.
The customer often does not know that he needs low (P90), best estimate ( P50), high (P10) estimate of resources volumes.
As a forecaster, you should always treat the range in ultimate recovery as an objective function, at least as a secondary objective. In this example, all the forecasts were already made and they did not have to be re-run, they only had to be re-sorted. Simply plotting the forecasts as cumulative production vs. time would have given a proper low (P90),best estimate (P50), and high (P10) range to be used for forecasting, for resource estimation and for input into the exploration data base Fig 2.
INSERT Figure 2 - Prospect forecasts, resorted by EUR (Pending permission approval)
What about decision-based modeling – surely this will have different objectives than ultimate recovery.
Decision-based modeling is an essential concept in the oil industry. It is a highly recommended technique and absolutely necessary when it comes to making timely decisions based on limited data. It appears at first look not to be consistent with always making a low/best/high forecast for EUR; however, the lean approach would be to still provide a forecast adequate for reserves and business planning after the decision has been made. This can save a lot of work duplication later on.
Prospect A is a deepwater discovery made with a single well, a full logging suite including an MDT and good-quality seismic. A decision needs to be made whether this field can be developed economically and whether there is a preferred development concept. In many cases, this decision can be supported by a simple tank model with the uncertainty reflecting the range in HIIP and some connectivity adjustment factors derived from appropriate analogs. The outcomes are tested against development concepts of a FPSO (Floating Production Storage Offloading vessel), TLP (Tension Leg Platform) and a subsea tie-back to a nearby host facility. The development options are evaluated with a Monte Carlo analysis and ranked by NPV (Net Present Value) and VIR (Value Investment Ratio) for a wide range of outcomes. It is very apparent that the subsea tie-back is the frontrunner: it is economic in more than 90% of the subsurface realizations and it has a better NPV than an FPSO or TLP in 90% of all realizations Fig 3. The decision is thus clear cut from the simple tank model and is made without delay.
Figure 3 - NPV for three development options (Pending permission approval)
As the decision is taken, the subsurface team is congratulated by the decision makers for their pragmatic, decision-based approach, but the subsurface team is also requested to now produce a slightly more complex model that is adequate for reserves and forecasting and reflects all the available data and the full uncertainty in ultimate recovery.
This does not take much time to build this more detailed model and derive a resource estimate because all the data are available and only the winning concept of a tie-back needs to be evaluated in more detail Fig 4. (Pending permission approval)
Contingent resources can be evaluated with this more granular model. The corporate forecast can also be made with the same model.
INSERT Figure 4 - Prospect A cumulative production vs. time
Six months later, the owner of the host facility comes back with an offer of more capacity at a higher tariff. This does not affect the decision because it only makes the selected concept better, but it will affect the reserves and corporate forecast. This is now an optimization and requires a more detailed model than the concept selection. As the team has just built the more granular model for corporate forecasting and reserves, the optimization of more export capacity for more money can be quickly evaluated with this model Fig 5.
INSERT Figure 5 - Optimization carried out with reserves model (Pending permission approval)
Consequence of NOT building the comprehensive model could have been in the worst case: rebuilding of three dedicated simulation or analytical models, depending on how fragmented the organization is: one for reserves, one for the corporate forecast and one for the optimization. The tariff optimization would have taken much longer.
If we support a simple decision with a complex model, we are wasting valuable time in the decision-making process. If we don’t close out the decision with a model that is sufficiently detailed for subsequent forecasting and resource estimation and by estimating the ultimate recovery uncertainty range, we waste even more time subsequent to the decision.
Does the resource forecast have to be consistent with the corporate forecast?
There is strictly speaking no explicit requirement that the corporate forecast and the reserves must be equivalent. However, there are many undesired and unintended consequences if they are not:
- Data management and planning stability: maintaining different forecasts will make data management unnecessarily messy. Many companies have introduced a single forecast database that holds both the corporate forecasts and the reserves and enforce this internal consistency (see Section 1.4)
- Credibility will suffer both internally and in relation to your partners and competitors and the financial world because analysts will scrutinize published forecasts for inconsistencies.
- It unnecessarily complicates the decision making process as it is likely the decision maker will want to reconcile these forecasts as they will want clear and consistent information to base their decision on.
Look at it from the time traveler point of view: with hindsight, there will be only one “correct” forecast. If you produce two different forecasts for different objectives, then at least one of them will be wrong (probably both).
Company A presents a reserves forecast for a major heavy oil steam drive project to its partner, Company B, for review. The forecast is challenged and Company B requests to see a comparison with the corporate forecast, which was submitted as part of the operating plan for budget approval a few weeks earlier. The operating plan forecast is significantly different. While the ultimate recovery at abandonment is the same as the best estimate resource forecast, the EOR response is much more pronounced and much more favorable, and the rate forecast is not even within the high-low uncertainty range of the reserves forecast.
INSERT Figure 6 - Corporate forecast vs. reserves forecast (Pending permission approval)
Company A is requested to make a consistent set of forecast by the partner. Company A loses a lot of technical credibility and their forecasts will be heavily scrutinized for many years to come.
Consistency and credibility is key in forecasting and reserves. Inconsistencies will ultimately be discovered by somebody at an inconvenient time.
Does the whole company have to follow the same forecasting principles?
Management would expect that all forecasts are produced based on the same principles ie. Adhering to the same definition of Minimum Expectations and Definition of Low, Best and High Forecasts. Strategic decisions are made based on representative risk profiles of quite diverse projects (e.g., investing in an infill project in a mature field onshore vs. a deepwater green field development). These decisions are not based only on the best case forecast, but on the full uncertainty range of these projects. Furthermore, the robustness of the corporate forecast may be assessed based on the uncertainty ranges of individual forecasts.
Company D consistently underperforms against its best estimate (P50) forecast. It is suggested that probabilistic forecasting might improve the situation. The CEO request a probabilistic aggregation of the corporate forecast. Statisticians are hired to aggregate individual asset forecasts to the corporate level probabilistically.
However, it is found that the range is based on different concepts of forecast uncertainty: some have NPV as the objective function, some have EUR as the objective, other forecasts focus only on the short-term deferments during the first one or two years. It is concluded that there is no sensible way to aggregate or compare these forecasts.
The effort is aborted. Time and resources were wasted. Probabilistic forecasting is discredited in Company D, perhaps for the wrong reasons.
It is essential to have a consistent definition of low (P90), best estimate (P50), high (P10) within each company, and it would be even better (ideal state) to have a consistent definition across the whole industry.
Does the short term forecast also have to be consistent with the resource forecast
It is often argued that a short-term forecast is very different from the long-term forecast because the main uncertainty drivers are short-term activities and system availability. However, a lean process would attempt to unify the long- and short-term forecast by making the short-term simply the early time uncertainty in the long-term forecast.
While this was organizationally difficult in the past, we now have IPSM (Integrated Production System Modelling) tools available for reservoir to point of sale forecasting that can be used by both long-term and short-term forecasters, so theoretically the short-term forecast could be derived from the long-term forecast by explicitly modeling activities and downtimes within the same model, thus automatically updating the long-term forecast.
Company X has deployed a fully comprehensive IPSM model for some of its gas assets, which include all forecast building blocks from reservoir to point of sale. This model is shared by both the long-term forecasters and the short-term, operational forecasters.
For example, an ESP change-out and stimulation is planned on Well P1. There is uncertainty on the time when the workover can start, how long the well will be shut-in and what the well potential will be after the stimulation Fig 7. The operational forecaster reflects this uncertainty in the overall reservoir to point of sale IPSM model, thereby updating the overall uncertainty in the forecast. Once the stimulation has been carried out, this uncertainty is reduced to zero and the forecast will be continued with the new well potential
Whether the workover has an impact on reserves or not can be easily tested by running the IPSM to the end of field life.
INSERT Figure 7 – Short-term uncertainty due to well intervention (Pending permission approval)
These gas assets have a fully consistent and lean method of forecasting. One set of forecasts can be used for reserves, corporate planning, well and reservoir management and short-term activity planning. The forecast is always up to date and reflects the latest operational status of the wells and facilities.
Why is this example discussed in the reserves and forecasting? Having a shared model is a win-win for both the reserves forecaster and the operational forecasters:
- Operations need to accurately predict available capacity and will always have reliable reservoir decline trends whether they need to predict capacity for hours, days, months or years.
- The reserves forecaster will always have the latest, reliable operational updates to well skins, pipeline obstructions, compressor settings and downtime already built into the model by the experts in the model.
- Reporting any resource as a reserve will require them to meet the requirements as defined in the PRMS, which requires economic modeling of the flow streams with the related costs and investments for the PRMS classifications (1P, 2P, 3P and 1C, 2C, 3C).
Unfortunately, this integrated method has been mainly successful only for gas assets so far. At the time of writing (2014), the author is not aware of any major oil assets that have successfully implemented such a tool. System modeling tools currently available are too complex, have too elaborate a learning curve, are not versatile enough, or are simply too slow for short-term operational forecasting . Vendors are encouraged to work towards improving their tools. It is estimated that by 2020 the industry will have a wider penetration of such tools. In the meantime, adapting the unified definitions will prepare the ground for a wider implementation. For example, the long-term forecast could be represented by decline curves and type curves in an integrated model and updates could be manually exchanged between the long- and short-term forecasters.
For many gas systems, these shared IPSM models have been implemented and are working quite successfully already. This gives some confidence that the more complex oil assets will soon follow with a wider penetration.
Is there guidance on how wide the uncertainty range should be, why not just use ±20%
This would defeat the purpose of characterizing the risks and upsides in a forecast by an uncertainty range. Every field is unique in character and in terms of the amount and quality of data available. As a time traveler you should ask yourself the question: Do I feel comfortable that any reasonable outcomes will fall within the low/high range, and if the actual production will deviate significantly from my best estimate forecast, will I be able to justify my assumptions?
Field A is a subsea tie-back to a shared facility. During the appraisal phase, STOIIP is estimated to be 100 MMBOE and best estimate (P50) reserves are estimated at 25 MMBOE. Two wells are planned in the FDP that are fully funded. As the first well is tied back, the pressure declines rapidly and it is discovered that the well has been completed in a small fault slither within a larger fault block Fig 8. The remaining well is cancelled due to risk of compartmentalization and the best estimate reserves estimate is reduced from 25 to 5 MMBOE.
In the next three years, the well declines further and dies after 5 MMBOE have been produced. Figure 2.12 shows the production forecast and the actual production (in red) at the time the well was first tested. It is clear that an uncertainty range of +- 20% (black bars) is not nearly enough to describe the uncertainty in the original forecast.
INSERT Figure 8 - Field A Forecast and actual production (Pending permission approval)
Note: Example 7 will show that the well restarts after a year of shut-in, but the cumulative production of 8 MMBOE after 10 years will still be below the original low (P90) forecast.
One can argue whether the best estimate forecast was sound and the reserves estimate should have been 25 MMBOE. However, there was a geological study done that shows that the main fault is clearly visible on seismic, but has small throw and clear sand to sand juxtaposition Fig 9. A fault seal analysis showed a less than 20% chance that the main fault is fully sealing. Some subseismic faulting was identified, but was estimated to have minor baffling effect at worst. Hence, the best estimate forecast is still supportable and was considered with hindsight. However, a symmetric uncertainty range is not supportable. A low-case forecast must reflect a main fault that is fully sealing and subseismic faults that act as major baffles.
INSERT Figure 9 - Reservoir A wells and faulting (Pending permission approval)
Uncertainty ranges are important information and need to be carefully evaluated and justified. A symmetric range should never be used without detailed justification. There is no default guidance on how wide the uncertainty range should be.
Is there guidance on how the uncertainty narrows over time?
The PRMS presents a conceptual chart that gives some indication on how the uncertainly may change over time. This chart is quite useful as a general trend, however, it should be treated as a rough indication only. Specifically, any skew in the outcome must be carefully analyzed on a case by case basis. It should not be used without some uncertainty analysis, and it should definitely not be assumed that the range will be symmetric or will always narrow in this fashion or that the best estimate ultimate recovery will remain constant over time. Fig 10 
INSERT Figure 10 - Conceptual uncertainty range (Pending permission approval)
Consider Field A of Example 6. The well has died after 4 years of rapid decline and was shut-in. The reserves were set to zero. A year later, the well is re-entered to be prepared for abandonment. As the well was re-entered, it was noticed that the pressure had surged and the well was put back on production. It produced for another few months at fairly high rates Fig 11.
When the well first dies and is shut-in, the remaining reserves are set to zero with no upside or downside. This is quite wrong with hindsight because the well starts to produce again as the pressure recharges and this uncertainty should have been reflected with a very high upside (i.e., another 3 MMBOE were produced after the reserves were set to zero).
What should the forecast look like after the well first has started to produce again? The forecaster decides to make the low (P90) and best (P50) forecast zero because as he considers the likelihood that the well can be restarted again as less than 50%. The high (P10) forecast is estimate to produce another 4 MMBOE. In reality, the well produces another two cycles with a total of 3 MMBOE. Figure 2.16 shows the reserves history over time.
INSERT Figure 11 - Field A, actual production and forecast after the well has died (Pending permission approval)
While the forecaster has initially underestimated the downside, he or she has underestimated the upside at the end. The uncertainty near the end of field life was much higher than expected: When the well was first shut-in in Year 5 and the reserves were set to zero, the actual remaining recovery was still 3 MMBOE, the uncertainty was therefore 40% of ultimate recovery (3 MMBOE /8 MMBOE) and 100% of the remaining reserves (3 MMBOE /3 MMBOE).
It is quite possible that an asset forecast has initially a very significant downside skew and then develops a significant skew to the upside toward the end of field life. There is no guarantee the uncertainty will reduce toward the end of field life. See Lookbacks for more examples of this.
The example also shows that toward the end of field life, the uncertainty on (remaining) reserves often increases again with time (e.g., due to water production risk or pressure re-charging, see also SPE 94680). Often the terminal conditions (liquid loading, minimum well or facility rates, artificial lift, etc.) will give a wider range of the remaining reserve volumes just before abandonment than during the producing life.
Why should reserves and resources be project-based?
The SPE PRMS is project-based and this has added much clarity to the resource classification, it allows for a clean differentiation between project maturity and resource volume uncertainty by splitting the resources of a field or reservoir into projects of various degrees of maturity. Note that at the time of forecasting of Examples 6 and Example 7, the PRMS was not published yet.
Consider Field A of Example 6. We have so far only discussed this example in terms of field reserves. The PRMS project-based definition allows for a much clearer classification into two projects by well: Well P1 and well P2. Well P1 has been drilled and tied-back and is therefore classified as developed reserves. Well P2 has not been drilled yet, but is firmly approved and budgeted and is therefore classified as undeveloped reserves.
Both projects carry their own uncertainty range within their distinct resource class. When Well P2 was cancelled, it is “deleted from reserves“; however, it still carries contingent or prospective resources with the same low/best/high uncertainty estimate. It is thus only reclassified to a less mature resource class. If the oil price increases or if there are new data that would justify the risk of drilling again in the second fault block, then the well/project can be matured again to undeveloped reserves.
Note that according to the original development plan, Well P2 would have been drilled and reclassified to developed reserves. In both cases, the forecast would not change (unless there were new data), only the classification of the second well/project would change in line with its approval status.
From Year 5 onward, Field A has no more best estimate (P50) reserves, but still carries contingent resources Fig 13.
INSERT Figure 13 - Field A Resource History by maturity class (Pending permission approval)
PRMS project-based classification system allows for effective tracking of individual projects with their uncertainty range as the development plan evolves over time.
Project-based resource classes allow for better management of resources without having to redo the forecasts. You may account for volumes that no longer qualify as reserves by reclassifying (de-maturing) those to contingent or prospective resources.
What about SEC proved reserves – do they fit in these forecasting principles?
SEC proved reserves is about compliance with the SEC definitions and some special considerations apply that are discussed in “Modernization of oil and gas reporting” and will not be further discussed in this text. Most companies have a group of trained reserves assessors for review and management of SEC reserves or use external reviewers to ensure compliance. The recommended approach is to provide a good technical low (P90) forecast as defined in this text as a starting point and then work with the assurers to ensure that this low (P90) forecast is compliant or make the appropriate modifications.
Note: if a company files a SEC report or discloses a number as reserves in an investor presentation, the figure must meet SEC definitions which may be different from the PRMS, which is applied within the company for its business plan.
Consider Field A of Example 6. The main fault appears to be nonsealing. The throw is smaller than the sand thickness and there is sand-to sand juxtaposition across the whole fault plane. A fault seal study has quantified the chance of a sealing fault to be 20% or less. The forecaster models the whole reservoir with a range of fault seal factors ranging from 0.0 to 0.5 and proposes that the resulting low (P90) forecast be used for SEC proved reserves booking.
The reserves assurer advises that although this is a technically sound approach, he or she still deems this fault to be “major and potentially sealing.” Therefore, there may be no reserves assigned to the undrilled block, it should be classified as prospective resources , in line with SEC definition (a) (26): “Note to paragraph (a)(26): Reserves should not be assigned to adjacent reservoirs isolated by major, potentially sealing, faults until those reservoirs are penetrated and evaluated as economically producible. ….. Such areas may contain prospective resources (i.e., potentially recoverable resources from undiscovered accumulations).”
Note that SEC definitions would also allow possible reserves if the fault was deemed to be minor due to sand-to sand juxtaposition.
In practice, this may be achieved by looking only at a subset of subsurface realizations where the fault is fully sealing and taking the low (P90)of this subset. Note that SEC also requires the proved reserves to be limited to volumes above the LKH (ODT) in Fig 13 and the forecaster would have to make sure that the chosen scenario has the OWC coinciding with the ODT: “(ii) In the absence of data on fluid contacts, proved quantities in a reservoir are limited by the lowest known hydrocarbons (LKH) as seen in a well penetration unless geoscience, engineering, or performance data and reliable technology establishes a lower contact with reasonable certainty.”.
Note that there are no volumes “lost” by this correct approach. The P2 well will get the same forecast as in the previous example, but it is initially only reclassified from “undeveloped reserves” to “prospective resources” and this is what happened at hindsight when the well was cancelled due to risk of compartmentalization Fig 14. Had the well been drilled as planned, it would then have matured from prospective resource to proved developed reserves.
INSERT Figure 14 - Field A, SEC compliant areas and Resource history (Pending permission approval)
It is good practice to record technical low (P90) forecast and SEC proved and reasons why they are different in case there are differences. Some companies record both technical low case reserves and SEC proved reserves in their internal reserves databases.
Experience shows that if SEC proved and technical low (P90) forecasts are tabulated against each other, at least 90% will be the same and some 5 to 10% will be different. Note that the only difference in Example 8 is that P2 is classified as prospective resources from the start. All the forecasts are the same.
SEC compliant reserves require special considerations. A sound technical low (P90) forecast should always be the basis for a proved estimate, but compliance with SEC definitions may require some changes. Often the forecasting model can be used to derive SEC proved reserves with only small modifications. It is important to document reasons for different assumptions and how they were implemented in the forecast model.
Are there other implications of a project-based resource classification?
Project-based resource classification allows for a much cleaner link between forecasts and resource estimates and more transparency in the resource estimates, but it also implies that some legacy methods of resource estimation and classification are no longer consistent with these principles.
Incremental forecasts need to be made for new projects, specifically for secondary and tertiary projects, but also for infill drilling; i.e., the forecaster has to subtract a forecast with the incremental project from the base forecast with the existing development to determine the incremental resources (and economics) due to a new project.
Look-backs that compare forecasts vs. actual need to take the underlying projects into account – many only look at field reserves growing over time and sometimes neglect that this is due to more investment than originally planned. A proper look-back analysis should differentiate between reserves that have grown because forecasts may have been pessimistic or because of incremental projects, such as infill drilling, secondary and tertiary recovery.
Some reserves practitioners in the industry use the best fit in DCA as the low case and use this best fit to estimate the low remaining ultimate recovery and 1P developed reserves. Other practitioners use a best estimate type curve derived from analogs, especially in unconventional plays, for the PUD wells, for the probable wells and for the possible wells, and go on to add the proved wells to the probable wells and to the possible wells to report a 2P or 3P production forecasts and a 2P or 3P reserves.
This is in contrast with the COGEH (Canadian Oil and Gas Evaluation Handbook), which guides the reader on how the low, best and high forecast can be derived with DCA examples.
COGEH would be consistent with a project-based approach and hence the PRMS, while the practitioners in the example above would not be.
Fig 15 (from the PRMS) clearly shows that every project subclass needs to have an uncertainty range. This also includes developed reserves. A best-fit DCA would normally imply a 50% chance of being exceeded, but 1P reserves require a 90% chance of being exceeded.
An uncertainty range is essential for undeveloped reserves (approved or justified for development) and for contingent resources because these wells have not even been drilled yet.
Assume that all locations in a property were proved, then there would be no uncertainty range for the whole project and a single type-curve cannot represent the 1P, 2P and 3P reserves of a project.
INSERT Figure 15 - Uncertainty for project subclasses (Pending permission approval)
Project-based reserves classification has many advantages, but it also requires some discipline in applying these principles consistently. Some legacy methods are not consistent with a project-based classification system.
- "Petroleum Resources Management System." Petroleum Reserves and Resources Definitions. Society of Petroleum Engineers. Section 4.1. http://www.spe.org/industry/reserves.php
- Securities and Exchange Commission, 17 cfr parts 210, 211, 229, and 249, [release nos. 33-8995; 34-59192; fr-78; file no. S7-15-08], rin 3235-ak00. Modernization of oil and gas reporting, revisions and additions to the definition section in rule 4-10 of regulation s-x. http://www.sec.gov/rules/final/2008/33-8995.pdf
Noteworthy papers in OnePetro
Production forecasts and reserves estimates in unconventional resources. Society of Petroleum Engineers. http://www.spe.org/training/courses/FPE.php
Production Forecasts and Reserves Estimates in Unconventional Resources. Society of Petroleum Engineers. http://www.spe.org/training/courses/FPE1.php