Quantative uncertainty scoring
"Uncertainty is not a number"
The quantification of yield gaps is subject to various sources of uncertainty, such as uncertainties in data, models, and concepts. Identification and estimation of these uncertainties provides valuable information to users of the atlas. The reported yield gaps in the atlas therefore are accompanied by qualitative estimations of uncertainties regarding the presented results, including remarks by experts on the justification of these estimations. The protocol given here describes the used approach how the uncertainty estimates are obtained. The assumption is that the GYGA framework is ‘true', i.e. it is accepted as given and there are no relevant conceptual uncertainties. Other sources of uncertainties have been framed within this conceptual view.
Terminology and explanation
Protocol parameter: phase in the GYGA protocol as described in the methods section. The relevant parameters are weather data, soil data, crop area, cropping system, modelling, and scaling.
Yield indicator: one of Yx = Yp (potential yield), Yw (water-limited yield), Ya (actual yield), or Yg (yield gap)
Level of uncertainty:
- A qualitative estimation of the ‘size' of uncertainties (from 1 to 3 to indicate an increasing level of uncertainty) in the data used for estimating a given parameter.
- For example, how much confidence is there in the weather data? Are there "suspicious" data?
- The remarks section provides a concise justification of the assessment.
- The guideline on the classification criteria for level of uncertainty given below can be used, but experts may use their own approach for estimation uncertainties if there are good reasons for this.
Sensitivity:
- A qualitative estimation of the sensitivity of the protocol parameter (from 1 to 3 to indicate an increasing level of sensitivity), i.e. in how far is it expected that the protocol parameter relatively contributes to the end result (Yx).
- Example: weather plays a more important role for Yw than for Yp; therefore the sensitivity to weather is lower for Yp than for Yw.
- Example: sensitivity to rooting depth will be greater for countries with predominantly shallow soils versus countries with predominantly deeper soils.
- Example: In countries with low actual yields (Ya), the yield gap (Yg) is less sensitive to Ya.
Effect on yield:
- A qualitative estimation of the propagated effect of the estimated uncertainty on the end result (Yx) given the sensitivity. The effect on yield is determined by using the estimated level of uncertainty (1, 2, or 3) and sensitivity (1, 2, or 3) at the hand of the look-up table given below. In this table, the levels of uncertainty (U) and sensitivity (S) have been extended to five levels to accommodate intermediate scores arising from averaging scores from two or more individual assessments. For instance, when expert A scores 1 and expert B scores 2, the average score is 1.5. In case of a divergent score (e.g. one expert scores a 1, and another a 3) this is explicitly indicated in the remarks.
Overall Uncertainty Score:
- The "overall uncertainty score" is calculated from the values in the "effect on yield" column.
- This score is determined by using an arithmetic mean. However, users can use the raw scores to calculate different scores if desired.
- The overall uncertainty score is represented using the five-color-scale in the atlas, which is similar to the one given above. It is the primary information presented to the Atlas users.
Classification criteria for level of uncertainty
Potential yield
Protocol parameter | Criteria | Low uncertainty (1) | Medium uncertainty (2) | High uncertainty (3) |
Weather | # years of measured quality data (Tmax, Tmin) | >10 | 3-10 | 0-2 |
If "suspicious data" : level -1 (please indicate why you think that the data are suspicious) |
|
|
| |
|
|
| ||
soil | NA |
|
|
|
|
|
| ||
Crop area | observed deviations between SPAM-2005 and current land use; corrected or not; expectation there will be many more deviations (please justify this expectation) | No observed deviations after checks | Observed deviations, but corrected | Observed deviations, not corrected |
|
|
| ||
Cropping system | Crop calendar: Recent (>2000); sufficient high-quality resolution (i.e. from experiments or expert networks) | Recent (>2000); high resolution | Not recent (<2000) or low resolution | Global database (not recent & low resolution) |
|
|
| ||
Data for model calibration | Calibration based on recent, location-specific experimental data | Recent local high-quality experiments that allow detailed calibration of phenology and gorwth parameters | Recent local high-quality experiments (or from similar environments) that allow a limited calibration (e.g., phenology coefficients) | Calibration based on defaults model parameters or experiments of lower quality. |
|
|
| ||
Scaling | Coverage (crop area in selected buffer zones / national crop area) | >50% | 25-50% | <25% |
Also consider whether important CZ's (>5% of national area) are missing (eg if no weather data available) |
|
|
| |
|
|
|
Water-limited yield
Protocol parameter | Criteria | Low uncertainty (1) | Medium uncertainty (2) | High uncertainty (3) |
Weather | # years of measured quality data (Prec, Tmax, Tmin, RH, Wind) | >10 | 3-10 | 1-2 |
If "suspicious data" : level -1 (please indicate why you think that the data are suspicious) |
|
|
| |
|
|
| ||
Soil | Resolution of the underlying database & quality of the pedotransfer functions (please justify the estimation of the resolution and the quality of the pedotransfer functions) | good resolution / PTF validated for the targeted area | moderate resolution / PTF validated for the targeted area | low resolution / PTF not validated for targeted area |
E.g. for Europe: Confidence levels of available water content (AWC) and rooting depth (ROO) | (AWC and ROO) are "High" | (AWC or ROO) are "moderate" | (AWC or ROO) are "Low, Very Low or Unknown" | |
|
|
| ||
Crop area | observed deviations between SPAM-2005 and current land use; corrected or not; expectation there will be many more deviations (please justify this expectation) | No observed deviations after checks | Observed deviations, but corrected | Observed deviations, not corrected |
|
|
| ||
Cropping system | Crop calender: Recent (2000); sufficient high-quality coverage (i.e. from experimental results) | Recent (>2000); high coverage | Not recent (<2000) or low coverage | Global database (not recent & low coverage) |
|
|
| ||
Model | Calibration based on recent, location-specific experimental data | Recent experiments on location | Regional literature or experiments at related sites | Global literature |
|
|
| ||
Scaling | Final coverage after soil selection step | >30% | 10-30% | <10% |
|
|
| ||
|
|
|
Actual yield
Protocol parameter | Criteria | Low uncertainty (1) | Medium uncertainty (2) | High uncertainty (3) |
Yield data | Source and disaggregation by crop-water regime; if suspicious level -1 (please indicate why you think that the data are suspicious) | Crop-specific national data | Crop-aggregated national data | expert data |
|
|
| ||
Number of (recent) years | # recent years for Rainfed / Irrigated | > 10 / > 5 | 5-10 / 3-5 | <5 / <3 |
|
|
| ||
Scaling | Spatial resolution | District - NUTS3 "equivalents" | Province - NUTS2 "equivalents" | Region - NUTS1 "equivalents" |
Yield Gap
- The uncertainty of Yp, Yw and Ya has been assessed separately
- We take the average values of "uncertainty" from the Yp-, Yw- and Ya-sheets to fill in the score
- (Yp or Yw) and Ya are given a weight to calculate the overall "effect on Yg". Currently, both components have an identical weight. The weights are given as 1, but may be adjusted by the user if there is proper justification for doing this. We are currently considering to make the weight dependent on the level of the actual yield, as uncertainty of low actual yields will have a minor effect on the uncertainty of the yield gap.
Score sheet
- The Template is named GYGA_Uncertainty_COUNTRY_CROP.xls
- please rename the file for each country-crop combination (eg GYGA_Uncertainty_GHANA_MILLET.xls)
- Grey fields to be filled (other fields are locked):
- Four sheets for Yp, Yw, Ya and Yg
- Country and Crop in first sheet only (Yp)
- "Uncertainty" and "Remarks" per country-crop combination ;
- "Sensitivity" per crop;
- For the assessment follow the criteria given for each protocol step. Please supply concise justifications for each protocol parameter (one to three lines) in the remarks section when doing the assessment.
- The guideline is intended not to be compulsory, but as a first estimate. If you feel that a score should be different that is perfectly acceptable as long as it is justified by proper arguments. Please note that the remarks will be explicitly included in the atlas.
References
Schils, R. L. M., van Voorn, G. A. K., Grassini, P., & van Ittersum, M. K. 2022). Uncertainty is more than a number or colour: Involving experts in uncertainty assessments of yield gaps. Agricultural Systems, 195.