TY - JOUR

T1 - Estimating the Significance of the Added Skill from Initializations

T2 - The Case of Decadal Predictions

AU - Christiansen, Bo

AU - Yang, Shuting

AU - Matte, Dominic

PY - 2023/5/1

Y1 - 2023/5/1

N2 - A considerable part of the skill in decadal forecasts often comes from the forcings, which are present in both initialized and uninitialized model experiments. This makes the added value from initialization difficult to assess. We investigate statistical tests to quantify if initialized forecasts provide skill over the uninitialized experiments. We consider three correlation-based statistics previously used in the literature. The distributions of these statistics under the null hy-pothesis that initialization has no added values are calculated by a surrogate data method. We present some simple exam-ples and study the statistical power of the tests. We find that there can be large differences in both the values and power for the different statistics. In general, the simple statistic defined as the difference between the skill of the initialized and uninitialized experiments behaves best. However, for all statistics the risk of rejecting the true null hypothesis is too high compared to the nominal value. We compare the three tests on initialized decadal predictions (hindcasts) of near-surface temperature performed with a climate model and find evidence for a significant effect of initializations for small lead times. In contrast, we find only little evidence for a significant effect of initializations for lead times longer than 3 years when the experience from the simple experiments is included in the estimation.

AB - A considerable part of the skill in decadal forecasts often comes from the forcings, which are present in both initialized and uninitialized model experiments. This makes the added value from initialization difficult to assess. We investigate statistical tests to quantify if initialized forecasts provide skill over the uninitialized experiments. We consider three correlation-based statistics previously used in the literature. The distributions of these statistics under the null hy-pothesis that initialization has no added values are calculated by a surrogate data method. We present some simple exam-ples and study the statistical power of the tests. We find that there can be large differences in both the values and power for the different statistics. In general, the simple statistic defined as the difference between the skill of the initialized and uninitialized experiments behaves best. However, for all statistics the risk of rejecting the true null hypothesis is too high compared to the nominal value. We compare the three tests on initialized decadal predictions (hindcasts) of near-surface temperature performed with a climate model and find evidence for a significant effect of initializations for small lead times. In contrast, we find only little evidence for a significant effect of initializations for lead times longer than 3 years when the experience from the simple experiments is included in the estimation.

KW - Statistical techniques

KW - Forecast verification

KW - skill

KW - Climate models

KW - NONLINEARITY

KW - SYSTEM

KW - POWER

U2 - 10.1175/JCLI-D-22-0605.1

DO - 10.1175/JCLI-D-22-0605.1

M3 - Journal article

VL - 36

SP - 2781

EP - 2793

JO - Journal of Climate

JF - Journal of Climate

SN - 0894-8755

IS - 9

ER -