| Title: | Generalized Discrimination Score |
|---|---|
| Description: | This is an implementation of the Generalized Discrimination Score (also known as Two Alternatives Forced Choice Score, 2AFC) for various representations of forecasts and verifying observations. The Generalized Discrimination Score is a generic forecast verification framework which can be applied to any of the following verification contexts: dichotomous, polychotomous (ordinal and nominal), continuous, probabilistic, and ensemble. A comprehensive description of the Generalized Discrimination Score, including all equations used in this package, is provided by Mason and Weigel (2009) <doi:10.1175/MWR-D-10-05069.1>. |
| Authors: | Andreas Weigel [aut], MeteoSwiss [cph], Jonas Bhend [cre, ctb] |
| Maintainer: | Jonas Bhend <[email protected]> |
| License: | GPL-3 |
| Version: | 1.4.0 |
| Built: | 2024-12-16 06:42:22 UTC |
| Source: | CRAN |
This package is a collection of routines to calculate the "generalized discrimination score", which is also known as "two alternatives forced-choice score" or short: "2AFC-score". The 2AFC is a generic forecast verification framework which can be applied to any of the following verification contexts: dichotomous, polychotomous (ordinal and nominal), continuous, probabilistic, and ensemble. A comprehensive description of the 2AFC-score, including all equations used in this package, is provided by Mason and Weigel (2009).
The master routine is afc. For a given set of observation and
forecast data, and for a specified verification context, afc
calls the appropriate functions which are necessary to calculate the 2AFC
score.
Why the 2AFC-score? There are numerous reasons for calculating forecast verification scores, and considerable attention has been given to designing and analyzing the properties of scores that can be used for scientific purposes. Much less attention has been given to scores that may be useful for administrative reasons, such as communicating changes in forecast quality to bureaucrats, and providing indications of forecast quality to the general public. The 2AFC test a scoring procedure that is sufficiently generic to be useable on forecasts ranging from simply "yes"/"no" forecasts of dichotomous outcomes to continuous variables, and can be used with deterministic or probabilistic forecasts without seriously reducing the more complex information when available. Although, as with any single verification score, the 2AFC has limitations, it does have broad intuitive appeal in that the expected score of an unskilled set of forecasts (random guessing or perpetually identical forecasts) is 50%, and is interpretable as an indication of how often the forecasts are correct, even when the forecasts are expressed probabilistically and/or the observations are not discrete.
| Package: | afc |
| Version: | 1.03 |
| Date: | 2010-01-07 |
| License: | GPL-2 |
afc Calculate Generalized Discrimination Score 2AFC
afc-package Generalized Discrimination Score 2AFC
afc.cc 2AFC For Continuous Observations And Continuous
Forecasts
afc.ce 2AFC For Ordinal Polychotomous Observations And
Ensemble Forecasts
afc.dc 2AFC For Dichotomous Observations And
Continuous Forecasts
afc.dd 2AFC For Dichotomous Observations And
Dichotomous Forecasts
afc.de 2AFC for Dichotomous Observations and Ensemble
Forecasts
afc.dm 2AFC For Dichotomous Observations And
Polychotomous Forecasts
afc.dp 2AFC For Dichotomous Observations And
Probabilistic Forecasts
afc.mc 2AFC For Ordinal Polychotomous Observations And
Continuous Forecasts
afc.me 2AFC For Ordinal Polychotomous Observations And
Ensemble Forecasts
afc.mm 2AFC For Ordinal Polychotomous Observations And
Ordinal Polychotomous Forecasts
afc.mp 2AFC For Ordinal Polychotomous Observations And
Probabilistic Forecasts
afc.nn 2AFC For Nominal Polychotomous Observations And
Nominal Polychotomous Forecasts
afc.np 2AFC For Nominal Polychotomous Observations And
Probabilistic Forecasts
cnrm.nino34.cc Example Data of Continuous Observations and
Continuous Forecasts
cnrm.nino34.ce Example Data of Continuous Observations and
Ensemble Forecasts
cnrm.nino34.dc Example Data of Dichotomous Observations and
Continuous Forecasts
cnrm.nino34.dd Example Data of Dichotomous Observations and
Dichotomous Forecasts
cnrm.nino34.de Example Data of Dichotomous Observations and
Ensemble Forecasts
cnrm.nino34.dm Example Data of Dichotomous Observations and
Polychotomous Forecasts
cnrm.nino34.dp Example Data of Dichotomous Observations and
Polychotomous Forecasts
cnrm.nino34.mc Example Data of Polychotomous Observations and
Continuous Forecasts
cnrm.nino34.me Example Data of Polychotomous Observations and
Ensembles Forecasts
cnrm.nino34.mm Example Data of Polychotomous Observations and
Polychotomous Forecasts
cnrm.nino34.mp Example Data of Polychotomous Observations and
Probabilistic Forecasts
rank.ensembles Rank Ensembles
Andreas Weigel, Federal Office of Meteorology and Climatology (MeteoSwiss), Zurich, Switzerland <[email protected]>
Mason, S.J. and A.P. Weigel, 2009: A generic forecast verification framework for administrative purposes. Mon. Wea. Rev., 137, 331-349
This is the master routine for the calculation the Generalized Discrimination Score (aka Two-Alternatives Forced Choice Score - 2AFC) as described in the Paper of Mason and Weigel (2009). This routine requires, as input, datasets of forecasts and corresponding observations, as well as a specification of the verification context. The routine checks whether the input data are consistent with the verification context, then calls the appropriate function to calculate the 2AFC, and finally returns the 2AFC skill value.
afc(obsv, fcst, obsv.type, fcst.type, m = 0, m2 = 0)afc(obsv, fcst, obsv.type, fcst.type, m = 0, m2 = 0)
obsv |
Vector of observations. The required format depends on the specific verification context. More details below. |
fcst |
Vector or two-dimensional array of forecasts. The required format depends on the specific verification context. More details below. |
obsv.type |
Character specifying the type of the observations. Possible values: "d" (dichotomous), "m" (polychotomous with ordinal categories), "n" (polychotomous with nominal categories), "c" (continuous). |
fcst.type |
Character specifying the type of the forecasts. Possible values: "d" (dichotomous), "m" (polychotomous with ordinal categories), "n" (polychotomous with nominal categories), "p" (probabilistic), "c" (continuous) and "e" (ensemble). |
m |
Number of observation or forecast categories (only required if obsv.type or fcst.type equals "m" or "n") |
m2 |
Number of forecast categories (only required if both obsv.type and fcst.type equal "m"). The number of observation categories is then specified by the argument m above. |
Depending on the specific verification context (i.e. the choice for
obsv.type and fcst.type), this routine calls the appropriate
function(s) to calculate the 2AFC score. The following combinations of
obsv.type and fcst.type are possible: (1) "d-d"; (2) "d-m";
(3) "d-p"; (4) "d-c"; (5) "d-e"; (6) "m-m"; (7) "m-p"; (8) "m-c"; (9) "m-e";
(10) "n-n"; (11) "n-p"; (12) "c-c"; (13) "c-e". The required format of the
input data obsv and fcst depends on the verification
context:
(1) "d-d":
obsv: vector with dichotomous observations (values in
0,1)
fcst: vector of same length as obsv with dichotomous
forecasts (values in 0,1)
(2) "d-m":
obsv: vector with dichotomous observations (values in
0,1)
fcst: vector of same length as obsv with
polychotomous forecasts (values in 1,..,m)
(3) "d-p":
obsv: vector with dichotomous observations (values in
0,1)
fcst: vector of same length as obsv with forecast
probabilities for the event to happen
(4) "d-c":
obsv: vector with dichotomous observations (values in
0,1)
fcst: vector of same length as obsv with real-valued
forecasts
(5) "d-e":
obsv: vector with dichotomous observations (values in
0,1)
fcst: two-dimensional array with ensemble forecasts;
dim(fcst)[1] = length(obsv); dim(fcst)[2] = ensemble size.
(6) "m-m":
obsv: vector with polychotomous observations (values
in 1,..,m)
fcst: vector of same length as obsv with
polychotomous forecasts (values in 1,..,m2)
(7) "m-p":
obsv: vector with polychotomous observations (values
in 1,..,m)
fcst: two-dimensional array with forecast
probabilities for the m categories; dim(fcst)[1] = length(obsv);
dim(fcst)[2] = m
(8) "m-c":
obsv: vector with polychotomous observations (values
in 1,..,m)
fcst: vector of same length as obsv with
real-valued forecasts
(9) "m-e":
obsv: vector with polychotomous observations (values
in 1,..,m)
fcst: two-dimensional array with ensemble forecasts;
dim(fcst)[1] = length(obsv); dim(fcst)[2] = ensemble size.
(10) "n-n":
obsv: vector with polychotomous observations (values
in 1,..,m)
fcst: vector of same length as obsv with
polychotomous forecasts (values in 1,..,m)
(11) "n-p":
Same as "m-p".
(12) "c-c":
obsv: vector with real-valued observations
fcst: vector of same length as obsv with real-valued forecasts
(13) "c-e":
obsv: vector with real-valued observations
fcst: two-dimensional array with ensemble forecasts; dim(fcst)[1] =
length(obsv); dim(fcst)[2] = ensemble size.
p.afc |
Value of Generalized Discrimination Score (2AFC) |
Andreas Weigel, Federal Office of Meteorology and Climatology, MeteoSwiss, Zurich, Switzerland
S.J. Mason and A.P. Weigel, 2009. A generic verification framework for administrative purposes. Mon. Wea. Rev., 137, 331-349
afc.dd afc.dm afc.dp
afc.dc afc.de afc.mm
afc.mp afc.mc afc.me
afc.nn afc.np afc.cc
afc.ce
# #In all following examples, forecasts of the Nino-3.4 index are evaluated # #---------------------- #Example 1: Dichotomous observations, dichotomous forecasts # --------------------- #Load set of dichotomous observations and dichotomous forecasts data(cnrm.nino34.dd) obsv = cnrm.nino34.dd$obsv fcst = cnrm.nino34.dd$fcst #Calculate skill score afc(obsv, fcst, obsv.type="d", fcst.type="d") # --------------------- #Example 2: Dichotomous observations, (ordinal) polychotomous forecasts # --------------------- #Load set of dichotomous observations and polychotomous forecasts (4 categories) data(cnrm.nino34.dm) obsv = cnrm.nino34.dm$obsv fcst = cnrm.nino34.dm$fcst #Calculate skill score afc(obsv, fcst, obsv.type="d", fcst.type="m", m=4) # --------------------- #Example 3: Dichotomous observations, probabilistic forecasts # --------------------- #Load set of dichotomous observations and probabilistic forecasts data(cnrm.nino34.dp) obsv = cnrm.nino34.dp$obsv fcst = cnrm.nino34.dp$fcst #Calculate skill score afc(obsv, fcst, obsv.type="d", fcst.type="p") # --------------------- #Example 4: Dichotomous observations, continuous forecasts # --------------------- #Load set of dichotomous observations and continuous forecasts data(cnrm.nino34.dc) obsv = cnrm.nino34.dc$obsv fcst = cnrm.nino34.dc$fcst #Calculate skill score afc(obsv, fcst, obsv.type="d", fcst.type="c") # --------------------- #Example 5: Dichotomous observations, ensemble forecasts # --------------------- #Load set of dichotomous observations and 9-member ensemble forecasts data(cnrm.nino34.de) obsv = cnrm.nino34.de$obsv fcst = cnrm.nino34.de$fcst #Calculate skill score afc(obsv, fcst, obsv.type="d", fcst.type="e") # --------------------- #Example 6: Polychotomous (ordinal) observations, polychotomous (ordinal) forecasts # --------------------- #Load set of polychotomous observations (4 categories) and polychotomous forecasts (4 categories) data(cnrm.nino34.mm) obsv = cnrm.nino34.mm$obsv fcst = cnrm.nino34.mm$fcst #Calculate skill score afc(obsv, fcst, obsv.type="m", fcst.type="m", m=4, m2=4) # --------------------- #Example 7: Polychotomous (ordinal) observations, probabilistic forecasts forecasts # --------------------- #Load set of polychotomous observations (4 categories) and probabilistic forecasts data(cnrm.nino34.mp) obsv = cnrm.nino34.mp$obsv fcst = cnrm.nino34.mp$fcst #Calculate skill score afc(obsv, fcst, obsv.type="m", fcst.type="p", m=4) # --------------------- #Example 8: Polychotomous (ordinal) observations, continuous forecasts # --------------------- #Load set of polychotomous observations (4 categories) and continuous forecasts data(cnrm.nino34.mc) obsv = cnrm.nino34.mc$obsv fcst = cnrm.nino34.mc$fcst #Calculate skill score afc(obsv, fcst, obsv.type="m", fcst.type="c", m=4) # --------------------- #Example 9: Polychotomous (ordinal) observations, ensemble forecasts # --------------------- #Load set of polychotomous observations (4 categories) and 9-member ensemble forecasts data(cnrm.nino34.me) obsv = cnrm.nino34.me$obsv fcst = cnrm.nino34.me$fcst #Calculate skill score afc(obsv, fcst, obsv.type="m", fcst.type="e", m=4) # --------------------- #Example 10: Polychotomous (nominal) observations, polychotomous (nominal) forecasts # --------------------- #Load set of polychotomous observations (4 categories) and polychotomous forecasts (4 categories) data(cnrm.nino34.mm) obsv = cnrm.nino34.mm$obsv fcst = cnrm.nino34.mm$fcst #Calculate skill score afc(obsv, fcst, obsv.type="n", fcst.type="n", m=4) # --------------------- #Example 11: Polychotomous (nominal) observations, probabilistic forecasts # --------------------- #Load set of polychotomous observations (4 categories) and probabilistic forecasts data(cnrm.nino34.mp) obsv = cnrm.nino34.mp$obsv fcst = cnrm.nino34.mp$fcst #Calculate skill score afc(obsv, fcst, obsv.type="n", fcst.type="p", m=4) # --------------------- #Example 12: Continuous observations, continuous forecasts # --------------------- #Load set of continuous observations and continuous forecasts data(cnrm.nino34.cc) obsv = cnrm.nino34.cc$obsv fcst = cnrm.nino34.cc$fcst #Calculate skill score afc(obsv, fcst, obsv.type="c", fcst.type="c") # --------------------- #Example 13: Continuous observations, ensemble forecasts # --------------------- #Load set of continuous observations and 9-member ensemble forecasts data(cnrm.nino34.ce) obsv = cnrm.nino34.ce$obsv fcst = cnrm.nino34.ce$fcst #Calculate skill score afc(obsv, fcst, obsv.type="c", fcst.type="e")# #In all following examples, forecasts of the Nino-3.4 index are evaluated # #---------------------- #Example 1: Dichotomous observations, dichotomous forecasts # --------------------- #Load set of dichotomous observations and dichotomous forecasts data(cnrm.nino34.dd) obsv = cnrm.nino34.dd$obsv fcst = cnrm.nino34.dd$fcst #Calculate skill score afc(obsv, fcst, obsv.type="d", fcst.type="d") # --------------------- #Example 2: Dichotomous observations, (ordinal) polychotomous forecasts # --------------------- #Load set of dichotomous observations and polychotomous forecasts (4 categories) data(cnrm.nino34.dm) obsv = cnrm.nino34.dm$obsv fcst = cnrm.nino34.dm$fcst #Calculate skill score afc(obsv, fcst, obsv.type="d", fcst.type="m", m=4) # --------------------- #Example 3: Dichotomous observations, probabilistic forecasts # --------------------- #Load set of dichotomous observations and probabilistic forecasts data(cnrm.nino34.dp) obsv = cnrm.nino34.dp$obsv fcst = cnrm.nino34.dp$fcst #Calculate skill score afc(obsv, fcst, obsv.type="d", fcst.type="p") # --------------------- #Example 4: Dichotomous observations, continuous forecasts # --------------------- #Load set of dichotomous observations and continuous forecasts data(cnrm.nino34.dc) obsv = cnrm.nino34.dc$obsv fcst = cnrm.nino34.dc$fcst #Calculate skill score afc(obsv, fcst, obsv.type="d", fcst.type="c") # --------------------- #Example 5: Dichotomous observations, ensemble forecasts # --------------------- #Load set of dichotomous observations and 9-member ensemble forecasts data(cnrm.nino34.de) obsv = cnrm.nino34.de$obsv fcst = cnrm.nino34.de$fcst #Calculate skill score afc(obsv, fcst, obsv.type="d", fcst.type="e") # --------------------- #Example 6: Polychotomous (ordinal) observations, polychotomous (ordinal) forecasts # --------------------- #Load set of polychotomous observations (4 categories) and polychotomous forecasts (4 categories) data(cnrm.nino34.mm) obsv = cnrm.nino34.mm$obsv fcst = cnrm.nino34.mm$fcst #Calculate skill score afc(obsv, fcst, obsv.type="m", fcst.type="m", m=4, m2=4) # --------------------- #Example 7: Polychotomous (ordinal) observations, probabilistic forecasts forecasts # --------------------- #Load set of polychotomous observations (4 categories) and probabilistic forecasts data(cnrm.nino34.mp) obsv = cnrm.nino34.mp$obsv fcst = cnrm.nino34.mp$fcst #Calculate skill score afc(obsv, fcst, obsv.type="m", fcst.type="p", m=4) # --------------------- #Example 8: Polychotomous (ordinal) observations, continuous forecasts # --------------------- #Load set of polychotomous observations (4 categories) and continuous forecasts data(cnrm.nino34.mc) obsv = cnrm.nino34.mc$obsv fcst = cnrm.nino34.mc$fcst #Calculate skill score afc(obsv, fcst, obsv.type="m", fcst.type="c", m=4) # --------------------- #Example 9: Polychotomous (ordinal) observations, ensemble forecasts # --------------------- #Load set of polychotomous observations (4 categories) and 9-member ensemble forecasts data(cnrm.nino34.me) obsv = cnrm.nino34.me$obsv fcst = cnrm.nino34.me$fcst #Calculate skill score afc(obsv, fcst, obsv.type="m", fcst.type="e", m=4) # --------------------- #Example 10: Polychotomous (nominal) observations, polychotomous (nominal) forecasts # --------------------- #Load set of polychotomous observations (4 categories) and polychotomous forecasts (4 categories) data(cnrm.nino34.mm) obsv = cnrm.nino34.mm$obsv fcst = cnrm.nino34.mm$fcst #Calculate skill score afc(obsv, fcst, obsv.type="n", fcst.type="n", m=4) # --------------------- #Example 11: Polychotomous (nominal) observations, probabilistic forecasts # --------------------- #Load set of polychotomous observations (4 categories) and probabilistic forecasts data(cnrm.nino34.mp) obsv = cnrm.nino34.mp$obsv fcst = cnrm.nino34.mp$fcst #Calculate skill score afc(obsv, fcst, obsv.type="n", fcst.type="p", m=4) # --------------------- #Example 12: Continuous observations, continuous forecasts # --------------------- #Load set of continuous observations and continuous forecasts data(cnrm.nino34.cc) obsv = cnrm.nino34.cc$obsv fcst = cnrm.nino34.cc$fcst #Calculate skill score afc(obsv, fcst, obsv.type="c", fcst.type="c") # --------------------- #Example 13: Continuous observations, ensemble forecasts # --------------------- #Load set of continuous observations and 9-member ensemble forecasts data(cnrm.nino34.ce) obsv = cnrm.nino34.ce$obsv fcst = cnrm.nino34.ce$fcst #Calculate skill score afc(obsv, fcst, obsv.type="c", fcst.type="e")
Routine to calculate the Generalized Discrimination Score (aka Two-Alternatives Forced Choice Score 2AFC) for the situation of continuous observations and continuous forecasts
afc.cc(obsv, fcst)afc.cc(obsv, fcst)
obsv |
vector with real-valued observations |
fcst |
vector of same length as obsv with real-valued forecasts |
This routine applies Eq.22 of Mason and Weigel (2009) to calculate the 2AFC.
p.afc |
Value of Generalized Discrimination (2AFC) Score |
Andreas Weigel, Federal Office of Meteorology and Climatology, MeteoSwiss, Zurich, Switzerland
S.J. Mason and A.P. Weigel, 2009. A generic verification framework for administrative purposes. Mon. Wea. Rev., 137, 331-349
#Forecasts and observations of Nino-3.4 index #Load set of continuous observations and continuous forecasts data(cnrm.nino34.cc) obsv = cnrm.nino34.cc$obsv fcst = cnrm.nino34.cc$fcst #Calculate skill score afc.cc(obsv,fcst)#Forecasts and observations of Nino-3.4 index #Load set of continuous observations and continuous forecasts data(cnrm.nino34.cc) obsv = cnrm.nino34.cc$obsv fcst = cnrm.nino34.cc$fcst #Calculate skill score afc.cc(obsv,fcst)
Routine to calculate the Generalized Discrimination Score (aka Two-Alternatives Forced Choice Score 2AFC) for the situation of continuous observations and ensemble forecasts
afc.ce(obsv, fcst)afc.ce(obsv, fcst)
obsv |
vector with real-valued observations |
fcst |
two-dimensional array with ensemble forecasts; dim(fcst)[1] = length(obsv); dim(fcst)[2] = ensemble size |
This routine first ranks the ensemble forecasts (see
rank.ensembles) and then calculates the 2AFC-score with Eq.22
of Mason and Weigel (2009).
p.afc |
Value of Generalized Discrimination (2AFC) Score |
Andreas Weigel, Federal Office of Meteorology and Climatology, MeteoSwiss, Zurich, Switzerland
S.J. Mason and A.P. Weigel, 2009. A generic verification framework for administrative purposes. Mon. Wea. Rev., 137, 331-349
#Forecasts and observations of Nino-3.4 index #Load set of continuous observations and 9-member ensemble forecasts data(cnrm.nino34.ce) obsv = cnrm.nino34.ce$obsv fcst = cnrm.nino34.ce$fcst #Calculate skill score afc.ce(obsv,fcst)#Forecasts and observations of Nino-3.4 index #Load set of continuous observations and 9-member ensemble forecasts data(cnrm.nino34.ce) obsv = cnrm.nino34.ce$obsv fcst = cnrm.nino34.ce$fcst #Calculate skill score afc.ce(obsv,fcst)
Routine to calculate the Generalized Discrimination Score (aka Two-Alternatives Forced Choice Score 2AFC) for the situation of dichotomous observations and continuous forecasts
afc.dc(obsv, fcst)afc.dc(obsv, fcst)
obsv |
vector with dichotomous observations (values in 0,1) |
fcst |
vector of same length as obsv with real-valued forecasts |
This routine applies Eq.8 of Mason and Weigel (2009) to calculate the 2AFC.
p.afc |
Value of Generalized Discrimination (2AFC) Score |
Andreas Weigel, Federal Office of Meteorology and Climatology, MeteoSwiss, Zurich, Switzerland
S.J. Mason and A.P. Weigel, 2009. A generic verification framework for administrative purposes. Mon. Wea. Rev., 137, 331-349
#Forecasts and observations of Nino-3.4 index #Load set of dichotomous observations and continuous forecasts data(cnrm.nino34.dc) obsv = cnrm.nino34.dc$obsv fcst = cnrm.nino34.dc$fcst #Calculate skill score afc.dc(obsv,fcst)#Forecasts and observations of Nino-3.4 index #Load set of dichotomous observations and continuous forecasts data(cnrm.nino34.dc) obsv = cnrm.nino34.dc$obsv fcst = cnrm.nino34.dc$fcst #Calculate skill score afc.dc(obsv,fcst)
Routine to calculate the Generalized Discrimination Score (aka Two-Alternatives Forced Choice Score 2AFC) for the situation of dichotomous observations and dichotomous forecasts
afc.dd(obsv, fcst)afc.dd(obsv, fcst)
obsv |
vector with dichotomous observations (values in 0,1) |
fcst |
vector of same length as obsv with dichotomous forecasts (values in 0,1) |
This routine applies Eq.2 of Mason and Weigel (2009) to calculate the 2AFC.
p.afc |
Value of Generalized Discrimination (2AFC) Score |
Andreas Weigel, Federal Office of Meteorology and Climatology, MeteoSwiss, Zurich, Switzerland
S.J. Mason and A.P. Weigel, 2009. A generic verification framework for administrative purposes. Mon. Wea. Rev., 137, 331-349
#Forecasts and observations of Nino-3.4 index #Load set of dichotomous observations and dichotomous forecasts data(cnrm.nino34.dd) obsv = cnrm.nino34.dd$obsv fcst = cnrm.nino34.dd$fcst #Calculate skill score afc.dd(obsv,fcst)#Forecasts and observations of Nino-3.4 index #Load set of dichotomous observations and dichotomous forecasts data(cnrm.nino34.dd) obsv = cnrm.nino34.dd$obsv fcst = cnrm.nino34.dd$fcst #Calculate skill score afc.dd(obsv,fcst)
Routine to calculate the Generalized Discrimination Score (aka Two-Alternatives Forced Choice Score 2AFC) for the situation of dichotomous observations and ensemble forecasts
afc.de(obsv, fcst)afc.de(obsv, fcst)
obsv |
vector with dichotomous observations (values in 0,1) |
fcst |
two-dimensional array with ensemble forecasts; dim(fcst)[1] = length(obsv); dim(fcst)[2] = ensemble size |
This routine first ranks the ensemble forecasts (see
rank.ensembles) and then calculates the 2AFC-score with Eq.2
of Mason and Weigel (2009).
p.afc |
Value of Generalized Discrimination (2AFC) Score |
Andreas Weigel, Federal Office of Meteorology and Climatology, MeteoSwiss, Zurich, Switzerland
S.J. Mason and A.P. Weigel, 2009. A generic verification framework for administrative purposes. Mon. Wea. Rev., 137, 331-349
#Forecasts and observations of Nino-3.4 index #Load set of dichotomous observations and 9-member ensemble forecasts data(cnrm.nino34.de) obsv = cnrm.nino34.de$obsv fcst = cnrm.nino34.de$fcst #Calculate skill score afc.de(obsv,fcst)#Forecasts and observations of Nino-3.4 index #Load set of dichotomous observations and 9-member ensemble forecasts data(cnrm.nino34.de) obsv = cnrm.nino34.de$obsv fcst = cnrm.nino34.de$fcst #Calculate skill score afc.de(obsv,fcst)
Routine to calculate the Generalized Discrimination Score (aka Two-Alternatives Forced Choice Score 2AFC) for the situation of dichotomous observations and (ordinal) polychotomous forecasts
afc.dm(obsv, fcst, mf = 3)afc.dm(obsv, fcst, mf = 3)
obsv |
vector with dichotomous observations (values in 0,1) |
fcst |
vector of same length as obsv with polychotomous forecasts (values in 1,..,m) |
mf |
number of forecast categories (default = 3) |
This routine applies Eq.5 of in Mason and Weigel (2009) to calculate the 2AFC.
p.afc |
Value of Generalized Discrimination (2AFC) Score |
Andreas Weigel, Federal Office of Meteorology and Climatology, MeteoSwiss, Zurich, Switzerland
S.J. Mason and A.P. Weigel, 2009. A generic verification framework for administrative purposes. Mon. Wea. Rev., 137, 331-349
#Forecasts and observations of Nino-3.4 index #Load set of dichotomous observations and polychotomous forecasts (4 categories) data(cnrm.nino34.dm) obsv = cnrm.nino34.dm$obsv fcst = cnrm.nino34.dm$fcst #Calculate skill score afc.dm(obsv,fcst,4)#Forecasts and observations of Nino-3.4 index #Load set of dichotomous observations and polychotomous forecasts (4 categories) data(cnrm.nino34.dm) obsv = cnrm.nino34.dm$obsv fcst = cnrm.nino34.dm$fcst #Calculate skill score afc.dm(obsv,fcst,4)
Routine to calculate the Generalized Discrimination Score (aka Two-Alternatives Forced Choice Score 2AFC) for the situation of dichotomous observations and discrete probabilistic forecasts
afc.dp(obsv, fcst)afc.dp(obsv, fcst)
obsv |
vector with dichotomous observations (values in 0,1) |
fcst |
vector of same length as obsv with forecast probabilities for the event to happen |
This routine applies Eq.5 of Mason and Weigel (2009) to calculate the 2AFC.
p.afc |
Value of Generalized Discrimination (2AFC) Score |
Andreas Weigel, Federal Office of Meteorology and Climatology, MeteoSwiss, Zurich, Switzerland
S.J. Mason and A.P. Weigel, 2009. A generic verification framework for administrative purposes. Mon. Wea. Rev., 137, 331-349
#Forecasts and observations of Nino-3.4 index #Load set of dichotomous observations and probabilistic forecasts data(cnrm.nino34.dp) obsv = cnrm.nino34.dp$obsv fcst = cnrm.nino34.dp$fcst #Calculate skill score afc.dp(obsv,fcst)#Forecasts and observations of Nino-3.4 index #Load set of dichotomous observations and probabilistic forecasts data(cnrm.nino34.dp) obsv = cnrm.nino34.dp$obsv fcst = cnrm.nino34.dp$fcst #Calculate skill score afc.dp(obsv,fcst)
Routine to calculate the Generalized Discrimination Score (aka Two-Alternatives Forced Choice Score 2AFC) for the situation of polychotomous observations (ordinal) and continuous forecasts
afc.mc(obsv, fcst, m = 3)afc.mc(obsv, fcst, m = 3)
obsv |
vector with polychotomous observations (values in 1,..,m) |
fcst |
vector of same length as obsv with real-valued forecasts |
m |
number of observation categories (default = 3) |
This routine applies Eq.18 of Mason and Weigel (2009) to calculate the 2AFC.
p.afc |
Value of Generalized Discrimination (2AFC) Score |
Andreas Weigel, Federal Office of Meteorology and Climatology, MeteoSwiss, Zurich, Switzerland
S.J. Mason and A.P. Weigel, 2009. A generic verification framework for administrative purposes. Mon. Wea. Rev., 137, 331-349
#Forecasts and observations of Nino-3.4 index #Load set of polychotomous observations (4 categories) and continuous forecasts data(cnrm.nino34.mc) obsv = cnrm.nino34.mc$obsv fcst = cnrm.nino34.mc$fcst #Calculate skill score afc.mc(obsv,fcst,4)#Forecasts and observations of Nino-3.4 index #Load set of polychotomous observations (4 categories) and continuous forecasts data(cnrm.nino34.mc) obsv = cnrm.nino34.mc$obsv fcst = cnrm.nino34.mc$fcst #Calculate skill score afc.mc(obsv,fcst,4)
Routine to calculate the Generalized Discrimination Score (aka Two-Alternatives Forced Choice Score 2AFC) for the situation of polychotomous observations (ordinal) and ensemble forecasts
afc.me(obsv, fcst, m = 3)afc.me(obsv, fcst, m = 3)
obsv |
vector with polychotomous observations (values in 1,..,m) |
fcst |
two-dimensional array with ensemble forecasts; dim(fcst)[1] = length(obsv); dim(fcst)[2] = ensemble size |
m |
number of observation categories (default = 3) |
This routine first ranks the ensemble forecasts (see
rank.ensembles) and then calculates the 2AFC-score with Eq.18
of Mason and Weigel (2009).
p.afc |
Value of Generalized Discrimination (2AFC) Score |
Andreas Weigel, Federal Office of Meteorology and Climatology, MeteoSwiss, Zurich, Switzerland
S.J. Mason and A.P. Weigel, 2009. A generic verification framework for administrative purposes. Mon. Wea. Rev., 137, 331-349
#Forecasts and observations of Nino-3.4 index #Load set of polychotomous observations (4 categories) and 9-member ensemble forecasts data(cnrm.nino34.me) obsv = cnrm.nino34.me$obsv fcst = cnrm.nino34.me$fcst #Calculate skill score afc.me(obsv,fcst,4)#Forecasts and observations of Nino-3.4 index #Load set of polychotomous observations (4 categories) and 9-member ensemble forecasts data(cnrm.nino34.me) obsv = cnrm.nino34.me$obsv fcst = cnrm.nino34.me$fcst #Calculate skill score afc.me(obsv,fcst,4)
Routine to calculate the Generalized Discrimination Score (aka Two-Alternatives Forced Choice Score 2AFC) for the situation of polychotomous observations (ordinal) and polychotomous forecasts (ordinal)
afc.mm(obsv, fcst, mv = 3, mf = 3)afc.mm(obsv, fcst, mv = 3, mf = 3)
obsv |
vector with polychotomous observations (values in 1,..,mv) |
fcst |
vector of same length as obsv with polychotomous forecasts (values in 1,..,mf) |
mv |
number of observation categories (default = 3) |
mf |
number of forecast categories (default = 3) |
This routine applies Eq.14 of Mason and Weigel (2009) to calculate the 2AFC.
p.afc |
Value of Generalized Discrimination (2AFC) Score |
Andreas Weigel, Federal Office of Meteorology and Climatology, MeteoSwiss, Zurich, Switzerland
S.J. Mason and A.P. Weigel, 2009. A generic verification framework for administrative purposes. Mon. Wea. Rev., 137, 331-349
#Forecasts and observations of Nino-3.4 index #Load set of polychotomous observations (4 categories) and polychotomous forecasts (4 categories) data(cnrm.nino34.mm) obsv = cnrm.nino34.mm$obsv fcst = cnrm.nino34.mm$fcst #Calculate skill score afc.mm(obsv,fcst,4,4)#Forecasts and observations of Nino-3.4 index #Load set of polychotomous observations (4 categories) and polychotomous forecasts (4 categories) data(cnrm.nino34.mm) obsv = cnrm.nino34.mm$obsv fcst = cnrm.nino34.mm$fcst #Calculate skill score afc.mm(obsv,fcst,4,4)
Routine to calculate the Generalized Discrimination Score (aka Two-Alternatives Forced Choice Score 2AFC) for the situation of polychotomous observations (ordinal) and discrete probabilistic forecasts
afc.mp(obsv, fcst, m = 3)afc.mp(obsv, fcst, m = 3)
obsv |
vector with polychotomous observations (values in 1,..,m) |
fcst |
two-dimensional array with forecast probabilities for the m categories; dim(fcst)[1] = length(obsv); dim(fcst)[2] = m |
m |
number of observation categories (default = 3) |
This routine applies Eq.16 of Mason and Weigel (2009) to calculate the 2AFC.
p.afc |
Value of Generalized Discrimination (2AFC) Score |
Andreas Weigel, Federal Office of Meteorology and Climatology, MeteoSwiss, Zurich, Switzerland
S.J. Mason and A.P. Weigel, 2009. A generic verification framework for administrative purposes. Mon. Wea. Rev., 137, 331-349
#Forecasts and observations of Nino-3.4 index #Load set of polychtomous observations (4 categories) and probabilistic forecasts data(cnrm.nino34.mp) obsv = cnrm.nino34.mp$obsv fcst = cnrm.nino34.mp$fcst #Calculate skill score afc.mp(obsv,fcst,4)#Forecasts and observations of Nino-3.4 index #Load set of polychtomous observations (4 categories) and probabilistic forecasts data(cnrm.nino34.mp) obsv = cnrm.nino34.mp$obsv fcst = cnrm.nino34.mp$fcst #Calculate skill score afc.mp(obsv,fcst,4)
Routine to calculate the Generalized Discrimination Score (aka Two-Alternatives Forced Choice Score 2AFC) for the situation of polychotomous observations (nominal) and polychotomous forecasts (nominal)
afc.nn(obsv, fcst, m = 3)afc.nn(obsv, fcst, m = 3)
obsv |
vector with polychotomous observations (values in 1,..,m) |
fcst |
vector of same length as obsv with polychotomous forecasts (values in 1,..,m) |
m |
number of observation and forecast categories (default = 3) |
This routine applies Eq.15 of Mason and Weigel (2009) to calculate the 2AFC.
p.afc |
Value of Generalized Discrimination (2AFC) Score |
Andreas Weigel, Federal Office of Meteorology and Climatology, MeteoSwiss, Zurich, Switzerland
S.J. Mason and A.P. Weigel, 2009. A generic verification framework for administrative purposes. Mon. Wea. Rev., 137, 331-349
#Forecasts and observations of Nino-3.4 index #Load set of polychotomous observations and polychotomous forecasts (4 categories) data(cnrm.nino34.mm) obsv = cnrm.nino34.mm$obsv fcst = cnrm.nino34.mm$fcst #Calculate skill score afc.nn(obsv,fcst,4)#Forecasts and observations of Nino-3.4 index #Load set of polychotomous observations and polychotomous forecasts (4 categories) data(cnrm.nino34.mm) obsv = cnrm.nino34.mm$obsv fcst = cnrm.nino34.mm$fcst #Calculate skill score afc.nn(obsv,fcst,4)
Routine to calculate the Generalized Discrimination Score (aka Two-Alternatives Forced Choice Score 2AFC) for the situation of polychotomous (nominal) observations and discrete probabilistic forecasts
afc.np(obsv, fcst, m = 3)afc.np(obsv, fcst, m = 3)
obsv |
vector with dichotomous observations (values in 0,1) |
fcst |
two-dimensional array with forecast probabilities for the m categories; dim(fcst)[1] = length(obsv); dim(fcst)[2] = m |
m |
number of observation categories (default = 3) |
This routine applies Eq.17 of Mason and Weigel (2009) to calculate the 2AFC.
p.afc |
Value of Generalized Discrimination (2AFC) Score |
Andreas Weigel, Federal Office of Meteorology and Climatology, MeteoSwiss, Zurich, Switzerland
S.J. Mason and A.P. Weigel, 2009. A generic verification framework for administrative purposes. Mon. Wea. Rev., 137, 331-349
#Forecasts and observations of Nino-3.4 index #Load set of polychotomous observations (4 categories) and probabilistic forecasts data(cnrm.nino34.mp) obsv = cnrm.nino34.mp$obsv fcst = cnrm.nino34.mp$fcst #Calculate skill score afc.np(obsv,fcst,4)#Forecasts and observations of Nino-3.4 index #Load set of polychotomous observations (4 categories) and probabilistic forecasts data(cnrm.nino34.mp) obsv = cnrm.nino34.mp$obsv fcst = cnrm.nino34.mp$fcst #Calculate skill score afc.np(obsv,fcst,4)
Continuous observations (obsv) and continuous forecasts (fcst) of the Nino-3.4 index for January 1961-2000. The forecast data stem from model runs initialized using data for the preceding August 1960-1999.
The format is:
List of 3
\$ years: int [1:40] 1961 1962 1963
1964 1965 1966 1967 1968 1969 1970 ...
\$ obsv : num [1:40] 26.2 26.2
25.9 27.4 25.8 ...
\$ fcst : num [1:40] 26.5 26.3 26.6 27.7 25.8 ...
The forecast data are from the coupled ocean-atmosphere model of the Centre National de Recherches Meteorologiques (CNRM) of Meteo France and were generated as part of the "Development of a European Multimodel Ensemble System for Seasonal to Interannual Prediction (DEMETER)" project (Palmer et al. 2004). Note that also the examples presented in the 2AFC paper by Mason and Weigel (2009) are based on these data.
Palmer, T.N. and Coauthors, 2004: Development of a European
ensemble system for seasonal to inter-annual prediction (DEMETER). Bull.
Amer. Meteor. Soc., 85, 853-872.
Mason, S.J. and A.P. Weigel, 2009: A
generic forecast verification framework for administrative purposes. Mon.
Wea. Rev., 137, 331-349
Continuous observations (obsv) and 9-member ensemble forecasts (fcst) of the Nino-3.4 index for January 1961-2000. The forecast data stem from model runs initialized using data for the preceding August 1960-1999.
The format is:
List of 3
\$ years: int [1:40] 1961 1962 1963
1964 1965 1966 1967 1968 1969 1970 ...
\$ obsv : num [1:40] 26.2 26.2
25.9 27.4 25.8 ...
\$ fcst : num [1:40, 1:9] 26.5 25.9 27.0 27.5 25.8 ...
The forecast data are from the coupled ocean-atmosphere model of the Centre National de Recherches Meteorologiques (CNRM) of Meteo France and were generated as part of the "Development of a European Multimodel Ensemble System for Seasonal to Interannual Prediction (DEMETER)" project (Palmer et al. 2004). Note that also the examples presented in the 2AFC paper by Mason and Weigel (2009) are based on these data.
Palmer, T.N. and Coauthors, 2004: Development of a European
ensemble system for seasonal to inter-annual prediction (DEMETER). Bull.
Amer. Meteor. Soc., 85, 853-872.
Mason, S.J. and A.P. Weigel, 2009: A
generic forecast verification framework for administrative purposes. Mon.
Wea. Rev., 137, 331-349
Dichotomous observations (obsv) and continuous forecasts (fcst) of the Nino-3.4 index for January 1961-2000. The forecast data stem from model runs initialized using data for the preceding August 1960-1999. Definition of an "event" in the observations: obsv >= 27 C.
The format is:
List of 3
\$ years: int [1:40] 1961 1962 1963
1964 1965 1966 1967 1968 1969 1970 ...
\$ obsv : num [1:40] 0 0 0 1 0 1 0
0 1 1 ...
\$ fcst : num [1:40] 26.5 26.3 26.6 27.7 25.8 ...
The forecast data are from the coupled ocean-atmosphere model of the Centre National de Recherches Meteorologiques (CNRM) of Meteo France and were generated as part of the "Development of a European Multimodel Ensemble System for Seasonal to Interannual Prediction (DEMETER)" project (Palmer et al. 2004). Note that also the examples presented in the 2AFC paper by Mason and Weigel (2009) are based on these data.
Palmer, T.N. and Coauthors, 2004: Development of a European
ensemble system for seasonal to inter-annual prediction (DEMETER). Bull.
Amer. Meteor. Soc., 85, 853-872.
Mason, S.J. and A.P. Weigel, 2009: A
generic forecast verification framework for administrative purposes. Mon.
Wea. Rev., 137, 331-349
Dichotomous observations (obsv) and dichotomous forecasts (fcst) of the Nino-3.4 index for January 1961-2000. The forecast data stem from model runs initialized using data for the preceding August 1960-1999. Definition of an "event" in the observations respectively forecasts: obsv/fcst >= 27 C.
The format is:
List of 3
\$ years: int [1:40] 1961 1962 1963
1964 1965 1966 1967 1968 1969 1970 ...
\$ obsv : num [1:40] 0 0 0 1 0 1 0
0 1 1 ...
\$ fcst : num [1:40] 0 0 0 1 0 1 0 0 1 1 ...
The forecast data are from the coupled ocean-atmosphere model of the Centre National de Recherches Meteorologiques (CNRM) of Meteo France and were generated as part of the "Development of a European Multimodel Ensemble System for Seasonal to Interannual Prediction (DEMETER)" project (Palmer et al. 2004). Note that also the examples presented in the 2AFC paper by Mason and Weigel (2009) are based on these data.
Palmer, T.N. and Coauthors, 2004: Development of a European
ensemble system for seasonal to inter-annual prediction (DEMETER). Bull.
Amer. Meteor. Soc., 85, 853-872.
Mason, S.J. and A.P. Weigel, 2009: A
generic forecast verification framework for administrative purposes. Mon.
Wea. Rev., 137, 331-349
Dichotomous observations (obsv) and 9-member ensemble forecasts (fcst) of the Nino-3.4 index for January 1961-2000. The forecast data stem from model runs initialized using data for the preceding August 1960-1999. Definition of an "event" in the observations: obsv >= 27 C.
The format is: List of 3
\$ years: int [1:40] 1961 1962 1963
1964 1965 1966 1967 1968 1969 1970 ...
\$ obsv : num [1:40] 0 0 0 1 0 1 0
0 1 1 ...
\$ fcst : num [1:40, 1:9] 26.5 25.9 27.0 27.5 25.8 ...
The forecast data are from the coupled ocean-atmosphere model of the Centre National de Recherches Meteorologiques (CNRM) of Meteo France and were generated as part of the "Development of a European Multimodel Ensemble System for Seasonal to Interannual Prediction (DEMETER)" project (Palmer et al. 2004). Note that also the examples presented in the 2AFC paper by Mason and Weigel (2009) are based on these data.
Palmer, T.N. and Coauthors, 2004: Development of a European
ensemble system for seasonal to inter-annual prediction (DEMETER). Bull.
Amer. Meteor. Soc., 85, 853-872.
Mason, S.J. and A.P. Weigel, 2009: A
generic forecast verification framework for administrative purposes. Mon.
Wea. Rev., 137, 331-349
Dichotomous observations (obsv) and polychotomous forecasts with four categories (fcst) of the Nino-3.4 index for January 1961-2000. The forecast data stem from model runs initialized using data for the preceding August 1960-1999. Definition of an "event" in the observations: obsv >= 27 C. Definition of the four forecast "categories": (1) fcst < 26 C; (2) fcst in [26 C,27 C[; (3) fcst in [27 C,28 C[; (4) fcst >= 28 C.
The format is:
List of 3
\$ years: int [1:40] 1961 1962 1963
1964 1965 1966 1967 1968 1969 1970 ...
\$ obsv : num [1:40] 0 0 0 1 0 1 0
0 1 1 ...
\$ fcst : num [1:40] 2 2 2 3 1 4 2 2 3 3 ...
The forecast data are from the coupled ocean-atmosphere model of the Centre National de Recherches Meteorologiques (CNRM) of Meteo France and were generated as part of the "Development of a European Multimodel Ensemble System for Seasonal to Interannual Prediction (DEMETER)" project (Palmer et al. 2004). Note that also the examples presented in the 2AFC paper by Mason and Weigel (2009) are based on these data.
Palmer, T.N. and Coauthors, 2004: Development of a European
ensemble system for seasonal to inter-annual prediction (DEMETER). Bull.
Amer. Meteor. Soc., 85, 853-872.
Mason, S.J. and A.P. Weigel, 2009: A
generic forecast verification framework for administrative purposes. Mon.
Wea. Rev., 137, 331-349
Dichotomous observations (obsv) and discrete probabilistic forecasts (fcst) of the Nino-3.4 index for January 1961-2000. The forecast data stem from model runs initialized using data for the preceding August 1960-1999. Definition of an "event" in the observations: obsv >= 27 C. The probabilistic forecasts have been estimated by taking the fraction of ensemble members predicting the event.
The format is:
List of 3
\$ years: int [1:40] 1961 1962 1963
1964 1965 1966 1967 1968 1969 1970 ...
\$ obsv : num [1:40] 0 0 0 1 0 1 0
0 1 1 ...
\$ fcst : num [1:40] 0.000 0.000 0.111 1.000 0.000 ...
The forecast data are from the coupled ocean-atmosphere model of the Centre National de Recherches Meteorologiques (CNRM) of Meteo France and were generated as part of the "Development of a European Multimodel Ensemble System for Seasonal to Interannual Prediction (DEMETER)" project (Palmer et al. 2004). Note that also the examples presented in the 2AFC paper by Mason and Weigel (2009) are based on these data.
Palmer, T.N. and Coauthors, 2004: Development of a European
ensemble system for seasonal to inter-annual prediction (DEMETER). Bull.
Amer. Meteor. Soc., 85, 853-872.
Mason, S.J. and A.P. Weigel, 2009: A
generic forecast verification framework for administrative purposes. Mon.
Wea. Rev., 137, 331-349
Polychotomous observations with four categories (obsv) and continuous forecasts (fcst) of the Nino-3.4 index for January 1961-2000. The forecast data stem from model runs initialized using data for the preceding August 1960-1999. Definition of the four observation "categories": (1) obsv < 26 C; (2) obsv in [26 C,27 C[; (3) obsv in [27 C,28 C[; (4) obsv >= 28 C.
The format is:
List of 3
\$ years: int [1:40] 1961 1962 1963
1964 1965 1966 1967 1968 1969 1970 ...
\$ obsv : num [1:40] 2 2 1 3 1 3 2
1 3 3 ...
\$ fcst : num [1:40] 26.5 26.3 26.6 27.7 25.8 ...
The forecast data are from the coupled ocean-atmosphere model of the Centre National de Recherches Meteorologiques (CNRM) of Meteo France and were generated as part of the "Development of a European Multimodel Ensemble System for Seasonal to Interannual Prediction (DEMETER)" project (Palmer et al. 2004). Note that also the examples presented in the 2AFC paper by Mason and Weigel (2009) are based on these data.
Palmer, T.N. and Coauthors, 2004: Development of a European
ensemble system for seasonal to inter-annual prediction (DEMETER). Bull.
Amer. Meteor. Soc., 85, 853-872.
Mason, S.J. and A.P. Weigel, 2009: A
generic forecast verification framework for administrative purposes. Mon.
Wea. Rev., 137, 331-349
Polychotomous observations with four categories (obsv) and 9-member ensemble forecasts (fcst) of the Nino-3.4 index for January 1961-2000. The forecast data stem from model runs initialized using data for the preceding August 1960-1999. Definition of the four observation "categories": (1) obsv < 26 C; (2) obsv in [26 C,27 C[; (3) obsv in [27 C,28 C[; (4) obsv >= 28 C.
The format is:
List of 3
\$ years: int [1:40] 1961 1962 1963
1964 1965 1966 1967 1968 1969 1970 ...
\$ obsv : num [1:40] 2 2 1 3 1 3 2
1 3 3 ...
\$ fcst : num [1:40, 1:9] 26.5 25.9 27.0 27.5 25.8 ...
The forecast data are from the coupled ocean-atmosphere model of the Centre National de Recherches Meteorologiques (CNRM) of Meteo France and were generated as part of the "Development of a European Multimodel Ensemble System for Seasonal to Interannual Prediction (DEMETER)" project (Palmer et al. 2004). Note that also the examples presented in the 2AFC paper by Mason and Weigel (2009) are based on these data.
Palmer, T.N. and Coauthors, 2004: Development of a European
ensemble system for seasonal to inter-annual prediction (DEMETER). Bull.
Amer. Meteor. Soc., 85, 853-872.
Mason, S.J. and A.P. Weigel, 2009: A
generic forecast verification framework for administrative purposes. Mon.
Wea. Rev., 137, 331-349
Polychotomous observations with four categories (obsv) and polychotomous forecasts with four categories (fcst) of the Nino-3.4 index for January 1961-2000. The forecast data stem from model runs initialized using data for the preceding August 1960-1999. Definition of the four observation respectively forecast "categories": (1) obsv/fcst < 26 C; (2) obsv/fcst in [26 C,27 C[; (3) obsv/fcst in [27 C,28 C[; (4) obsv >= 28 C.
The format is:
List of 3
\$ years: int [1:40] 1961 1962 1963
1964 1965 1966 1967 1968 1969 1970 ...
\$ obsv : num [1:40] 2 2 1 3 1 3 2
1 3 3 ...
\$ fcst : num [1:40] 2 2 2 3 1 4 2 2 3 3 ...
The forecast data are from the coupled ocean-atmosphere model of the Centre National de Recherches Meteorologiques (CNRM) of Meteo France and were generated as part of the "Development of a European Multimodel Ensemble System for Seasonal to Interannual Prediction (DEMETER)" project (Palmer et al. 2004). Note that also the examples presented in the 2AFC paper by Mason and Weigel (2009) are based on these data.
Palmer, T.N. and Coauthors, 2004: Development of a European
ensemble system for seasonal to inter-annual prediction (DEMETER). Bull.
Amer. Meteor. Soc., 85, 853-872.
Mason, S.J. and A.P. Weigel, 2009: A
generic forecast verification framework for administrative purposes. Mon.
Wea. Rev., 137, 331-349
Polychotomous observations with four categories (obsv) and discrete probabilistic forecasts (fcst) of the Nino-3.4 index for January 1961-2000. The forecast data stem from model runs initialized using data for the preceding August 1960-1999. Definition of the four observation "categories": (1) obsv < 26 C; (2) obsv in [26 C,27 C[; (3) obsv in [27 C,28 C[; (4) obsv >= 28 C. The probabilistic forecasts have been estimated by taking the fraction of ensemble members falling into each of the four categories.
The format is:
List of 3
\$ years: int [1:40] 1961 1962 1963
1964 1965 1966 1967 1968 1969 1970 ...
\$ obsv : num [1:40] 2 2 1 3 1 3 2
1 3 3 ...
\$ fcst : num [1:40, 1:4] 0.000 0.222 0.000 0.000 0.889 ...
The forecast data are from the coupled ocean-atmosphere model of the Centre National de Recherches Meteorologiques (CNRM) of Meteo France and were generated as part of the "Development of a European Multimodel Ensemble System for Seasonal to Interannual Prediction (DEMETER)" project (Palmer et al. 2004). Note that also the examples presented in the 2AFC paper by Mason and Weigel (2009) are based on these data.
Palmer, T.N. and Coauthors, 2004: Development of a European
ensemble system for seasonal to inter-annual prediction (DEMETER). Bull.
Amer. Meteor. Soc., 85, 853-872.
Mason, S.J. and A.P. Weigel, 2009: A
generic forecast verification framework for administrative purposes. Mon.
Wea. Rev., 137, 331-349
Routine to rank a set of given ensemble forecasts according to their "value"
rank.ensembles(fcst)rank.ensembles(fcst)
fcst |
two-dimensional array with ensemble forecasts; dim(fcst)[1] = number of ensemble forecasts; dim(fcst)[2] = number of ensemble members |
This routine ranks a set of ensemble forecasts according to their "value". The higher the "value" of an ensemble forecasts, the higher the rank. The following principle is applied: Assume two ensembles A and B are to be ranked. Without loss of generality, we define A>B if the probability of a random ensemble member of A being larger than a random ensemble member of B exceeds 0.5. This probability is calculated by a 2AFC-like approach based on Eq. 8 of Mason and Weigel (2009). By pairwise comparison of all ensembles, the final ranking is obtained.
ranks |
vector with the ranks of the ensemble forecasts |
Andreas Weigel, Federal Office of Meteorology and Climatology, MeteoSwiss, Zurich, Switzerland
S.J. Mason and A.P. Weigel, 2009. A generic verification framework for administrative purposes. Mon. Wea. Rev., 137, 331-349
#Load a set of ensemble forecasts data(cnrm.nino34.ce) fcst = cnrm.nino34.ce$fcst #Rank ensemble forecasts rank.ensembles(fcst)#Load a set of ensemble forecasts data(cnrm.nino34.ce) fcst = cnrm.nino34.ce$fcst #Rank ensemble forecasts rank.ensembles(fcst)