| Title: | Visualisation, Verification and Calibration of Ternary Probabilistic Forecasts |
|---|---|
| Description: | A suite of functions for visualising ternary probabilistic forecasts, as discussed in the paper by Jupp (2012) <doi:10.1098/rsta.2011.0350>. |
| Authors: | Tim Jupp |
| Maintainer: | Tim Jupp <[email protected]> |
| License: | GPL-2 |
| Version: | 1.2 |
| Built: | 2024-12-07 06:29:52 UTC |
| Source: | CRAN |
A suite of functions for visualising ternary probabilistic forecasts.
| Package: | ternvis |
| Type: | Package |
| Version: | 1.0 |
| Date: | 2012-03-29 |
| License: | GPL-2 |
Colours can be assigned to ternary probabilistic forecasts using tcolour.
These colours can be used to produce forecast maps as in the example function tmap.
A set of ternary forecasts p can be compared with subsequent ternary observations o using the function tverify. plot.tverify then displays this information in a Ternary Reliability Diagram. Calibration is performed using tgetcal and tcalibrate.
Tim Jupp
Maintainer: Tim Jupp <[email protected]>
Jupp TE, Lowe R, Stephenson DB, Coelho CAS (2012) On the visualization, verification and recalibration of ternary probabilistic forecasts, Philosophical Transactions of the Royal Society, volume 370, pages 1100-1120.
http://rsta.royalsocietypublishing.org/content/370/1962/1100.full
http://arxiv.org/abs/1103.1303
Some concepts adapted from those in package vcd.
data(foot) # see the distribution of forecasts tplot(foot$p,main="Bookmaker forecasts of \n football matches", dimnames=c("Home Win","Draw","Away Win")) # see how well forecasts compare with results # create object of class tverify foot.verify <- tverify(p=foot$p,o=foot$o) # plot ternary reliability diagram dev.new() plot(foot.verify, thresh=3) # get a (linear) calibration of these data # create an object of class tverify foot.calib <- tgetcal(foot.verify) # plot ternary reliability diagram of calibrated dev.new() plot(foot.calib, thresh=3) data(rain) tmap(rain,iyr=17,palette=TRUE,circles=FALSE,fac=10)data(foot) # see the distribution of forecasts tplot(foot$p,main="Bookmaker forecasts of \n football matches", dimnames=c("Home Win","Draw","Away Win")) # see how well forecasts compare with results # create object of class tverify foot.verify <- tverify(p=foot$p,o=foot$o) # plot ternary reliability diagram dev.new() plot(foot.verify, thresh=3) # get a (linear) calibration of these data # create an object of class tverify foot.calib <- tgetcal(foot.verify) # plot ternary reliability diagram of calibrated dev.new() plot(foot.calib, thresh=3) data(rain) tmap(rain,iyr=17,palette=TRUE,circles=FALSE,fac=10)
A collection of bookmakers' football odds (for home win/draw/away win, expressed as ternary probabilistic forecasts) and subsequent results. The data cover several years of English league games.
data(foot)data(foot)
The quoted odds, transformed to probabilities, are in foot$p (a 3-column matrix). The corresponding results of the games are in foot$o (a 3-column matrix).
Data taken from www.football-data.co.uk.
data(foot) # see the distribution of forecasts tplot(foot$p,main="Bookmaker forecasts of \n football matches", dimnames=c("Home Win","Draw","Away Win")) # see how well forecasts compare with results # create object of class tverify foot.verify <- tverify(p=foot$p,o=foot$o) # plot ternary reliability diagram dev.new() plot(foot.verify, thresh=3) # get a (linear) calibration of these data # create an object of class tverify foot.calib <- tgetcal(foot.verify) # plot ternary reliability diagram of calibrated forecasts dev.new() plot(foot.calib, thresh=3)data(foot) # see the distribution of forecasts tplot(foot$p,main="Bookmaker forecasts of \n football matches", dimnames=c("Home Win","Draw","Away Win")) # see how well forecasts compare with results # create object of class tverify foot.verify <- tverify(p=foot$p,o=foot$o) # plot ternary reliability diagram dev.new() plot(foot.verify, thresh=3) # get a (linear) calibration of these data # create an object of class tverify foot.calib <- tgetcal(foot.verify) # plot ternary reliability diagram of calibrated forecasts dev.new() plot(foot.calib, thresh=3)
An array of monthly seasonal precipitation forecasts and observations over South America, covering 25 years.
data(rain)data(rain)
A list containing 7 objects
rain$lats a vector of 35 latitudes
rain$lons a vector of 27 longitudes
rain$obs array of ternary observations
rain$preds array of ternary forecast probabilities
rain$rel array of reliabilities
rain$res array of resolutions
rain$q array of climatologies
Data from Caio Coelho at http://eurobrisa.cptec.inpe.br.
data(rain) ilon <- 5 # pick a longitude ilat <- 6 # pick a latitude iyr <- 17 # pick a year (in the range 1:25) rain$lons[ilon] # the longitude rain$lats[ilat] # the latitude rain$pred[ilon,ilat,iyr,] # the ternary forecast rain$obs[ilon,ilat,iyr,] # the ternary observationdata(rain) ilon <- 5 # pick a longitude ilat <- 6 # pick a latitude iyr <- 17 # pick a year (in the range 1:25) rain$lons[ilon] # the longitude rain$lats[ilat] # the latitude rain$pred[ilon,ilat,iyr,] # the ternary forecast rain$obs[ilon,ilat,iyr,] # the ternary observation
Function to apply a calibration to a matrix of ternary forecasts.
tcalibrate(tv, p)tcalibrate(tv, p)
tv |
An object of class tverify, for example produced by |
p |
An n-by-3 matrix of ternary forecasts which will be calibrated according to the calibration function embedded in dat. |
An n-by-3 array of calibrated ternary forecasts.
Tim Jupp
Jupp TE, Lowe R, Stephenson DB, Coelho CAS (2012) On the visualization, verification and recalibration of ternary probabilistic forecasts, Philosophical Transactions of the Royal Society, volume 370, pages 1100-1120.
http://rsta.royalsocietypublishing.org/content/370/1962/1100.full
http://arxiv.org/abs/1103.1303
# get football betting data data(foot) # get verification info for these data tv <- tverify(p=foot$p,o=foot$o) # get a linear calibration based on verification info lc <- tgetcal(tv) # consider an uncalibrated forecast p p <- cbind(0,0,1) # see where p is mapped by this calibration pcal <- tcalibrate(lc,cbind(0,0,1)) # plot the uncalibrated forecast tplot(p) # plot the calibrated forecast tplot(pcal,col="blue",newpage=FALSE,bg="transparent")# get football betting data data(foot) # get verification info for these data tv <- tverify(p=foot$p,o=foot$o) # get a linear calibration based on verification info lc <- tgetcal(tv) # consider an uncalibrated forecast p p <- cbind(0,0,1) # see where p is mapped by this calibration pcal <- tcalibrate(lc,cbind(0,0,1)) # plot the uncalibrated forecast tplot(p) # plot the calibrated forecast tplot(pcal,col="blue",newpage=FALSE,bg="transparent")
Function to assign a colour to a ternary forecast p, given a ternary climatology q.
tcolour(p = cbind(1, 1, 1)/3, q = cbind(1, 1, 1)/3, m = 0.7, flip = FALSE, dich = "none", theta0 = 0)tcolour(p = cbind(1, 1, 1)/3, q = cbind(1, 1, 1)/3, m = 0.7, flip = FALSE, dich = "none", theta0 = 0)
p |
An n-by-3 matrix of ternary forecasts. |
q |
A 1-by-3 matrix containing the climatology. |
m |
Exponent controlling rate of colour saturation. |
flip |
Logical to transpose categories "B" and "A" in barycentric coordinates. |
dich |
Option to render colours as they would appear to a dichromat. Default choice |
theta0 |
Angle (degrees) through which to rotate colour palette around climatology |
A colour, of the format produced by rgb.
Tim Jupp
Jupp TE, Lowe R, Stephenson DB, Coelho CAS (2012) On the visualization, verification and recalibration of ternary probabilistic forecasts, Philosophical Transactions of the Royal Society, volume 370, pages 1100-1120.
http://rsta.royalsocietypublishing.org/content/370/1962/1100.full
http://arxiv.org/abs/1103.1303
p <- cbind(0.7,0.2,0.1) # define a forecast q <- cbind(1,1,1)/3 # define a climatology pcol <- tcolour(p=p,q=q) # assign a colour pcol # print the colour code to terminal tplot(p,col=pcol) # use this colour in a plotp <- cbind(0.7,0.2,0.1) # define a forecast q <- cbind(1,1,1)/3 # define a climatology pcol <- tcolour(p=p,q=q) # assign a colour pcol # print the colour code to terminal tplot(p,col=pcol) # use this colour in a plot
Function to obtain a best-fitting calibration to a set of ternary forecasts p and corresponding observations o.
tgetcal(tv, quad = FALSE)tgetcal(tv, quad = FALSE)
tv |
An object of the tverify class (as produced by |
quad |
A logical describing which functional form to use for calibration. Linear calibration ( |
Tim Jupp
Jupp TE, Lowe R, Stephenson DB, Coelho CAS (2012) On the visualization, verification and recalibration of ternary probabilistic forecasts, Philosophical Transactions of the Royal Society, volume 370, pages 1100-1120.
http://rsta.royalsocietypublishing.org/content/370/1962/1100.full
http://arxiv.org/abs/1103.1303
data(foot) # see the distribution of forecasts tplot(foot$p,main="Bookmaker forecasts of \n football matches", dimnames=c("Home Win","Draw","Away Win")) # see how well forecasts compare with results # create object of class tverify foot.verify <- tverify(p=foot$p,o=foot$o) # plot ternary reliability diagram dev.new() plot(foot.verify, thresh=3) # get a (linear) calibration of these data # create an object of class tverify foot.calib <- tgetcal(foot.verify) # plot ternary reliability diagram of calibrated forecasts dev.new() plot(foot.calib, thresh=3)data(foot) # see the distribution of forecasts tplot(foot$p,main="Bookmaker forecasts of \n football matches", dimnames=c("Home Win","Draw","Away Win")) # see how well forecasts compare with results # create object of class tverify foot.verify <- tverify(p=foot$p,o=foot$o) # plot ternary reliability diagram dev.new() plot(foot.verify, thresh=3) # get a (linear) calibration of these data # create an object of class tverify foot.calib <- tgetcal(foot.verify) # plot ternary reliability diagram of calibrated forecasts dev.new() plot(foot.calib, thresh=3)
Utility function to generate a grid of ternary forecasts equally spaced in baycentric coordinates.
tgrid(n)tgrid(n)
n |
An integer describing the number of ‘bins’ along each side of the barycentric triangle. |
An n-by-3 matrix of ternary forecasts.
Tim Jupp
Jupp TE, Lowe R, Stephenson DB, Coelho CAS (2012) On the visualization, verification and recalibration of ternary probabilistic forecasts, Philosophical Transactions of the Royal Society, volume 370, pages 1100-1120.
http://rsta.royalsocietypublishing.org/content/370/1962/1100.full
http://arxiv.org/abs/1103.1303
tg <- tgrid(6) # make a grid of forecasts in the triangle tg # print forecasts to the terminal tplot(tg) # plot forecaststg <- tgrid(6) # make a grid of forecasts in the triangle tg # print forecasts to the terminal tplot(tg) # plot forecasts
An example of how one might make forecast maps using tpalette.
tmap(dat, iyr, circles = FALSE, fac = 10, theta0 = 0, dich = "none", m = 0.7, palette = TRUE, flip = FALSE)tmap(dat, iyr, circles = FALSE, fac = 10, theta0 = 0, dich = "none", m = 0.7, palette = TRUE, flip = FALSE)
dat |
An array of forecasts in a suitable format. See Example code below |
iyr |
In this example, an integer in the range 1 to 25. |
circles |
|
fac |
Scaling factor for size of circles when |
theta0 |
Angle to rotate colours on palette. See |
dich |
Optionally render colours as they would appear to a dichromat. See |
m |
Parameter controlling rate at which colours saturate in palette. See |
palette |
Logical. If |
flip |
Logical to choose whether to switch categories B and A. See |
Tim Jupp
Jupp TE, Lowe R, Stephenson DB, Coelho CAS (2012) On the visualization, verification and recalibration of ternary probabilistic forecasts, Philosophical Transactions of the Royal Society, volume 370, pages 1100-1120.
http://rsta.royalsocietypublishing.org/content/370/1962/1100.full
http://arxiv.org/abs/1103.1303
data(rain) tmap(rain,iyr=17,palette=TRUE,circles=FALSE,fac=10)data(rain) tmap(rain,iyr=17,palette=TRUE,circles=FALSE,fac=10)
Function to fill barycentric coordinates with a colour palette.
tpalette(q = cbind(1, 1, 1)/3, bars = TRUE, m = 0.7, flip = FALSE, lmain = FALSE, theta0 = 0, dich = "none", cex = 1, nhist = 12)tpalette(q = cbind(1, 1, 1)/3, bars = TRUE, m = 0.7, flip = FALSE, lmain = FALSE, theta0 = 0, dich = "none", cex = 1, nhist = 12)
q |
A 1-by-3 matrix containing a climatology. |
bars |
Logical. If "FALSE" a simple colour palette is produced. If "TRUE" barplots representing the forecasts are overlain. |
m |
Parameter controlling the rate at which colours saturate towards the corners of the triangle. |
flip |
Logical to transpose categories "B" and "A" in barycentric coordinates. |
lmain |
Logical. If "TRUE" text is overlaid. |
theta0 |
Angle (degrees) through which to rotate colour palette around climatology |
dich |
Option to render colours as they would appear to a dichromat. Default choice |
cex |
Factor for expanding the size of the dots when bars is "FALSE". |
nhist |
Number of barplots along each side of the triangle when bars is "TRUE". |
Tim Jupp
Jupp TE, Lowe R, Stephenson DB, Coelho CAS (2012) On the visualization, verification and recalibration of ternary probabilistic forecasts, Philosophical Transactions of the Royal Society, volume 370, pages 1100-1120.
http://rsta.royalsocietypublishing.org/content/370/1962/1100.full
http://arxiv.org/abs/1103.1303
dev.new() tpalette(bars=FALSE) # standard palette dev.new() tpalette(bars=TRUE) # palette with histograms overlaiddev.new() tpalette(bars=FALSE) # standard palette dev.new() tpalette(bars=TRUE) # palette with histograms overlaid
This function is almost exactly the same as the ternaryplot function in package vcd (on which it is based) except that the ordering of the vertices has been change to bottom left/top/bottom right, and the code has been extended work for an aribtrary scoring matrix L.
tplot(x = cbind(1, 1, 1)/3, L = diag(c(1, 1, 1))/sqrt(2), scale = 1, dimnames = NULL, dimnames_position = c("corner", "none"), dimnames_color = "black", id = NULL, id_color = "black", coordinates = FALSE, grid = TRUE, grid_color = "gray", labels = c("inside", "outside", "none"), labels_color = "darkgray", border = "grey", bg = "white", pch = 19, cex = 1, prop_size = FALSE, col = "red", main = "ternary plot", sub = NULL, newpage = TRUE, pop = TRUE, col.main = "black", col.sub = "black", ...)tplot(x = cbind(1, 1, 1)/3, L = diag(c(1, 1, 1))/sqrt(2), scale = 1, dimnames = NULL, dimnames_position = c("corner", "none"), dimnames_color = "black", id = NULL, id_color = "black", coordinates = FALSE, grid = TRUE, grid_color = "gray", labels = c("inside", "outside", "none"), labels_color = "darkgray", border = "grey", bg = "white", pch = 19, cex = 1, prop_size = FALSE, col = "red", main = "ternary plot", sub = NULL, newpage = TRUE, pop = TRUE, col.main = "black", col.sub = "black", ...)
x |
an n-by-3 matrix of ternary forecasts. |
L |
A matrix describing the quadratic score function. (Default corresponds to the Brier score). |
scale |
row sums scale to be used. |
dimnames |
dimension labels (defaults to the column names of |
dimnames_position, dimnames_color
|
position and color of dimension labels. |
id |
optional labels to be plotted below the plot symbols. |
id_color |
color of these labels. |
coordinates |
if |
grid |
if |
grid_color |
grid colour. |
labels, labels_color
|
position and colour of the grid labels. |
border |
colour of the triangle border. |
bg |
triangle background. |
pch |
plotting character. Defaults to filled dots. |
cex |
a numerical value giving the amount by which plotting text and symbols should be scaled relative to the default. Ignored for the symbol size if |
prop_size |
if |
col |
plotting colour. |
main |
main title. |
sub |
a sub title. |
newpage |
if ‘TRUE’, the plot will appear on a new graphics page. |
pop |
logical; if ‘TRUE’, all newly generated viewports are popped after plotting. |
col.main |
colour for the title. |
col.sub |
colour for the sub title. |
... |
additional graphics parameters (see |
Tim Jupp (using code for vcd:ternaryplot as source).
tg <- tgrid(6) # make a grid of forecasts in the triangle tg # print to terminal tplot(tg) # plot themtg <- tgrid(6) # make a grid of forecasts in the triangle tg # print to terminal tplot(tg) # plot them
Utility function to rescale 3-vectors to be probability distributions.
tscale(p)tscale(p)
p |
a vector of length 3. |
A utility function.
A vector of length 3 rescaled so that elements are all >= 0 and sum to unity.
Tim Jupp
tscale(cbind(1,2,3)) # rescale (1,2,3) to be a ternary forecast.tscale(cbind(1,2,3)) # rescale (1,2,3) to be a ternary forecast.
A utility function.
tscore(p, o, L = diag(c(1, 1, 1))/sqrt(2))tscore(p, o, L = diag(c(1, 1, 1))/sqrt(2))
p |
An n-by-3 matrix of ternary forecasts. |
o |
An n-by-3 matrix of ternary observations. |
L |
A 3-by-3 matrix defining the quadratic score being used. (Default corresponds to Brier score.) |
Tim Jupp
Jupp TE, Lowe R, Stephenson DB, Coelho CAS (2012) On the visualization, verification and recalibration of ternary probabilistic forecasts, Philosophical Transactions of the Royal Society, volume 370, pages 1100-1120.
http://rsta.royalsocietypublishing.org/content/370/1962/1100.full
http://arxiv.org/abs/1103.1303
data(foot) tscore(p=foot$p,o=foot$o)data(foot) tscore(p=foot$p,o=foot$o)
This is just a useful internal function.
tsetup(L = diag(c(1, 1, 1))/sqrt(2), q = cbind(1, 1, 1)/3)tsetup(L = diag(c(1, 1, 1))/sqrt(2), q = cbind(1, 1, 1)/3)
L |
A 3-by-3 matrix defining the quadratic score being used. (Default corresponds to Brier score.) |
q |
A 1-by-3 matrix containing the climatology. |
Tim Jupp
Jupp TE, Lowe R, Stephenson DB, Coelho CAS (2012) On the visualization, verification and recalibration of ternary probabilistic forecasts, Philosophical Transactions of the Royal Society, volume 370, pages 1100-1120.
http://rsta.royalsocietypublishing.org/content/370/1962/1100.full
http://arxiv.org/abs/1103.1303
tsetup() # returns a set of useful objectstsetup() # returns a set of useful objects
Function to create verification information associated with a set of forecasts p and associated observations o.
tverify(p, o, q, forceq, ncirc, L) ## Default S3 method: tverify(p = cbind(1, 1, 1)/3, o = cbind(0, 0, 1), q = cbind(1, 1, 1)/3, forceq = FALSE, ncirc = 11, L = diag(c(1, 1, 1))/sqrt(2)) ## S3 method for class 'tverify' plot(x, thresh = 0, lsharp = TRUE, L = diag(c(1, 1, 1))/sqrt(2),...)tverify(p, o, q, forceq, ncirc, L) ## Default S3 method: tverify(p = cbind(1, 1, 1)/3, o = cbind(0, 0, 1), q = cbind(1, 1, 1)/3, forceq = FALSE, ncirc = 11, L = diag(c(1, 1, 1))/sqrt(2)) ## S3 method for class 'tverify' plot(x, thresh = 0, lsharp = TRUE, L = diag(c(1, 1, 1))/sqrt(2),...)
p |
An n-by-3 matrix of ternary forecasts. |
o |
An n-by-3 matrix of ternary observations. |
q |
A 1-by-3 matrix containing the climatology. |
forceq |
A logical. If |
ncirc |
Controls discretisation of barycentric coordinates. (The number of hexagons along each side of the triangle.) |
x |
An object of the tverify class (as produced by |
thresh |
A ‘threshold’ number of forecasts in a ‘bin’, below which the forecast will not be plotted. (Prevents diagram being cluttered by very rare forecasts). |
lsharp |
Logical controlling whether a ternary sharpness diagram is to be included. |
L |
A quadratic scoring matrix. The matrix for the Brier score is the default. Full functionality for other scores not yet implemented. |
... |
Not used. |
An object of class tverify, which is a list containing the following components:
pbin |
array of bins in barycentric coordinates. |
Nobs |
number of forecasts in each bin. |
obar |
mean onservation associated with each forecast bin. |
score |
score associated with each forecast bin. |
unc |
uncertainty associated with each forecast bin. |
rel |
reliability associated with each forecast bin. |
res |
resolution associated with each forecast bin. |
scorebar |
overall score. |
uncbar |
overall uncertainty. |
relbar |
overall reliability. |
resbar |
overall resolution. |
ncirc |
number of bins along each side of triangle. |
p |
input forecast data. |
o |
input observation data. |
assigned |
index of bin assigned to each forecast. |
L |
scoring matrix used. |
hexc |
array of hexagon centres. |
q |
climatology. |
pk |
forecasts set to centre of each bin ( |
ok |
mean observations corresponding to forecasts |
pars |
parameters in calibration ( |
opt |
details of calibration ( |
f |
calibration function ( |
Tim Jupp
Jupp TE, Lowe R, Stephenson DB, Coelho CAS (2012) On the visualization, verification and recalibration of ternary probabilistic forecasts, Philosophical Transactions of the Royal Society, volume 370, pages 1100-1120.
http://rsta.royalsocietypublishing.org/content/370/1962/1100.full
http://arxiv.org/abs/1103.1303
data(foot) # get football betting data tv <- tverify(p=foot$p,o=foot$o) # get verification info plot(tv) # plot verification infodata(foot) # get football betting data tv <- tverify(p=foot$p,o=foot$o) # get verification info plot(tv) # plot verification info
This is a utility function to calculate the x-coordinate of a point in barycentric coordinates. It is unlikely that the user will want to use this function directly.
xf(p = cbind(1, 1, 1)/3, M = tsetup()$M32)xf(p = cbind(1, 1, 1)/3, M = tsetup()$M32)
p |
An n-by-3 matrix of ternary forecasts. |
M |
A 2-by-3 matrix describing the transfromation for R^3 to R^2. |
An n-by-1 array of x-values for use in plotting.
Tim Jupp
Jupp TE, Lowe R, Stephenson DB, Coelho CAS (2012) On the visualization, verification and recalibration of ternary probabilistic forecasts, Philosophical Transactions of the Royal Society, volume 370, pages 1100-1120.
http://rsta.royalsocietypublishing.org/content/370/1962/1100.full
http://arxiv.org/abs/1103.1303
xf(p=cbind(.1,.2,.7))xf(p=cbind(.1,.2,.7))
This is a utility function to calculate the y-coordinate of a point in barycentric coordinates. It is unlikely that the user will want to use this function directly.
yf(p = cbind(1, 1, 1)/3, M = tsetup()$M32)yf(p = cbind(1, 1, 1)/3, M = tsetup()$M32)
p |
An n-by-3 matrix of ternary forecasts. |
M |
A 2-by-3 matrix describing the transfromation for R^3 to R^2. |
An n-by-1 array of y-values for use in plotting.
Tim Jupp
Jupp TE, Lowe R, Stephenson DB, Coelho CAS (2012) On the visualization, verification and recalibration of ternary probabilistic forecasts, Philosophical Transactions of the Royal Society, volume 370, pages 1100-1120.
http://rsta.royalsocietypublishing.org/content/370/1962/1100.full
http://arxiv.org/abs/1103.1303
yf(p=cbind(.1,.2,.7))yf(p=cbind(.1,.2,.7))