| Title: | Intraclass Correlations for Quantifying Inter-Rater Reliability |
|---|---|
| Description: | Calculates various intraclass correlation coefficients used to quantify inter-rater and intra-rater reliability. The assumption here is that the raters produced quantitative ratings. Most of the statistical procedures implemented in this package are described in details in Gwet, K.L. (2014, ISBN:978-0970806284): "Handbook of Inter-Rater Reliability," 4th edition, Advanced Analytics, LLC. |
| Authors: | Kilem L. Gwet, Ph.D. |
| Maintainer: | Kilem L. Gwet <[email protected]> |
| License: | GPL (>= 2) |
| Version: | 1.0 |
| Built: | 2024-12-12 07:01:09 UTC |
| Source: | CRAN |
This function computes the lower and upper confidence bounds associated with the ICC under the one-factor ANOVA model where each subject may be rated by a different group of raters.
ci.ICC1a(ratings, conflev = 0.95)ci.ICC1a(ratings, conflev = 0.95)
ratings |
This is a data frame containing 3 columns or more. The first column contains subject numbers (there could be duplicates if a subject was assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
conflev |
This is the optional confidence level associated with the confidence interval. If not specified, the default value will be 0.95, which is the most commonly-used valuee in the literature. |
This function returns a vector containing the lower confidence (lcb) and the upper confidence bound (ucb).
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. chapter 8, section 8.3.1, equations 8.3.1 and 8.3.2. Advanced Analytics, LLC.
#iccdata3 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata3 #see what the iccdata3 dataset looks like ci.ICC1a(iccdata3)#iccdata3 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata3 #see what the iccdata3 dataset looks like ci.ICC1a(iccdata3)
This function computes the lower and upper confidence bounds associated with the ICC under the one-factor ANOVA model where each rater may rate a different group of subjects.
ci.ICC1b(ratings, conflev = 0.95)ci.ICC1b(ratings, conflev = 0.95)
ratings |
This is a data frame containing 3 columns or more. The first column contains subject numbers (there could be duplicates if a subject was assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
conflev |
This is the optional confidence level associated with the confidence interval. If not specified, the default value will be 0.95, which is the most commonly-used valuee in the literature. |
This function returns a vector containing the following the lower confidence (lcb) and the upper confidence bound (ucb).
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. chapter 8, section 8.3.4, equations 8.3.5 and 8.3.6. Advanced Analytics, LLC.
#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata3 dataset looks like ci.ICC1b(iccdata1)#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata3 dataset looks like ci.ICC1b(iccdata1)
This function computes the confidence interval of the Intraclass Correlation Coefficient ICCa(2,1) under the random factorial ANOVA model with subject-rater interaction. ICCa(2,1) is formulated as a measure of intra-rater reliability coefficient. This function computes the lower and upper confidence bounds of the confidence interval.
ci.ICC2a.inter(dfra, conflev = 0.95)ci.ICC2a.inter(dfra, conflev = 0.95)
dfra |
This is a data frame containing 3 columns or more. The first column contains subject numbers (there could be duplicates if a subject was assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
conflev |
This is the optional confidence level associated with the confidence interval. If not specified, the default value will be 0.95, which is the most commonly-used valuee in the literature. |
This function returns a vector containing the lower confidence (lcb) and the upper confidence bound (ucb).
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. chapter 9, section 9.3.2, equations 9.3.7 and 9.3.8. Advanced Analytics, LLC.
#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like ci.ICC2a.inter(iccdata1) ci.ICC2a.inter(iccdata1)$ucb #this only gives the upper confidence bound ci.ICC2a.inter(iccdata1,0.90) #this gives you the 90% confidence interval#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like ci.ICC2a.inter(iccdata1) ci.ICC2a.inter(iccdata1)$ucb #this only gives the upper confidence bound ci.ICC2a.inter(iccdata1,0.90) #this gives you the 90% confidence interval
This function computes the confidence interval associated with the Intraclass Correlation Coefficient (ICC) formulated as a measure of Intra-Rater Reliability under the random factorial ANOVA model (Model 2) without subject-rater interaction. This function produces the lower and upper confidence bounds.
ci.ICC2a.nointer(dfra, conflev = 0.95)ci.ICC2a.nointer(dfra, conflev = 0.95)
dfra |
This is a data frame containing 3 columns or more. The first column contains subject numbers (there could be duplicates if a subject was assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
conflev |
This is the optional confidence level associated with the confidence interval. If not specified, the default value will be 0.95, which is the most commonly-used valuee in the literature. |
This function returns a vector containing the lower confidence (lcb) and the upper confidence bound (ucb).
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. chapter 9, section 9.5.1, equations 9.5.11 and 9.5.12,page 259. Advanced Analytics, LLC.
#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like ci.ICC2a.nointer(iccdata1)#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like ci.ICC2a.nointer(iccdata1)
This function computes the confidence interval associated with the Intraclass Correlation Coefficient (ICC) used as a measure of inter-rater reliability, under the Random Factorial ANOVA model with interaction. It produces the lower and upper confidence bounds.
ci.ICC2r.inter(dfra, conflev = 0.95)ci.ICC2r.inter(dfra, conflev = 0.95)
dfra |
This is a data frame containing 3 columns or more. The first column contains subject numbers (there could be duplicates if a subject was assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
conflev |
This is the optional confidence level associated with the confidence interval. If not specified, the default value will be 0.95, which is the most commonly-used valuee in the literature. |
This function returns a vector containing the lower confidence (lcb) and the upper confidence bound (ucb).
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. chapter 9, section 9.3.1, equations 9.3.1 and 9.3.2. Advanced Analytics, LLC.
#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like ci.ICC2r.inter(iccdata1)#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like ci.ICC2r.inter(iccdata1)
This function computes the confidence interval associateed with the Intraclass Correlation Coefficient (ICC) used as a measure of inter-rater reliability, under the random factorial ANOVA model (Model 2) with no subject-rater interaction. This function computes the lower and upper confidence bounds.
ci.ICC2r.nointer(dfra, conflev = 0.95)ci.ICC2r.nointer(dfra, conflev = 0.95)
dfra |
This is a data frame containing 3 columns or more. The first column contains subject numbers (there could be duplicates if a subject was assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
conflev |
This is the optional confidence level associated with the confidence interval. If not specified, the default value will be 0.95, which is the most commonly-used valuee in the literature. |
This function returns a vector containing the lower confidence (lcb) and the upper confidence bound (ucb).
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. chapter 9, section 9.5.1, equations 9.5.7 and 9.5.8 for inter-rater reliability coefficients. Advanced Analytics, LLC.
#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like ci.ICC2r.nointer(iccdata1)#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like ci.ICC2r.nointer(iccdata1)
This function computes the confidence interval of the Intraclass Correlation Coefficient ICCa(3,1) under the mixed factorial ANOVA model (Model 3) with subject-rater interaction. ICCa(3,1) is formulated as a measure of intra-rater reliability coefficient. This function computes the lower and upper confidence bounds of the confidence interval.
ci.ICC3a.inter(dfra, conflev = 0.95)ci.ICC3a.inter(dfra, conflev = 0.95)
dfra |
This is a data frame containing 3 columns or more. The first column contains subject numbers (there could be duplicates if a subject was assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
conflev |
This is the optional confidence level associated with the confidence interval. If not specified, the default value will be 0.95, which is the most commonly-used valuee in the literature. |
This function returns a vector containing the lower confidence (lcb) and the upper confidence bound (ucb).
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. chapter 10, section 10.3.2, equations 10.3.10 and 10.3.11. Advanced Analytics, LLC.
#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like ci.ICC3a.inter(iccdata1) ci.ICC3a.inter(iccdata1)$ucb #this only gives the upper confidence bound ci.ICC3a.inter(iccdata1,0.90) #this gives you the 90% confidence interval#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like ci.ICC3a.inter(iccdata1) ci.ICC3a.inter(iccdata1)$ucb #this only gives the upper confidence bound ci.ICC3a.inter(iccdata1,0.90) #this gives you the 90% confidence interval
This function computes the confidence interval associated with the Intraclass Correlation Coefficient (ICC) as a measure of inter-rater reliability under the mixed factorial ANOVA model with interaction. It produces the lower and upper confidence bounds.
ci.ICC3r.inter(dfra, conflev = 0.95)ci.ICC3r.inter(dfra, conflev = 0.95)
dfra |
This is a data frame containing 3 columns or more. The first column contains subject numbers (there could be duplicates if a subject was assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
conflev |
This is the optional confidence level associated with the confidence interval. If not specified, the default value will be 0.95, which is the most commonly-used valuee in the literature. |
This function returns a vector containing the lower confidence (lcb) and the upper confidence bound (ucb).
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. chapter 10, section 10.3.1, equations 10.3.1 and 10.3.2. Advanced Analytics, LLC.
#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like ci.ICC3r.inter(iccdata1) ci.ICC3r.inter(iccdata1)$ucb #to get upper confidence bound only#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like ci.ICC3r.inter(iccdata1) ci.ICC3r.inter(iccdata1)$ucb #to get upper confidence bound only
This function computes the confidence interval associateed with the Intraclass Correlation Coefficient (ICC) used as a measure of inter-rater reliability, under the mixed factorial ANOVA model (Model 3) with no subject-rater interaction. This function computes the lower and upper confidence bounds.
ci.ICC3r.nointer(dfra, conflev = 0.95)ci.ICC3r.nointer(dfra, conflev = 0.95)
dfra |
This is a data frame containing 3 columns or more. The first column contains subject numbers (there could be duplicates if a subject was assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
conflev |
This is the optional confidence level associated with the confidence interval. If not specified, the default value will be 0.95, which is the most commonly-used valuee in the literature. |
This function returns a vector containing the lower confidence (lcb) and the upper confidence bound (ucb).
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. chapter 10, section 10.3.1, equations 10.3.6 and 10.3.7, Advanced Analytics, LLC.
#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like ci.ICC3r.nointer(iccdata1)#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like ci.ICC3r.nointer(iccdata1)
This ICC is associated with the one-factor ANOVA model where each subject could be rated by a different group of raters. This ICC represents a measure of inter-rater reliability among all raters involved in the experiment.
icc1a.fn(ratings)icc1a.fn(ratings)
ratings |
This is a data frame containing 3 columns or more. The first column contains subject numbers (there could be duplicates if a subject was assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
This function returns a list containing the following 9 values: 1. sig2s: the subject variance component. 2. sig2e: the error variance component. 3. icc1a: the ICC/inter-rater reliability coefficient 4. n: the number of subjects. 5. r: the number of raters. 6. max.rep: the maximum number of ratings per subject. 7. min.rep: the minimum number of ratings per subjects. 8. M: the total number of ratings for all subjects and raters. 9. ov.mean: the overall mean rating.
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. - Equation #8.1.3, chapter 8. Advanced Analytics, LLC.
#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like icc1a.fn(iccdata1) coeff <- icc1a.fn(iccdata1)$icc1a coeff#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like icc1a.fn(iccdata1) coeff <- icc1a.fn(iccdata1)$icc1a coeff
This ICC is associated with the one-factor ANOVA model where each rater may rate a different group of subjects. This ICC represents a global measure of intra-rater reliability coefficient for all raters involved in the experiment.
icc1b.fn(ratings)icc1b.fn(ratings)
ratings |
This is a data frame containing 3 columns or more. The first column contains subject numbers (there could be duplicates in subject numbers if a subject was rated multiple times) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
This function returns a list containing the following 9 values: 1. sig2r: the rater variance component. 2. sig2e: the error variance component. 3. icc1b: the ICC/intra-rater reliability coefficient 4. n: the number of subjects. 5. r: the number of raters. 6. max.rep: the maximum number of ratings per subject. 7. min.rep: the minimum number of ratings per subjects. 8. M: the total number of ratings for all subjects and raters. 9. ov.mean: the overall mean rating.
See equation 8.2.4 of chapter 8 in Gwet, 2014: Handbook of Inter-Rater Reliability - 4th ed.
#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like icc1b.fn(iccdata1) coeff <- icc1b.fn(iccdata1)$icc1b #this only gives you the ICC coefficient coeff#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like icc1b.fn(iccdata1) coeff <- icc1b.fn(iccdata1)$icc1b #this only gives you the ICC coefficient coeff
This functions computes 2 ICC estimates for the inter-rater reliability and intra-rater reliability coefficients. It requires some subjects to have multiple ratings and assumes the ANOVA model was specified with interaction.
icc2.inter.fn(ratings)icc2.inter.fn(ratings)
ratings |
This is a data frame containing 3 columns or more. The first column contains subject numbers (some duplicates are expected, as some subject are assumed to have assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
This function returns a list containing the following 12 values: 1. sig2s: the subject variance component. 2.sig2r: the rater variance component 3. sig2e: the error variance component. 4. sig2sr: the subject-rater interaction variance component. 5. icc2r: ICC as a measure of inter-rater relliability. 6. icc2a: ICC as a measure of intra-rater reliability. 7. n: the number of subjects. 8. r: the number of raters. 9. max.rep: the maximum number of ratings per subject. 10. min.rep: the minimum number of ratings per subjects. 11. M: the total number of ratings for all subjects and raters. 12. ov.mean: the overall mean rating.
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. - Equation #9.2.3 of chapter 9, pages 231-232 (for the inter-rater reliability ICC(2,1)), and Equation #9.2.10 of chapter 9, page 236 (for the intra-rater reliability), Advanced Analytics, LLC.
#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like icc2.inter.fn(iccdata1) coeff <- icc2.inter.fn(iccdata1)$icc2r #this only gives you the ICC coefficient coeff#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like icc2.inter.fn(iccdata1) coeff <- icc2.inter.fn(iccdata1)$icc2r #this only gives you the ICC coefficient coeff
This function computes 2 Intraclass Correlation Coefficients (ICC) ICC(2,1) and ICCa(2,1) under the random factorial ANOVA model (Model 2) without any subject-rater interaction. ICC(2,1) is formulated as a measure of inter-rater reliability and ICCa(2,1) as a measure of intra-rater reliability.
icc2.nointer.fn(ratings)icc2.nointer.fn(ratings)
ratings |
This is a data frame containing 3 columns or more. The first column contains subject numbers (some duplicates are expected, as some subject are assumed to have assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
This function returns a list containing the following 11 values:
1. sig2s: the subject variance component.
2.sig2r: the rater variance component
3. sig2e: the error variance component.
4. icc2r: ICC as a measure of inter-rater relliability.
5. icc2a: ICC as a measure of intra-rater reliability.
6. n: the number of subjects.
7. r: the number of raters.
8. max.rep: the maximum number of ratings per subject.
9. min.rep: the minimum number of ratings per subjects.
10. M: the total number of ratings for all subjects and raters.
11. ov.mean: the overall mean rating.
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. - Equations 9.5.2 and 9.5.3 of chapter 9, page 258. Advanced Analytics, LLC.
#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like icc2.nointer.fn(iccdata1) coeff <- icc2.nointer.fn(iccdata1)$icc2r #this only gives you the ICC coefficient coeff#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like icc2.nointer.fn(iccdata1) coeff <- icc2.nointer.fn(iccdata1)$icc2r #this only gives you the ICC coefficient coeff
This function computes 2 ICC estimates ICC(3,1) and ICCa(3,1) as measures of inter-rater reliability and intra-rater reliability coefficients under Model 3. It is the the mixed factorial ANOVA model with interaction. Some subjects are expected to have multiple ratings and due to the assumed interaction effect.
icc3.inter.fn(dfra)icc3.inter.fn(dfra)
dfra |
This is a data frame containing 3 columns or more. The first column contains subject numbers (some duplicates are expected, as some subject are assumed to have assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
This function returns a list containing the following 12 values: 1. sig2s: the subject variance component. 2.sig2r: the rater variance component 3. sig2e: the error variance component. 4. sig2sr: the subject-rater interaction variance component. 5. icc2r: ICC as a measure of inter-rater relliability. 6. icc2a: ICC as a measure of intra-rater reliability. 7. n: the number of subjects. 8. r: the number of raters. 9. max.rep: the maximum number of ratings per subject. 10. min.rep: the minimum number of ratings per subjects. 11. M: the total number of ratings for all subjects and raters. 12. ov.mean: the overall mean rating.
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. - Equation #10.2.9 of chapter 10, page 279 (for the inter-rater reliability ICC(3,1)), and Equation #10.2.10 of chapter 10, page 279 (for the intra-rater reliability ICCa(3,1)), Advanced Analytics, LLC.
#iccdata2 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata2 #see what the iccdata2 dataset looks like icc3.inter.fn(iccdata2[,2:6]) #Here, you must omit the first column coeff <- icc3.inter.fn(iccdata2[,2:6])$icc2a #this gives you intra-rater reliability coefficient coeff#iccdata2 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata2 #see what the iccdata2 dataset looks like icc3.inter.fn(iccdata2[,2:6]) #Here, you must omit the first column coeff <- icc3.inter.fn(iccdata2[,2:6])$icc2a #this gives you intra-rater reliability coefficient coeff
This function computes 2 Intraclass Correlation Coefficients ICC(3,1) and ICCa(3,1) under the mixed factorial ANOVA model (Model 3) without any subject-rater interaction. ICC(3,1) is formulated as a measure of inter-rater reliability and ICCa(3,1) as a measure of intra-rater reliability.
icc3.nointer.fn(dfra)icc3.nointer.fn(dfra)
dfra |
This is a data frame containing 3 columns or more. The first column contains subject numbers (some duplicates are expected, as some subject are assumed to have assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
This function returns a list containing the following 11 values:
1. sig2s: the subject variance component.
2. sig2e: the error variance component.
3. icc2r: ICC as a measure of inter-rater relliability.
4. icc2a: ICC as a measure of intra-rater reliability.
5. n: the number of subjects. 6. r: the number of raters.
7. max.rep: the maximum number of ratings per subject.
8. min.rep: the minimum number of ratings per subjects.
9. M: the total number of ratings for all subjects and raters.
10. ov.mean: the overall mean rating.
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. - Equation 10.2.16 of chapter 10, Advanced Analytics, LLC.
#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like icc3.nointer.fn(iccdata1) coeff <- icc3.nointer.fn(iccdata1)$icc2r #this only gives you the ICC coefficient coeff#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like icc3.nointer.fn(iccdata1) coeff <- icc3.nointer.fn(iccdata1)$icc2r #this only gives you the ICC coefficient coeff
This dataset contains data from a reliability experiment where 4 judges scored 5 subjects on a continuous scale. Some of these 5 subjects were rated multiple times. Subject 1 for example was rated 3 times by judges 1, 3 and 4, and twice by judge 2. This explains the number of records being larger than the number of subjects.
iccdata1iccdata1
This dataset has 12 rows (for the 5 subjects) and 5 columns. All 5 columns are mandatory when using the ICC functions of this package. None of the 4 judges scored all 5 subjects. Judge 4 for example did not score subject 3. Moreover, the subjects scored by all judges did not receive the same number of ratings. Therefore, some missing ratings in the form of "NA" appear in the columns associated with the remaining all judges.
This variable represents the target or subject number and may contain duplicate value to indicate multiple ratings assigned to the same subjects by the same judge
All ratings from judge 1
All ratings from judge 2
All ratings from judge 3
All ratings from judge 4
Gwet, K.L. (2014) Handbook of Inter-Rater Reliability, 4th Edition. Advanced Analytics, LLC.
This dataset contains data from a reliability experiment where 4 judges scored 5 subjects on a continuous scale. The 5 subjects are distributed across 2 groups named A and B. Moreover, each of the 4 judges assigned 3 ratings to each subject on separate occasions.
iccdata2iccdata2
This dataset has 15 rows (for the 5 subjects) and 6 columns. Only the last 5 columns are mandatory when using the ICC functions of this package.
Group number
This variable represents the target or subject number and may contain duplicate value to indicate multiple ratings assigned to the same subjects by the same judge
All ratings from judge 1
All ratings from judge 2
All ratings from judge 3
All ratings from judge 4
Gwet, K.L. (2014) Handbook of Inter-Rater Reliability, 4th Edition. Advanced Analytics, LLC.
This small dataset contains data from a reliability experiment where 3 raters scored 4 subjects on a continuous scale. All raters rated each subject once.
iccdata3iccdata3
This dataset has 4 rows (for the 4 subjects) and 4 columns. All 4 columns are mandatory when using the ICC functions of this package. Rater3 only rated 2 of the 4 subjects (i.e. subjects 2 and 3), while each of the other raters rated all 4 subjects. Therefore, some missing ratings in the form of "NA" appear in the column associated with the Rater3.
This variable represents the target or subject number and may contain duplicate value to indicate multiple ratings assigned to the same subjects by the same judge
All ratings from judge 1
All ratings from judge 2
All ratings from judge 3
Gwet, K.L. (2014) Handbook of Inter-Rater Reliability, 4th Edition. Advanced Analytics, LLC.
This MSE is associated with the one-factor ANOVA model where each subject could be rated by a different group of raters. This MSE is used for calculating confidence intervals and p-values associated with the inter-rater reliability coefficient.
mse1a.fn(dfra)mse1a.fn(dfra)
dfra |
This is a data frame containing 3 columns or more. The first column contains subject numbers (there could be duplicates if a subject was assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
This function returns a vector containing the following 3 values: 1. mse: the MSE. 2. M: The total number of ratings from all raters and subjects. 3. n: The number of subjects that participated in the experiment.
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. chapter 8, section 8.3.1. Advanced Analytics, LLC.
This MSE is associated with the one-factor ANOVA model where each rater may rate a different group of subjects. This MSE is used for calculating confidence intervals and p-values associated with the intra-rater reliability coefficient.
mse1b.fn(dfra)mse1b.fn(dfra)
dfra |
This is a data frame containing 3 columns or more. The first column contains subject numbers (there could be duplicates if a subject was assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
This function returns a vector containing the following 3 values: 1. mse: the MSE. 2. M: The total number of ratings from all raters and subjects. 3. n: The number of subjects that participated in the experiment.
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. chapter 8, section 8.3.3. Advanced Analytics, LLC.
This function can be used to compute the MSE under the random (Model 2) and mixed (Model 3) efffects ANOVA model with interaction. This MSE is needed for calculating confidence intervals and p-values associated with the inter-rater and intra-rater reliability coefficients.
mse2.inter.fn(dfra)mse2.inter.fn(dfra)
dfra |
This is a data frame containing 3 columns or more. The first column contains subject numbers (there could be duplicates if a subject was assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. chapter 9, section 9.3.1 and chapter 10, section 10.3.1. Advanced Analytics, LLC.
This function computes the MSE for both the random factorial (Model 2) and mixed factorial (Model 3) without subject-rater Interaction. This MSE is used for calculating confidence intervals and p-values associated with the inter-rater and intra-rater reliability coefficients.
mse2.nointer.fn(dfra)mse2.nointer.fn(dfra)
dfra |
This is a data frame containing 3 columns or more. The first column contains subject numbers (there could be duplicates if a subject was assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. chapter 10, section 10.3.1, Advanced Analytics, LLC.
This function computes the MSI under both the random factorial (Model 2) and mixed factorial (Model 3) ANOVA model with subject-rater interaction. This MSI is used for calculating confidence intervals and p-values associated with the inter-rater and intra-rater reliability coefficients. coefficients under both models 2 and 3.
msi2.fn(dfra)msi2.fn(dfra)
dfra |
This is a data frame containing 3 columns or more. The first column contains subject numbers (there could be duplicates if a subject was assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. chapter 9, section 9.3.1 and chapter 10, section 10.3.1. Advanced Analytics, LLC.
This MSR is associated with the one-factor ANOVA model where each rater may rate a different group of subjects. This MSR is used for calculating confidence intervals and p-values associated with the intra-rater reliability coefficient.
msr1b.fn(dfra)msr1b.fn(dfra)
dfra |
This is a data frame containing 3 columns or more. The first column contains subject numbers (there could be duplicates if a subject was assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
This function returns a vector containing 2 values: 1. msr: the MSR. 2. r: The number of raters that participated in the experiment.
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. chapter 8, section 8.3.4. Advanced Analytics, LLC.
This function computes the MSR under the random factorial ANOVA model (MOdel 2). It can be used whether or not the subject-rater interaction is assumed. The MSR is used for calculating confidence intervals and p-values associated with the inter-rater and intra-rater reliability coefficients under model 2.
msr2.fn(dfra)msr2.fn(dfra)
dfra |
This is a data frame containing 3 columns or more. The first column contains subject numbers (there could be duplicates if a subject was assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. chapter 9, section 9.3.1, Advanced Analytics, LLC.
This MSS is associated with the one-factor ANOVA model where each subject may be rated a different group of raters. This MSS is used for calculating confidence intervals and p-values associated with the inter-rater reliability coefficient.
mss1a.fn(dfra)mss1a.fn(dfra)
dfra |
This is a data frame containing 3 columns or more. The first column contains subject numbers (there could be duplicates if a subject was assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
This function returns a vector containing the following 2 values: 1. mss: the MSS. 2. n: The number of subjects.
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. chapter 8, section 8.3.1. Advanced Analytics, LLC.
This function computes the MSS under the random factorial (Model 2) and mixed factorial (Model 3) ANOVA model. The MSS is used for calculating confidence intervals and p-values associated with the inter-rater and intra-rater reliability coefficients.
mss2.fn(dfra)mss2.fn(dfra)
dfra |
This is a data frame containing 3 columns or more. The first column contains subject numbers (there could be duplicates if a subject was assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. chapter 9, section 9.3.1 and chapter 10, section 10.3.1. Advanced Analytics, LLC.
This function computes the p-value associated with the Intraclass Correlation Coefficient (ICC) under the one-factor ANOVA model where each subject may be rated by a different group of raters.
pval.ICC1a(ratings)pval.ICC1a(ratings)
ratings |
This is a data frame containing 3 columns or more. The first column contains subject numbers (there could be duplicates if a subject was assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
This function returns a vector containing 6 p-values associated with the 6 null values 0,0.1,0.3,0.5,0.7,0.9.
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. chapter 8, section 8.3.2, equation 8.3.4. Advanced Analytics, LLC.
#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like pval.ICC1a(iccdata1)#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like pval.ICC1a(iccdata1)
This function computes the p-value associated with the Intraclass Correlation Coefficient (ICC) under the one-factor ANOVA model where each rater may rate a different group of subject.
pval.ICC1b(ratings)pval.ICC1b(ratings)
ratings |
This is a data frame containing 3 columns or more. The first column contains subject numbers (there could be duplicates if a subject was assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
This function returns a vector containing 6 p-values associated with the 6 null values 0,0.1,0.3,0.5,0.7,0.9.
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. chapter 8, section 8.3.5, equation 8.3.8. Advanced Analytics, LLC.
#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like pval.ICC1b(iccdata1)#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like pval.ICC1b(iccdata1)
This function computes 6 p-values for the Intraclass Correlation Coefficient (ICC) used as a measure of inter-rater reliability under the random factorial ANOVA model (Model 2) with subject-rater interaction. Each of the 6 p-values is associated with one of the null values 0,0.1,0.3,0.5,0.7,0.9.
pval.ICC2r.inter(dfra)pval.ICC2r.inter(dfra)
dfra |
This is a data frame containing 3 columns or more. The first column contains subject numbers (there could be duplicates if a subject was assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
This function returns a vector containing 6 p-values associated with the 6 null values 0,0.1,0.3,0.5,0.7,0.9.
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. chapter 9, section 9.3.1, equation 9.3.6. Advanced Analytics, LLC.
#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like pval.ICC2r.inter(iccdata1)#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like pval.ICC2r.inter(iccdata1)
This function computes the p-value associated with the Intraclass Correlation Coefficient (ICC) under the one-factor ANOVA model where each subject may be rated by a different group of raters.
pvals.ICC1a(ratings, rho.zero = 0)pvals.ICC1a(ratings, rho.zero = 0)
ratings |
This is a data frame containing 3 columns or more. The first column contains subject numbers (there could be duplicates if a subject was assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
rho.zero |
This is an optional parameter that represents a vector containing an arbitrary number of null values between 0 and 1 for which a p-value will be calculated. If not specified then its defauklt value will be 0. |
This function returns a vector containing p-values associated with the null values specified in the parameter rho.zero.
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. chapter 8, section 8.3.2, equation 8.3.4. Advanced Analytics, LLC.
#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like pvals.ICC1a(iccdata1,c(0,0.17,0.22,0.35))#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like pvals.ICC1a(iccdata1,c(0,0.17,0.22,0.35))
This function computes the p-value associated with the Intraclass Correlation Coefficient (ICC) under the one-factor ANOVA model where each rater may rate a different group of subjects.
pvals.ICC1b(ratings, gam.zero = 0)pvals.ICC1b(ratings, gam.zero = 0)
ratings |
This is a data frame containing 3 columns or more. The first column contains subject numbers (there could be duplicates if a subject was assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
gam.zero |
This is an optional parameter that represents a vector containing an arbitrary number of null values between 0 and 1 for which a p-value will be calculated. If left unspecified, its default value will be 0. |
This function returns a vector containing p-values associated with the null values specified in the parameter gam.zero.
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. chapter 8, section 8.3.5, equation 8.3.8. Advanced Analytics, LLC.
#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like #Let c(0.05,0.13,0.28,0.33) be an arbitrary vector of values between 0 and 1 pvals.ICC1b(iccdata1,c(0.05,0.13,0.28,0.33))#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like #Let c(0.05,0.13,0.28,0.33) be an arbitrary vector of values between 0 and 1 pvals.ICC1b(iccdata1,c(0.05,0.13,0.28,0.33))
This function can compute several p-values associated with the Intraclass Correlation Coefficient (ICC) used to quantify intra-rater reliability coefficient under the random factorial ANOVA model with subject-rater interaction (Model 2). This function computes the p-value for each of the null values specified in the parameter gam.zero.
pvals.ICC2a.inter(dfra, gam.zero = 0)pvals.ICC2a.inter(dfra, gam.zero = 0)
dfra |
This is a data frame containing 3 columns or more. The first column contains subject numbers (there could be duplicates if a subject was assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
gam.zero |
This is an optional parameter that represents a vector containing an arbitrary number of null values between 0 and 1 for which a p-value will be calculated. If not specified then its default value will be 0. |
This function returns a vector containing p-values associated with the null values specified in the parameter gam.zero.
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. chapter 9, section 9.3.2 (page 245) Advanced Analytics, LLC.
#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like pvals.ICC2a.inter(iccdata1,c(0.15,0.20,0.25))#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like pvals.ICC2a.inter(iccdata1,c(0.15,0.20,0.25))
This function can compute several p-values associated with the Intraclass Correlation Coefficient (ICC) used to quantify intra-rater reliability under the random factorial ANOVA model without subject-rater interaction (Model 2). This function computes the p-value for each of the null values specified in the parameter rho.zero.
pvals.ICC2a.nointer(dfra, gam.zero = 0)pvals.ICC2a.nointer(dfra, gam.zero = 0)
dfra |
This is a data frame containing 3 columns or more. The first column contains subject numbers (there could be duplicates if a subject was assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
gam.zero |
This is an optional parameter that represents a vector containing an arbitrary number of null values between 0 and 1 for which a p-value will be calculated. If not specified then its default value will be 0. |
This function returns a vector containing p-values associated with the null values specified in the parameter rho.zero. #iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like pvals.ICC2a.nointer(iccdata1) pvals.ICC2a.nointer(iccdata1,seq(0.2,0.5,0.05))
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. chapter 9, section 9.5.1, equation 9.5.17, Advanced Analytics, LLC.
This function computes p-values for the Intraclass Correlation Coefficients (ICC) used as a measure of inter-rater reliability under the random factorial ANOVA model (Model 2) with subject-rater interaction. The output is vector of p-values, one for each of the null values specified in the optional rho.zero parameter, whose default value is 0.
pvals.ICC2r.inter(dfra, rho.zero = 0)pvals.ICC2r.inter(dfra, rho.zero = 0)
dfra |
This is a data frame containing 3 columns or more. The first column contains subject numbers (there could be duplicates if a subject was assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
rho.zero |
This is an optional parameter that represents a vector containing an arbitrary number of null values between 0 and 1 for which a p-value will be calculated. If not specified then its default value will be 0. |
This function returns a vector containing p-values associated with the null values specified in the parameter rho.zero.
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. chapter 9, section 9.3.1 Advanced Analytics, LLC.
#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like pvals.ICC2r.inter(iccdata1,c(0.15,0.20,0.25))#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like pvals.ICC2r.inter(iccdata1,c(0.15,0.20,0.25))
This function can compute several p-values associated with the Intraclass Correlation Coefficient (ICC) used to quantify inter-rater reliability under the random factorial ANOVA model without subject-rater interaction (Model 2). This function computes the p-value for each of the null values specified in the parameter rho.zero.
pvals.ICC2r.nointer(dfra, rho.zero = 0)pvals.ICC2r.nointer(dfra, rho.zero = 0)
dfra |
This is a data frame containing 3 columns or more. The first column contains subject numbers (there could be duplicates if a subject was assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
rho.zero |
This is an optional parameter that represents a vector containing an arbitrary number of null values between 0 and 1 for which a p-value will be calculated. If not specified then its default value will be 0. |
This function returns a vector containing p-values associated with the null values specified in the parameter rho.zero.
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. chapter 9, section 9.5.1, equation 9.5.15, Advanced Analytics, LLC.
#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like pvals.ICC2r.nointer(iccdata1) pvals.ICC2r.nointer(iccdata1,seq(0.2,0.5,0.05))#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like pvals.ICC2r.nointer(iccdata1) pvals.ICC2r.nointer(iccdata1,seq(0.2,0.5,0.05))
This function can compute several p-values associated with the Intraclass Correlation Coefficient (ICC) used to quantify intra-rater reliability under the mixed factorial ANOVA model with subject-rater interaction (Model 3). This function computes the p-value for each of the null values specified in the parameter rho.zero.
pvals.ICC3a.inter(dfra, gam.zero = 0)pvals.ICC3a.inter(dfra, gam.zero = 0)
dfra |
This is a data frame containing 3 columns or more. The first column contains subject numbers (there could be duplicates if a subject was assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
gam.zero |
This is an optional parameter that represents a vector containing an arbitrary number of null values between 0 and 1 for which a p-value will be calculated. If not specified then its default value will be 0. |
This function returns a vector containing p-values associated with the null values specified in the parameter rho.zero.
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. chapter 10, section 10.3.3 (page 286) Advanced Analytics, LLC.
#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like pvals.ICC3a.inter(iccdata1) pvals.ICC3a.inter(iccdata1,seq(0.2,0.5,0.05))#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like pvals.ICC3a.inter(iccdata1) pvals.ICC3a.inter(iccdata1,seq(0.2,0.5,0.05))
This function computes the p-value associated with the ICC under the mixed factorial ANOVA model with subject-rater interaction. The ICC considered here is the one used as a measure of inter-rater reliability and the p-value is calculated for each of the null values specified in the parameter rho.zero.
pvals.ICC3r.inter(dfra, rho.zero = 0)pvals.ICC3r.inter(dfra, rho.zero = 0)
dfra |
This is a data frame containing 3 columns or more. The first column contains subject numbers (there could be duplicates if a subject was assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
rho.zero |
This is an optional parameter that represents a vector containing an arbitrary number of null values between 0 and 1 for which a p-value will be calculated. If not specified then its default value will be 0. |
This function returns a vector containing p-values associated with the null values specified in the parameter rho.zero.
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. chapter 10, section 10.3.3 Advanced Analytics, LLC.
#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like pvals.ICC3r.inter(iccdata1) #gives you the p-value associated with default null value of 0 pvals.ICC3r.inter(iccdata1,c(0,0.15,0.25,0.33)) #produces p-values for an arbitrary vector#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like pvals.ICC3r.inter(iccdata1) #gives you the p-value associated with default null value of 0 pvals.ICC3r.inter(iccdata1,c(0,0.15,0.25,0.33)) #produces p-values for an arbitrary vector
This function can compute several p-values associated with the Intraclass Correlation Coefficient (ICC) used to quantify inter-rater reliability under the mixed factorial ANOVA model without subject-rater interaction (Model 3). This function computes the p-value for each of the null values specified in the parameter rho.zero.
pvals.ICC3r.nointer(dfra, rho.zero = 0)pvals.ICC3r.nointer(dfra, rho.zero = 0)
dfra |
This is a data frame containing 3 columns or more. The first column contains subject numbers (there could be duplicates if a subject was assigned multiple ratings) and each of the remaining columns is associated with a particular rater and contains its numeric ratings. |
rho.zero |
This is an optional parameter that represents a vector containing an arbitrary number of null values between 0 and 1 for which a p-value will be calculated. If not specified then its default value will be 0. |
This function returns a vector containing p-values associated with the null values specified in the parameter rho.zero.
Gwet, K.L. (2014): Handbook of Inter-Rater Reliability - 4th ed. chapter 10, section 10.3.3 (page 286) Advanced Analytics, LLC.
#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like pvals.ICC3r.nointer(iccdata1) pvals.ICC3r.nointer(iccdata1,seq(0.2,0.5,0.05))#iccdata1 is a small dataset that comes with the package. Use it as follows: library(irrICC) iccdata1 #see what the iccdata1 dataset looks like pvals.ICC3r.nointer(iccdata1) pvals.ICC3r.nointer(iccdata1,seq(0.2,0.5,0.05))