If you have multiple advisors, calculate the percentage agreement as follows: MedCalc calculates Kappa inter-rater agreement statistics according to Cohen, 1960; The calculation details are also shown in Altman, 1991 (p. 406-407). The standard error and 95% confidence interval are calculated according to Fleiss et al., 2003. To interpret your Cohen`s kappa results, you can refer to the following guidelines (see Landis, JR-Koch, GG (1977). The measure of the compliance agreement for categorical data. Biometrics, 33, 159-174): A major error in this type of reliability between boards is that the random agreement ignores and overestimates the level of agreement. This is the main reason why the percentage of consent should not be used for scientific work (i.e. doctoral theses or scientific publications). IOA intervals for intervals. In short, the interval interval method assesses the proportion of intervals in which both observers agreed to determine whether the target reaction occurred. Note that this implies agreement on attendance and lack of response. This is calculated by adding the total number of agreed intervals to the sum of agreed intervals and divided at regular intervals. Not surprisingly, this approach often leads to high convergence statistics.
As Cooper et al. (2007) reports, this is especially true when partial interval recordings are used. In the examples in Figure 2, observers disagree on the first and seventh intervals, resulting in an interval agreement value of 71.4% (5/7). Total duration of IOA. Like the total number of IAOs for event-based data, the total duration of the IOA provides a relatively insensitive measure of observer agreement. The total duration of the IOA brings together all calendars in a cumulative duration for each observer, and is calculated by dissecting the smaller duration by the longer duration. So the more timings there are, the more possibilities there are for the discrepant data to be masked by this metric. As shown in Figure 3, the recorded durations of the two observers for the second, third and fourth occurrences of the response are essentially discrete.
However, each observer`s sums correspond to the others, giving a total duration of 100%. The partial-IOA interval agreement. To avoid the described disadvantage associated with the use of the IOA algorithm for the total number, the observation period is divided into small intervals, the partial approach of interval concordance (sometimes called “mean neck-per-interval” or “block-by-block”) dividing the observation time into small intervals, and then examining the intervals within each interval. This increases the accuracy of the chord ite by reducing the likelihood that total numbers have been deducted from the different events of the target responses within the observation. By deriding the example of Figure 1 into small steps of time/intervals (15 intervals of 1 m), the partial agreement approach calculates the IOA at intervals and divided by the total number of intervals.