A parallel evaluation of 5 indirect cost-effective methods for assessing failure of passive immunity transfer in neonatal calves

Graphical Abstract Summary: The aim of this study was to evaluate the diagnostic accuracy of 5 indirect methods to assess the transfer of maternal immunoglobulins in neonatal calves. Because it does not require any specialized laboratory equipment, a digital Brix refractometer is a valuable and inexpensive tool for on-farm use, to help producers and veterinary practitioners in their calf health management programs.

A parallel evaluation of 5 indirect cost-effective methods for assessing failure of passive immunity transfer in neonatal calves A. Mugnier,1

Summary
The aim of this study was to evaluate the diagnostic accuracy of 5 indirect methods to assess the transfer of maternal immunoglobulins in neonatal calves. Because it does not require any specialized laboratory equipment, a digital Brix refractometer is a valuable and inexpensive tool for on-farm use, to help producers and veterinary practitioners in their calf health management programs. 1 N ewborn calves are agammaglobulinemic, and protection from infectious diseases mostly depends on the quality of passive immunity transfer from colostrum (Elsohaby et al., 2015). Failure of transfer of passive immunity (FTPI) is associated with increased risk of mortality, severe diarrhea, and respiratory diseases in neonatal calves (Donovan et al., 1998). Passive immunity status also influences longevity, future milk production, and weight gain (Vogels et al., 2013;Thornhill et al., 2015;Raboisson et al., 2016). FTPI is classically diagnosed when serum IgG concentration is <10.0 g/L (Deelen et al., 2014;Elsohaby et al., 2015;Buczinski et al., 2018), but the 8.0 g/L threshold has sometimes been used (Buczinski et al., 2018). Despite the fact that best practices regarding colostrum management have been widely spread in the scientific and technical literature, FTPI remains a major issue in many herds (Weaver et al., 2000;Godden, 2008). Depending on the country, farming system, and decision thresholds, 19 to 40% of calves may suffer from FTPI (Vogels et al., 2013;Elsohaby et al., 2015;Buczinski et al., 2018;Raboisson et al., 2016).
Several methods have been developed to assess the adequacy of colostral immunoglobulin transfer in the neonatal calf. The historical gold standard is a direct measurement of serum IgG concentration using radial immunodiffusion (RID); however, this method requires 18 to 24 h to get the results, is expensive, and cannot be used on a farm. There is a need for easier-to-use and cost-effective methods that can rapidly determine the immune status of neonatal calves. Direct or indirect assessment of serum total protein (STP) concentration is considered a good proxy of IgG serum concentration (Tyler et al., 1998;Hernandez et al., 2016;Elsohaby et al., 2019a). The diagnostic accuracy of these methods has mostly been evaluated in controlled environments (laboratory tests done by the same researcher), and field experiments remain rare. In the current study, 5 methods conducted by veterinary practitioners under field conditions were evaluated in parallel and compared: assessment of STP by biochemistry analyzer in veterinary practice, assessment of STP by optical refractometry, digital Brix refractometry, calculated serum globulin concentration (GLOB), and serum γ-glutamyl transferase (GGT) activity level.
For a desired absolute precision of ±10%, expected values for sensitivity (Se) and specificity (Sp) estimates of 80%, and a confidence level of 95%, the minimum required sample size was 62 (Manitz et al., 2017). Under the assumption that the FTPI prevalence would be around 25% in our study population, the minimal total sample size was 248. The present study included 258 beef and dairy calves from 93 herds, with 1 to 6 calves (median 3) sampled per herd. Calves were randomly selected from each herd, and sampling was performed during regular farm visits by veterinary practitioners.
Whole blood was collected from calves by jugular venipuncture into 2 vacuum tubes without anticoagulant. Blood samples were refrigerated for 24 h before serum was separated by centrifugation, with one aliquot being immediately frozen at −20°C. One serum aliquot was used by veterinary practitioners to get an indirect es-A parallel evaluation of 5 indirect cost-effective methods for assessing failure of passive immunity transfer in neonatal calves A. Mugnier, 1 K. Pecceu, 2 F. Schelcher, 1 and F. Corbiere 1 * Short Communication Production: Health, Behavior, and Well-being timate of STP, using an optical refractometer (various trademarks, STP_OP, g/L) and a digital Brix refractometer was also applied (MA882, Milwaukee Instruments, Szeged, Hungary, %Brix). Veterinary practitioners used their VetTest biochemistry analyzers to estimate GGT (EC 2.3.2.2, IU/L) activity, STP (STP_BA, g/L), and serum albumin concentrations (ALB, g/L). Serum globulin concentration (g/L) was determined by subtracting ALB from STP_BA. All measures were performed by 15 veterinary practitioners, from 14 practices, who had previously been individually trained to use the optical and digital Brix refractometers.

JDS Communications
The frozen serum samples were transported to ISAE 35 laboratory (Combourg, France), where they were maintained at −20°C until thawing for IgG concentration determination using a commercial RID assay (RID-IgG, g/L, Bov IgG Ring Test, ID Biotech, Issoire, France).
The Pearson product moment correlation coefficient (r) between each test and the RID-IgG concentration were calculated. Then, we evaluated the diagnostic characteristics of each indirect method for detecting FTPI (IgG <10.0 g/L), using RID-IgG concentration as the reference standard. The Se of an indirect method was defined as the conditional probability of correctly identifying animals with FTPI (IgG concentration <10.0 g/L FTPI+) and Sp as the conditional probability of correctly identifying animals without FTPI (IgG ≥10.0 g/L FTPI−).
Receiver operating characteristic (ROC) curves were used to identify the optimal cutoff point for each indirect method. Cutoff values were selected based on the maximization of Youden's J statistic (J = Se + Sp − 1) (Hajian-Tilaki, 2013). Statistical uncertainty was assessed by calculating exact 95% binomial confidence intervals (95% CI). The area under the ROC curve (AUC) and the distance between the curve and the upper left-hand corner of the ROC space (d 2 ) were also calculated as an effective and combined measure of Se and Sp to estimate the ability of the test to discrimi-nate between the classes (FTPI+ vs. FTPI−) (Hajian-Tilaki, 2013). Finally, Cohen's kappa coefficients (κ) were computed to show the level of agreement between the evaluated tests and the reference RID assay at the selected optimal cutoffs.
The precision and repeatability of the RID assay were assessed by intra-assay CV, based on 10 repeated measures for 5 independent serum samples with IgG concentrations ranging from 2 to 30 g/L. Intra-assay coefficients of variation were between 4.8% and 8.2% (mean 6.2%), and RID was deemed to be repeatable enough for the purpose of the study.
Statistical analyses were performed in R software (version 3.5.0) with ROC curves fitted with the pROC package (Robin et al., 2011).
Descriptive statistics for tests results are shown in Table 1. The mean serum RID-IgG concentration was 16.26 g/L (SD ± 8.91). Seventy-six samples (31.0%) had IgG concentrations <10.0 g/L, indicating FTPI. Due to missing values for some calves, sample sizes were different from one test to the other. Results based on the sub-sample of calves having all test results available did not show any substantial variation and are not shown.
Results of all the indirect methods were significantly correlated with IgG concentrations as assessed by RID (all P < 10 −16 ). These correlations were positive and varied from high to very high (r = 0.73, 0.74, 0.87, 0.89, 0.67 for %Brix, STP_OP, STP_BA, GLOB, and GGT activity, respectively). STP_OP and %Brix were also highly correlated (r = 0.78, P < 10 −16 ), as were STP_OP and STP_BA (r = 0.85, P < 10 −16 ) and %Brix and STP_BA (r = 0.83, P < 10 −16 ) The diagnostic test characteristics of each indirect method for the assessment of FTPI are reported for optimal cutoff values in Table 2. All indirect methods performed globally well (Youden's J statistic close to or above 0.7, AUC over 0.80, and d 2 under 0.25), except STP_OP for which performances were moderate (J = 0.51, AUC = 0.80, and d 2 = 0.35). Following Landis and Koch (1977) guidelines, the agreement between the reference RID assay and STP_OP was "moderate" (κ = 0.52) at the selected optimal cutoff, and "substantial" for all other evaluated tests (κ ranging from 0.63 to 0.76).
The highest Se estimates were found for STP_BA (Se = 90.8%, 56 g/L cut-point), GLOB (Se = 89.4%, 34 g/L cut-point), and Finally, we report in Table 3 cut-point values for which calves are deemed to suffer or not to suffer from FTPI at the IgG <10.0 g/L threshold, with Sp greater than 95%.
Numerous studies have evaluated the diagnostic accuracy of indirect methods for the assessment of FTPI in calves. In the present study, we estimated the performance of 5 indirect methods applied by veterinary practitioners under field conditions.
The prevalence of FTPI in the study population was high (31.0%) and close to previously reported estimates (Beam et al., 2009;Windeyer et al., 2014;Elsohaby et al., 2015Elsohaby et al., , 2019bRabois-son et al., 2016). Because calves were selected at random within each herd, this highlighted the fact that FTPI was still frequent in French beef and dairy herds involved in the study. Although the overall sample size was limited, 76 calves were deemed to suffer from FTPI as assessed by the reference RID assay, allowing to get parameter estimates with the desired ±10% absolute precision.
The correlation coefficients of the 2 refractometers compared with the RID determination were similar (r = 0.73 and 0.74) and consistent with other studies using either optical or digital refractometers (Mc Beath et al., 1971;Elsohaby et al., 2015, Hernandez et al., 2016, Cuttance et al., 2017Elsohaby et al., 2019a), although estimates as high as 0.93 have been reported (Deelen et al., 2014).
For STP_OP and %Brix, the optimal cut-points (52 g/L and 8.4%, respectively) were the same or close to those previously reported for FTPI detection (Tyler et al., 1998;Calloway et al., 2002;Morrill et al., 2013;Deelen et al., 2014;Hernandez et al., 2016;Elsohaby et al., 2019a,b). For STP_OP a higher cut-point (55.0 g/L) has been reported by some authors (Elsohaby et al., 2015). This cut-point has also been suggested in moribund, clinically dehydrated calves . Applying this cut-point to our data led to a marginal increase in Se estimate (72.4% vs 69.7% at the 52.0 g/L cut-point), but a more important decrease in Sp estimate (74.5% vs 81.6%).
The correlation between the 2 refractometers (r = 0.78) was consistent with other studies (Elsohaby et al., 2015) but lower than elsewhere reported (r = 0.91 to 0.97; Hernandez et al., 2016). However, these 2 tests rely on the same principle of measurement. As reported by some veterinarians involved in the present study, these discrepancies may be explained by reading errors or imprecisions on the optical measuring scale, whereas %Brix was found to be easier to read on a digital refractometer. Unfortunately, the intra-or interrater reliability of the evaluated tests were not assessed in the present study, which would have been helpful to investigate this assumption. Instrument variation between the different optical refractometer trademarks might also be invoked to explain these discrepancies (Calloway et al., 2002;Vandeputte et al., 2011;Elsohaby et al., 2015). The high correlation between %Brix and STP (measured by refractometry) found in some studies may be attributed to the     use of digital refractometer to determine the STP (Hernandez et al., 2016;Deelen et al., 2014). In the present study, no significant difference was evidenced between the proportion of calves classified as having FTPI by the 2 refractometers (McNemar's test for the Se and Sp comparison, P-values = 0.114 and 0.571, respectively), indicating that the 2 refractometers performed similarly for FTPI assessment in calves. The fact that refractometry can be applied to serum harvested without centrifugation from blood samples that have been let to clot (Wallace et al., 2006) reinforces the practicality of STP_OP and %Brix under field conditions. The assessment of GLOB appeared to have the highest diagnostic performances. This is not surprising, as the possible variations in albumin levels, which influence the STP concentration, were considered (Pfeiffer et al., 1977). Indeed, %Brix, STP_OP, and STP_BA values were all significantly correlated with ALB (r = 0.59, 0.58 and 0.60 respectively, all P-values < 10 −16 ). Substantial inter-individual variations in ALB were evidenced (Table 1), which may have reflected the hydration status.
Some biological markers, such as STP and globulins concentrations, are highly influenced by dehydration (Fecteau et al., 2013). One advantage of GGT, which is abundant in colostrum and absorbed by the calf intestine (Blum and Hammon, 2000), is its low dependence on hydration status. GGT activity has already been reported to be of informative value for FTPI detection, but it is very sensitive to age (Parish et al., 1997;Šlosárková et al., 2014). In the present study, the mean GGT activity was indeed significantly higher in 2-to 3-d-old calves (640.46 IU/L; SD ± 350.31.20) than in 4-to 6-d-old calves (456.56 IU/L; SD ± 293.68, Wilcoxon signed-rank test: P-value <10 −03 ). Our optimal cutoffs (Table 2) were consistent with findings in 1-to 7-d-old calves (Perino et al., 1993) and 2-to 8-d-old calves (Cuttance et al., 2017). Lower cutpoints have been suggested, however, for d 1 (200 IU/L) or d 4 (100 IU/L) in dairy calves (Parish et al., 1997;Hogan et al., 2015). In contrast to these results, one study was unable to substantiate the utility of serum GGT activity in the prediction of passive transfer status in 69 beef calves younger than 8 d (Wilson et al., 1999). Because of these contradictory findings, studies to determine more accurate thresholds for GGT to assess FTPI depending on calves age are warranted. However, some authors claimed that the use of GGT activity levels should be discouraged, as it offers no advantage over other indirect methods (Weaver et al., 2000).
In almost all studies, including the present one, the assessment of diagnostic test accuracy was made under the assumption that RID is a perfect reference standard. However, imprecision in RID assay replicates from the same sample have been noted (Ameri and Wilkerson, 2008). In the present study, the intra-assay CV of RID for serum samples with IgG expected value around 10.0 g/L was 6.1%. Although intra-assay CV lower than 10% are classically considered acceptable for immunoassays (European Medicines Agency, 2011), the imprecision in RID assay may have led to misclassification of calves actually having or not FTPI at the IgG <10.0 g/L threshold. Recently a Bayesian latent class modeling approach has been proposed to account for the absence of a perfect reference standard (Elsohaby et al., 2019b). Results indicated that RID is not a perfect test, with Se and Sp estimates ranging from 0.92 to 0.96 and 0.91 to 0.93, respectively, depending on the assumptions made for other evaluated tests. This appealing approach requires several populations with different FTPI prevalences and large sample sizes in each herd to yield identifiable statistical models and accurate estimates (Kostoulas et al., 2017). This was not the case for the present study, based on a small number of calves in numerous herds, and this approach could not be applied. However, the influence of imprecision in RID assay could be illustrated by discarding calves with RID-IgG concentration between 9.0 and 11.0 g/L from the analysis. Because optimal cut-points remained unchanged, Se and Sp estimates were all improved (STP_OP: Se = 72.6% and Sp = 84.0%; %Brix: Se = 90.7% and Sp = 87.0%; STP_BA: Se = 92.1% and Sp = 82.9%; GLOB: Se = 93.7% and Sp = 93.0%; GGT: 92.6% and Sp = 90.0% for 2-to 3-d-old calves and Se = 93.8% and Sp = 91.5% for 4-to 6-d-old calves).
Finally, from a practical point of view, farmers or veterinarians are often interested in decision rules associated with a high level of confidence (≥95%) that a calf is suffering or not from FTPI. Comparisons of our suggested decision thresholds (Table 3) with other studies can hardly be made, because such cut-points are seldom reported. However, the suggested cut-points associated with a high level of confidence that a calf is actually suffering from FTPI lie in the range of values reported in a recent meta-analysis (Buczinski et al., 2018) for STP_OP (40-48 g/L) and %Brix (7.6-7.8 g/L). We therefore encourage authors to report such results, as they are of practical interest.
All indirect methods evaluated here can provide reasonably accurate results and be used as effective decision-making tools for determining passive immunity transfer status. From a practicable point of view, however, and despite widespread skepticism and concerns regarding the effects of hydration status, the use of digital refractometer (either measuring STP in g/L, or on the %Brix scale) should be promoted for use FTPI assessment under field conditions.