If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Evaluation of an on-farm culture test (OFCT) for subclinical mastitis in dairy cows.
•
The OFCT has high specificity and selectivity for Staphylococcus in the quarter/pooled milk.
•
The OFCT has high agreement for detecting the absence of bacteria.
•
The OFCT supports the management of mastitis and the optimal use of antibiotics.
Abstract
The purpose of this observational study was to compare the performance of a novel on-farm culture (OFC) test with the reference method (RM) in identifying pathogens, and in particular Staphylococcus aureus, associated with subclinical mastitis (SCM) in dairy cattle. The OFC test (Mastatest HiSCC; Mastaplex Limited) for SCM uses a cartridge with 2 × 12 wells allowing 1 sample to be analyzed in duplicate (24 wells) or 2 samples analyzed simultaneously, each in 12 wells. Results of the milk analyses are reported hierarchically (Staph. aureus → coagulase-negative staphylococci (CNS) → other gram positive or coliform/gram negative → no bacteria present) and emailed within 24 h. Milk samples (617 quarter level from 158 cows and 70 cow level) were collected from 288 cows [individual cow somatic cell count (ICSCC) ≥150,000 cells/mL] on 9 purposefully selected farms known to have a high prevalence of clinical and subclinical Staph. aureus mastitis in Southland New Zealand. Quarter samples were analyzed individually (617 samples) and after animal-level pooling, providing 228 (158 + 70) cow-level samples. Samples were analyzed by the OFC test (in duplicate) and the RM (culture agar medium and latex test based on the recommendation by the National Mastitis Council) and identifications confirmed with MALDI-TOF mass spectrometry. Sensitivity (Se), specificity (Sp), positive predictive value (PPV), and negative predictive value (NPV) for detection of Staph. aureus were all ~90% with tight 95% confidence limits, and Cohen's kappa (κ) for agreement between the OFC test and RM was 0.81. Kappa for agreement between the OFC test duplicates was 0.93. About 35% of cows had only one quarter infected with Staph. aureus and all these animals could still be identified when pooled cow-level milk was analyzed. Although the high prevalence of Staph. aureus in the herds used in this study does not affect the Se and Sp values, it does elevate the PPV value (and decrease the NPV) and therefore use of PPV to extrapolate to a population with lower prevalence is not appropriate. For CNS, Sp, PPV, and NPV were all >0.8, κ was ≥0.6, and Se was >0.7. Kappa for agreement between the OFC test duplicates was 0.83. A result of “no bacteria detected” was reported in 13% of the cows with 93% agreement between OFC test and RM. We conclude that the OFC test provides a reliable method for detecting Staph. aureus in pooled cow-level milk even if only one quarter is infected; in the absence of Staph. aureus in the milk, it reliably identified CNS in pooled cow-level milk; it reliably identified cows with <10 cfu/10 µL of their milk. Compared with the RM, the method was rapid with results returned in 24 h of loading the cartridge.
Graphical Abstract
Graphical AbstractSummary: A novel on-farm culture method provides a reliable means for detecting Staphylococcus aureus in pooled cow-level milk from cows with subclinical mastitis (somatic cell count >150,000 cells/mL). The turn-around time for results is 24 hours.
Mastitis, an inflammation of one or more quarters of the udder of dairy cows, adversely affects milk quality and production, adversely affects animal welfare, and has a detrimental impact on the economic viability and health of the herd (
). When accompanied by clinical signs, the mastitis is diagnosed as clinical mastitis; however, if clinical signs are imperceptible, but there is an increase in the SCC of the milk, it is classified as subclinical mastitis (SCM). Most often, SCM is due to a bacterial IMI (
Evaluation of hand-held sodium, potassium, calcium, and electrical conductivity meters for diagnosing subclinical mastitis and intramammary infection in dairy cattle.
J. Vet. Intern. Med.2019; 33 (31294480): 2343-2353
). In this regard IMI due to Staphylococcus aureus are of concern for herd management because of their contagiousness and their impact on the quality of the bulk milk. An individual cow somatic cell count (ICSCC) >150,000 is considered high and a proxy for IMI, although milk culture is the gold standard for detecting IMI (
A comparison of indirect methods for diagnosis of subclinical intramammary infection in lactating dairy cows: Part 2: The discriminative ability of eight parameters in foremilk from individual quarters and cows.
). However, the cost, the turnaround times of milk culture, and the fact that results are often inconclusive discourage its use in routine practice, so on-farm surrogate methods are of interest (
Evaluation of hand-held sodium, potassium, calcium, and electrical conductivity meters for diagnosing subclinical mastitis and intramammary infection in dairy cattle.
J. Vet. Intern. Med.2019; 33 (31294480): 2343-2353
Only the results of a reliable on-farm test should be used to inform costly decisions (e.g., milk withhold, order of milking, culling of cows with recurring infections) and for selective use of antibiotics. Use of antibiotics in cows with a Staph. aureus SCM during the lactation period may be justified (
). Testing cows at dry-off to distinguish between those with IMI to receive an intramammary antibiotic and teat-sealant from those without an IMI (to receive teat sealant only) is an important step in optimizing antibiotic use (
). Thus, it is important to know the reliability of the test method [i.e., agreement with the reference standard (RM), specificity (Sp), and sensitivity (Se)] in classifying a cow (or indeed a quarter) as having an IMI or not. Preferably, this is done by comparing results of the on-farm test with a gold standard which is 100% accurate, but since no such gold standard exists for IMI, the comparison is made with the accepted reference standard while recognizing the limitations of this accepted standard (
The performances of several commercially available on-farm culture (OFC) plates that aim to identify pathogens associated with clinical mastitis were assessed by comparison with results from 2 reference laboratories (
) suggested the choice of a point-of-care test be guided by the ASSURED criteria (Affordable, Sensitive, Specific, User friendly, Rapid and Robust, Equipment free, and Deliverable to end users), criteria developed for selection of diagnostic tests in resource-constrained settings (
). The aim of this field study was to assess the sensitivity (Se), specificity (Sp), positive predictive value (PPV), negative predictive value (NPV), repeatability, and Cohen's kappa of the OFC test to identify mastitis-causing bacteria in cows with a high SCC with an emphasis on Staph. aureus IMI. A novel approach was used in that the OFC test was not only compared with a RM (plating + MALDI-TOF MS), but also the OFC test results for Staph. aureus were scrutinized by plating and MALDI-TOF MS.
Cows (228 cows in total) were sourced from 9 farms in Southland New Zealand over the period June 2019 to October 2020. Herds were selected purposefully based on a known high prevalence of clinical and subclinical Staph. aureus mastitis, willingness of the farmer to participate, and a herd test for evaluation of ICSCC within 6 wk of enrolment in the study. All cows with ICSCC ≥150,000 cell/mL at the most recent herd test were eligible for sampling with the exception of cows who were systemically unwell, had clinical mastitis symptoms on the day of sampling, or had been treated with antimicrobials within the previous 28 d. Where possible, milk samples (30 mL) were collected at quarter level to give 617 quarter samples from 158 cows (15 cows had only 3 functioning quarters). Composite level milk was collected from an additional 70 cows. All samples were collected aseptically by trained technicians or veterinarians. Since the primary aim of the trial was assessment at the cow level, equal portions of quarter-level samples were combined to form in total 228 (158 + 70) pooled cow-level samples. In addition, quarter-level milk was assessed for Staph. aureus (Figure 1). Freshly collected milk samples were shipped frozen for laboratory-based testing. The research protocol which specified the testing methodology was reviewed and approved by the AgResearch Animal Ethics Committee (Ruakura: AE 14793) before the trial began.
Figure 1Percentage of cows with 0, 1, 2, 3, or 4 quarters infected with Staphylococcus aureus, as determined by reference method (RM; filled) and on-farm culture (OFC) test with 24 wells (open). The values at the tops of the bars are numbers of cows.
All milk samples were cultured at Mastaplex, and purified culture plates were then analyzed via MALDI-TOF MS by the Southern Community Laboratories at the Dunedin hospital. Analysts were blinded to the results of the alternative test.
The RM used traditional culture agar media and latex test based on the recommendation by the
). First, the milk samples were thawed at room temperature and thoroughly vortexed. A milk sample volume of ~10 μL was spread on MacConkey and Columbia sheep blood agar plates (Fort Richard Laboratories Ltd.), incubated for 24 and 48 h at 37°C, and growth or no growth of the bacteria was recorded. A hierarchical system was not used; thus, if Staph. aureus was found, other colonies were tested and identified only as CNS and gram positive/gram negative but not at species level. The procedure involved morphological identification of the colonies, the number of colony-forming units, recorded as sparse (1–10 cfu/plate), moderate (11–50 cfu/plate), or abundant (>50 cfu/plate), for all bacterial types (
), and visible patterns of hemolysis. This was followed by a catalase test for gram-positive bacteria to differentiate the Staphylococcus genus from the Streptococcus genus. Catalase-positive isolates were further tested for agglutination using a Latex test kit (Pro-Lab Diagnostics) to differentiate coagulase-positive staphylococci (assumed to be Staph. aureus) from CNS. Latex-positive colonies were isolated and cultured on blood agar plates at 37°C for 24 h. The pure cultures were sent for MALDI-TOF analysis (Southern Community Laboratories, Dunedin, New Zealand). Staphylococcus aureus was reported as positive for ≥1 cfu on a plate. Both catalase-negative colonies from blood agar and gram-negative colonies from MacConkey were reported as positive if >10 cfu on a plate as other gram positive and gram negative, respectively (
). In the absence of Staph. aureus, strains with moderate colony-forming units were sent for MALDI-TOF identification. If bacteria from 3 different genera were present, the sample was regarded as contaminated unless Staph. aureus was isolated, in which case those were classified as Staph. aureus.
The OFC test evaluated in the field trial was a Mastatest HiSCC (
). The OFC test cartridge is an innovative culture approach to on-farm mastitis diagnosis requiring raw milk samples from dairy cows. The cartridge was developed using an iterative process to improve accuracy involving in vitro and farm-based studies. The farm-based study described in this paper was independent of the development process.
The OFC test is a color-change test using 12 wells containing a range of selective and differential broth media to carry out bacterial identification. Each OFC test cartridge has 2 sets of 12 wells. It can test 2 milk samples and is recommended for cows with ICSCC ≥150,000 cells/mL determined at herd testing or by a rapid mastitis test (e.g., California Mastitis Test). The milk-filled cartridge is placed in the OFC test Lapbox for analysis and results are emailed within 24 h. Pathogens are reported by priority only according to the following hierarchy based on treatment strategies:
1.
Is Staph. aureus present? → if yes, report Staph. aureus (no further test evaluation).
2.
If no, is CNS present? → if yes, report CNS (no further test evaluation).
3.
If no, is other gram positive or coliform/gram negative present → if yes, report result.
4.
If no → no bacteria detected.
Each sample was analyzed twice using ~1.5 mL in the OFC test top 12 wells and 1.5 mL in the bottom 12 wells in the 22-h test cycle followed by the cartridge evaluation over 1 to 2 h. If Staph. aureus (and some CNS) were identified, then milk (~10 μL) from those wells was cultured on blood agar plates at 37°C for 24 h, subcultured, then sent for MALDI-TOF analysis. This novel approach was used to identify false positives in the OFC test result, important in cases where the RM was negative, and the OFC test was positive for Staph. aureus.
Statistical analyses were performed using Excel (Microsoft Corp.) and MedCalc (
). The Se (probability that OFC test result will be positive when the RM is positive), Sp (probability that OFC result will be negative when RM is negative), PPV (probability that the RM is positive when the OFC test is positive), NPV (probability that the RM is negative when the OFC test is negative), and diagnostic agreement (overall probability that a milk sample is correctly classified) were calculated. For Se, Sp, PPV, and NPV, the literature has classified values >0.80 as high, >0.60 as intermediate, and ≤0.60 as low (
); however, it should be noted that a “high” sensitivity (e.g., 80%) could be disastrous (20% of false-negative results) for those trying to control contagious pathogens. The acceptable error in a test depends on the application and therefore we have avoided “high, intermediate, low” terminology in favor of reporting statistical values. Similarly, for Cohen's kappa statistic (κ), which is a measure of agreement beyond chance between the OFC test and the RM (
), we report the values rather than using the common classification guideline: κ >0.80 almost perfect, >0.60 to 0.80 substantial, >0.40 to 0.60 moderate, >0.20 to 0.40 fair, >0.00 to 0.20 slight agreement, and ≤0.00 poor agreement.
The aim of this study was to determine the reliability (i.e., Se, Sp, PPV, NPV, and kappa) of the OFC test to identify mastitis-causing pathogens in milk from cows with ICSCC ≥150,000 cells/mL. If reliable, the OFC test could be used to inform costly on-farm decisions (e.g., treatment vs. culling) and use of antibiotic plus internal teat-sealant (ITS) versus ITS-only at dry-off, an important consideration for the optimal use of antibiotic.
No samples were classified as “contaminated” by the RM, probably because >10 cfu/μL was used as the cut-off for a positive culture, except for Staph. aureus, and 32 samples were identified as mixed samples. As determined by both the RM and OFC test, about 45% of the 228 pooled milk samples contained Staph. aureus (Table 1) with the prevalence ranging from 20% to 84% across the 9 farms. This is a far higher prevalence than that reported recently and is related to cow and herd selection and stage of lactation (
). Staphylococcus aureus, CNS, and “no bacteria detected” also had enough for detailed analysis, but other gram positive, coliform/other gram negative, and mixed culture gram positive/negative prevalences were too low, as expected, to be analyzed in detail.
Table 1Numbers (and percentages) of samples with bacteria as identified by reference method (RM) and on-farm culture (OFC) test (Mastatest HiSCC;
For Staph. aureus, the κ value for agreement between the OFC and RM was >0.8 whether 12 or 24 of the OFC test wells were used (Table 2). The agreement between repeat measurements in the OFC test (i.e., top 12 and bottom 12) was κ = 0.93, indicating that accuracy is not improved by repeat measurements. Thus, with appropriate care to avoid cross-contamination, the OFC test cartridge could be used for 2 milk samples as a cost-saving measure without sacrificing agreement with a RM.
Table 2Analytical outcomes for presence of Staphylococcus aureus, CNS, and absence of bacteria in 228 cow-level (pooled) milk samples analyzed by the reference method (RM) and on-farm culture (OFC) test (T12 = top 12 wells, B12 = bottom 12 wells, 24 = combined T12 and B12)
The Se, Sp, PPV, and NPV for detection of Staph. aureus were all about 90%, and the 95% confidence limits were tight, all being less than 7 percentage points (Table 2).
A high PPV means that there is a low probability of a false positive diagnosis, which means limited unnecessary treatment or unnecessary culling, and better stewardship of antibiotics. A high NPV means that there is a low probability of a false-negative diagnosis, which means treatment is rarely wrongly withheld and animal welfare is rarely compromised. A disadvantage of the PPV and NPV statistics is that their values depend on the prevalence. Across the 9 farms in the study, for cows with ICSCC ≥150,000 cells/mL, the prevalence of Staph. aureus varied from 20% to 84%. Assuming Se and Sp of 90%, the PPV would be 69% to 98% and the NPV 97% to 63%.
), no RM or gold standard is 100% accurate and it is important to recognize the limitations of the RM. Therefore, whenever Staph. aureus was identified by the OFC test, this was cross-checked by plating/MALDI-TOF MS. Table 2 shows 13 cases for which the RM was negative but the OFC test 24 was positive. These would be referred to as false positives. However, plating/MALDI-TOF MS found that 8 of these samples were indeed positive for Staph. aureus (i.e., the RM was wrong), 2 were CNS, 2 were false positives, and for 3 of the 13 no plating/MALDI-TOF MS was carried out. Thus, at worst there were 5 false positives. If the RM had correctly identified these samples as containing Staph. aureus the performance for OFC test 24 would have been even higher: agreement 94%, Se 92%, Sp 96%, PPV 95%, NPV 93%, and κ = 0.88.
The 617 quarter milk samples available from 158 cows were analyzed for Staph. aureus by the OFC test and RM. Irrespective of whether 12 or 24 OFC test wells were used, the agreement, Se and Sp with RM were all about 90%. Numbers of quarters infected ranged from none to all 4 (Figure 1). In total 57% of cows based on quarter milk samples were infected with Staph. aureus compared with 53% of cows based on pooled milk samples (pooled from quarter samples). Importantly, 30% of cows had only one quarter infected with Staph. aureus and therefore almost all single quarter infections were analyzed positive for Staph. aureus in pooled samples. The agreement between the composite milk and quarter milk analyses for Staph. aureus was 95% (95% confidence limit = 90%–98%). That is, even when the infected quarter milk was diluted with non-Staph. aureus milk from the other 3 quarters, the OFC test was reliable. This supports the use of pooled cow-level milk thereby enabling a 4-fold reduction in the number of samples being tested, a substantial cost-saving measure when testing at dry-off, if cow-level antibiotic plus ITS, rather than quarter-level antibiotic, is to be used.
For those milk samples without Staph. aureus, both the RM and OFC test found 32% to 38% contained CNS (Tables 1) with the prevalence ranging from 5% to 50% across the 9 farms. Since milk samples that contained Staph. aureus were not eligible for measurement of CNS presence, this is the prevalence of CNS in cows without Staph. aureus. The Sp, PPV, and NPV for detection of CNS were all >0.8, κ ≥0.6, and Se >0.7 (Table 2). The 95% confidence limits were reasonably tight (<11 percentage points). As for Staph. aureus, the agreement between repeat measurements in the OFC test (i.e., top 12 and bottom 12) had a κ = 0.83, again indicating that the accuracy is not improved by repeat measurements and that the OFC test cartridge could be used for 2 milk samples as a cost-saving measure.
The OFC test reports by priority only, according to the hierarchy described in the methods above.
“No bacteria detected” was reported in 13% of the samples (Table 1), and the OFC test agreement with the RM was 92% (95% confidence limit = 87%–96%) and the κ was 0.63 to 0.67 (Table 2). Agreement between the repeat measurements (top 12 versus bottom 12) was κ = 0.88, again supporting the conclusion that the OFC test cartridge can be used for 2 separate milk samples.
It should be noted that all cows in this field study had ICSCC ≥150,000, which is taken as indicating a subclinical IMI. Guidelines state that cows (and heifers) are considered “at risk” of infection if the ICSCC is above an agreed threshold (i.e., between 150,000 and 250,000 cells/mL) in the current lactation period (
). However, this OFC test evaluation study found that 13% of the cows with an ICSCC ≥150,000 had no bacteria (i.e., <10 cfu/10 μL) in their milk, supporting the need for caution, as discussed in
), freezing has been shown to variably affect the recovery of some isolates, but this would not affect the comparison of the OFC test with the RM since all samples were treated alike. Like most comparative tests of farm-based test methods, the samples were finally tested in a laboratory rather than on-farm. It has been argued that tests should be validated in the host species and under the conditions where they are intended to be applied (
). All our samples were collected on-farm by trained personnel, an important issue to avoid cross-contamination. The only step not undertaken on farm was the transfer of each milk sample to the OFC test cartridge. Training of personnel to carry out this simple (OFC test meets the ASSURED criterion of user friendliness) operation would be important to avoid contamination. We have concluded that the OFC test cartridge could be used for 2 independent milk samples simultaneously. We did not test this but rather compared results for the same milk sample in the 2 sections of the cartridge and found the κ >0.8. However, the cartridge is designed to prevent transfer of milk between the top and bottom 12 wells, so provided cross-contamination does not occur during the loading, we believe our conclusion to be valid. It is important to note that this was an observational study comparing the OFC with the RM. It did not implement the OFC in an intervention trial where cows were treated or not based on test results. The study focused on Staph. aureus and did not evaluate the performance of the OFC for Streptococcus spp., which are a more common treatment target at dry-off in some regions. Because we focused on Staph. aureus, the prevalence of Staph. aureus was high. Since PPV and NPV are affected by the prevalence (increased prevalence increases PPV and decreases NPV), extrapolation to a population with a more usual (lower) prevalence should not be based on PPV values, whereas Se and Sp are not influenced by prevalence. The RM followed the consensus position (i.e., >1 for Staph. aureus and >10 cfu for non-Staph. aureus isolates as positive;
). The prevalences of other gram positive, coliform/other gram negative, and mixed culture gram positive/negative were too low to allow conclusions to be drawn.
We have evaluated the OFC test against the scientific criteria (Se, Sp) of the ASSURED criteria, criteria that were developed for selection of diagnostic tests in resource-constrained settings. The OFC test method reliably identifies the presence of Staph. aureus in quarter milk and pooled cow-level milk from cows with an ICSCC ≥150,000 cell/mL. In the absence of Staph. aureus in this milk, it reliably identifies CNS in pooled cow-level milk and it reliably identifies cows with <10 cfu/10 μL of their milk. Compared with the RM the method was rapid with results returned in 24 h of loading the cassette. Although the high prevalence of Staph. aureus in the herds used in this study does not affect the Se and Sp values, it does elevate the PPV value (and decrease the NPV) and therefore use of PPV to extrapolate to a population with lower prevalence is not appropriate. It is hypothesized that this OFC is a useful tool to support management of mastitis in dairy herds.
Notes
We thank the farmers who participated in the trial. This research was funded by Mastaplex Limited (Dunedin, New Zealand). Mastatest is one of the products of Mastaplex Limited. The study protocol was designed and developed independently by co-author Mark Bryan (VetNZ Ltd., Southland, New Zealand). Bork and Saila are employees of Mastaplex Ltd. and Tucker was a consultant. The authors have not stated any other conflicts of interest.
References
Andersen S.
Dohoo I.R.
Olde Riekerink R.
Stryhn H.
Diagnosing intramammary infections: Evaluating expert opinions on the definition of intramammary infection using conjoint analysis.
A comparison of indirect methods for diagnosis of subclinical intramammary infection in lactating dairy cows: Part 2: The discriminative ability of eight parameters in foremilk from individual quarters and cows.
Evaluation of hand-held sodium, potassium, calcium, and electrical conductivity meters for diagnosing subclinical mastitis and intramammary infection in dairy cattle.
J. Vet. Intern. Med.2019; 33 (31294480): 2343-2353
00024-8&id=fx1.jpg){kind=link}
00024-8&id=gr1.jpg){kind=link}