Before buyers raise their hands to bid on Thoroughbred racing prospects at a sale, they’ve had their veterinarians evaluate the horses head to toe. This includes using videoendoscopy to assess the function of the arytenoid cartilages, in the throat, which can make or break airway performance. But how consistent are veterinarians in conducting these evaluations? Could one practitioner be “passing” a horse that another veterinarian “fails”?
Sarah Plevin, BVMS, MRCVS, CVA, Dipl. ABVP, ACVSMR, and colleagues recently conducted a study on the topic, and she presented the results at the 2018 American Association of Equine Practitioners Convention, held Dec. 1-5 in San Francisco, California. Plevin is sports medicine specialist at Florida Equine Veterinary Associates, in Ocala.
One of the most common upper respiratory tract disorders in racehorses is recurrent laryngeal neuropathy, or roaring. This condition involves paralysis of one or both of those arytenoid cartilages, which form either side of the trachea entrance, abducting—or opening—during breathing, resulting in respiratory noise and decreased performance. In North America veterinarians grade these animals ordinally on a 4-point scale, where a Grade 4 indicates paralyzed arytenoid cartilages.
Arytenoid function can significantly impact sale purchase prices, said Plevin, so it’s important for both buyers and sellers that veterinarians evaluate function consistently.
Plevin and colleagues sought to determine intraobserver agreement (one person’s consistency in assessing) and interobserver agreement (consistency among a group) on arytenoid function in 2-year-old Thoroughbreds entered in a public auction.
One veterinarian performed videoendoscopic exams on 59 horses in a sales environment, collecting a total of 70 video clips. Then, Plevin and colleagues randomly chose 22 video clips to duplicate (for 92 total clips), so that vets would be unknowingly grading these clips twice. Then, they had 10 veterinarians of varying experience levels and specialties (including interns, junior associates, board-certified surgeons, sales veterinarians, and senior veterinarians) evaluate all 92 videos. They classified horses as meeting or not meeting conditions for sale; Plevin said any grade equal to 3.1 or higher does not meet conditions for sale based on current North American auction standards.
The researchers found that:
- Both interobserver and intraobserver agreement was good for pass/fail classifications;
- For ordinal grading interobserver agreement was fair to moderate, while intraobserver agreement was fair to good;
- Interns had the least intraobserver agreement for the pass/fail scoring, while sales and senior vets had excellent intraobserver agreement; and
- No one experience level excelled above the others when the researchers evaluated ordinal scoring.
In all, Plevin said, the current investigation showed veterinarians agreed most on assessments when they were using a pass/fail system to grade arytenoid function. She cautioned, however, that it’s unlikely such a system will be widely accepted in clinical practice due to the great significance veterinarians and owners currently place on exact grade assignments, even when arytenoid function meets conditions of sale.
“The current investigation exposed a lack of strong agreement for these exact grade assignments, both between and within veterinarians when using an ordinal scale to grade arytenoid function,” she said. “Most veterinarians agreed on perfect and completely paralyzed cases, but their opinions on the grades in between these two extremes varied. And, findings from this study suggest that the expectation of perfect grade assignment is unrealistic.”
Plevin suggested that future studies be directed toward investigating the effect of veterinary training on agreement levels.