Automatic lameness detection in cows
John Gardenier used commercial off-the-shelf Microsoft Kinect 3D cameras to set up a good working system of automatic lameness detection. As lameness is a major issue in the dairy industry and one of the most costly diseases in a herd, here we take a look at how Gardenier’s system works.
Gardenier mounted the cameras at the exit of a rotary milking platform. His system achieved the same ground-truth locomotion scores that experts gave. “And I’m pretty sure we would be able to improve on that,” Gardenier says.
How it all started
For obtaining his PhD degree at The University of Sydney, Gardenier’s supervisor at the Australian Centre for Field Robotics James Underwood suggested he join one of his projects. There were a few options in livestock but Gardenier eventually chose automatic lameness detection in dairy cows. This was also a project that Dairy Australia, the Australian national body for the dairy industry, was willing to sponsor.
John found the topic of lameness detection particularly relevant. “Of course it differs a lot on what production system is used or how important the farmer thinks lameness detection is. In most cases the level of lame dairy cows is a lot higher than farmers know. They will only pick up the really bad cases. If they get a vet to come or a lameness specialist, they will usually find four times as many lame cows in their herd.”
Unable to detect small changes
Systematic on-farm monitoring of lameness levels is not common, due to the time and skill required to perform locomotion scoring. An example of a locomotion scoring system is the Dairy Australia Healthy Hooves four-level scale, where cows are classified depending on their gait.
| John Gardenier (30), grew up in the Netherlands and New Zealand, moved to Australia in 2016 for his PhD. His thesis on automatic lameness detection in dairy cows was a collaboration between the Australian Centre for Field Robotics and the Livestock production and welfare group, both at the University of Sydney. He just completed his PhD at the University of Sydney. Research was carried out at Corstorphine farm in Cobbitty, New South Wales. His presentation, ‘How now lame cow – automatic lameness detection’ looked at taking existing technologies and machine vision, such as used in an X Box, to track and scan the gait of dairy cows to detect and monitor lameness. John has won the 2020 Australian Dairy Conference’s Young Dairy Scientist Award competition and his research was supported by Dairy Australia and the Faculty of Engineering. |
“Usually you will have a scorer watching the cows,” says Gardenier. “But humans are not very good at this. They don’t pick up on the small changes. Lameness scoring is very subjective at the best of times. I had two experts score videos of my cows and they only agreed 43% of the time when scoring with 4 levels. And even if they agreed, it might not have been the true score.”
Automatic lameness detection is still rare, Gardenier points out. “There are two commercial products out now. But the specificity of some of those products is not very good and they have not been a commercial success either.”
Cameras used
John decided to use locomotion as the centre of his detection method. In his research he used commercial off-the-shelf Microsoft Kinect 3D cameras, mounted at the exit of a rotary milking platform. These cameras provide depth images, where the value of a pixel represents how far away an object is from the sensor.
Two cameras were mounted above the race to record head, spine, and hip motion as the cow walked past, and two cameras are mounted at knee height alongside the race to record limb motion. A GoPro camera five metres to the side was used to record each cow passing for ground-truth lameness scoring by two expert scorers.
In each image, a convolutional neural network was used to detect body points of interest for lameness monitoring. These included hooves, fetlocks, carpal/tarsal joints, hip, collar and the tip of skull. Training this neural network involved manually clicking on the body points in hundreds of images, and running these images through the network for hours on end. That way the network learned what body parts to look for. After this process, the location of each body point could be determined in each new image.
From the body points in the side images, hoof placement location and time is determined. Multiple classical gait metrics are derived from this, such as stride length, stance time and step overlap. From the overhead body points, head bob, spine curvature and vertical hip displacement gait metrics were determined.
“My system is roughly as good as an expert”
Accuracy
“I wanted to find out how accurately I could predict the ground truth locomotion scores that the experts gave,” says Gardenier. “In predicting four class lameness, with scores of 0,1,2 or 3, we reached an accuracy of 55%. That was about the same range as that of the experts. You could say that my system is roughly as good as an expert.”
When these four levels were collapsed into two levels, lame or not lame, John’s system reached an accuracy of 85%. “With more data and more studies and if we would leave the system on a farm for a year and get all the cows to pass through the system, we would have a system that is more accurate than human experts. I recorded just over 3000 cows passing through the sensor system, a number that would be reached in days on a large farm.”
John’s sensing system can also contribute to better genetic selection on a longer timescale. Continuous large-scale objective monitoring of both lameness and other phenotypes such as udder conformation and body condition score can contribute to improved genetic traits, allowing for healthier, more efficient cattle for dairy farmers.
Availability
For a commercial system, suitable for long term outside use, higher grade cameras would be needed of more than AUS $1000 each compared to the consumer grade Kinect cameras at AUS $200 each “At the lower end the total system would cost about 10,000 dollars,” John says. “Hardware cost is only one component of a system price. You also have to take into account the software, installation, and support costs.” He emphasises that his system could be used in different business models.
Lameness worldwide
The lameness ratio on a dairy farm usually varies from 5% to 20% of the herd. “It depends on the time of year and specific circumstances,” John explains. “Like if it’s muddy or not.” Costs for farmers vary he says. “There are different production systems. On pasture based farms in Australia or New Zealand cows walk several kilometres per day, then get milked and go back to the paddock. In these countries the cows are stepping on stones or hurting themselves in the laneways. If cows have to do a lot of turns on concrete, they have a good chance of getting lame in one of the hooves because it just wears away the sole of the hooves.”
In Europe dairy cows are often in barns, where lameness is more often the result of an infection. “It can be damp and dark in barns, often hygiene and ventilation can cause issues.”
Real costs
“Even if you treat a lame cow, and it looks as though it’s not lame anymore, it might still feel pain in the hooves. The sensitivity can still be high for months on end after the cow’s locomotion has returned to normal. It’s a long term and complex issue, that is also intertwined with fertility and body condition. That makes it hard to estimate what the real costs are.”
