New AI from Fujitsu Recognizes and Confirms Proper Hand Washing Technique

Clean Tech Featured Future of Work Health Tech

Wash your hands.  It’s kind of a common sense thing to do, and something we all should do regularly anyway, but it’s become an area of extreme focus during the COVID-19 pandemic.  Everyone should be well aware of the CDC recommendations for when and how to wash your hands to reduce risk of infection.  Naturally, it’s even more imperative in some industries, like food service and healthcare.

If you’re not completely sure of you to do it, there are countless videos showing proper hand-washing technique, including the purple paint demo.

Or, if timing is the problem and you want to make sure you stick to the 20-second rule, you could just sing any number of songs to help keep track of your hand washing time.  The chorus to Prince’s Raspberry Beret, Toto’s Africa, or Culture Club’s Karma Chameleon, or the opening verse of Dolly Parton’s Jolene.  If you have little ones, try Happy Birthday twice, or Twinkle Twinkle Little Star or the ABCs.

Of course, that could be a little awkward in the office.  Fortunately, there may be tech to help.

Fujitsu has developed a hand wash movement recognition technology that leverages video data and AI and machine learning to identify the complex motions of hand washing.  Its Actlyzer behavioral analysis technology can replace time-consuming and error-prone manual confirmation and reporting at workplaces.  Fujitsu specifically sees the food industry, healthcare, schools, hotels, and event venues as ideal opportunities for leveraging the automated solution.

In short, the technology is designed to make it easy to monitor and manage hand washing protocols in any facility – whether required by regulatory bodies or simply to help people behave responsibly – without requiring human monitors.

While gesture recognition has been available for some time, using identification points on fingertips and knuckles, hand washing presents a much more complex scenario because of how fingers are interlocked and hands become overlapped.  Soap can also obscure detection points on fingers and prevent accurate detection.

The deep learning engine used in the solution is able to recognize the more complex movements through a combination of hand shape and repetitive rubbing motions.  A two-hand shape recognition engine uses a learned model of the basic shape of two hands, then uses its algorithms to extrapolate data when fingertip and joint identification points are not detected.  A motion recognition engine detects changing motion from successive frames and counts the number of iterations based on the iteration pattern and time.  The results from the two engines are combined to improve recognition accuracy.

Fujitsu says its High Durability Learning technology can track the motion even if camera position or lighting changes while someone is washing.

The system uses a visual display to identify each of six recommended steps for properly washing hands, and marks each step with a green “good” or red “failed” indicator, letting people monitor their progress as the wash.  Once they have passed all six steps, that information can be fed into other databases with time and date stamps as verification and to track health standards across facilities and businesses.

Fujitsu says a video data set with some 2,000 variations of people, camera positions, and soap types, confirmed the six hand washing steps with 95% accuracy, and the accuracy of the number of rubbing movements with >90% accuracy.

While Fujitsu hasn’t indicated this as part of the solution, it should be easy to integrate a notification component to immediately inform managers or supervisors if staff have failed to properly clean their hands before returning to their jobs.

In addition to providing reporting to help ensure compliance with health regulations, and simply promote safe workplaces, the visual component that lets people know when they have successfully passed each hand washing step serves as a training mechanism to help people better understand how to wash properly. While the COVID-19 pandemic was the driver for these developments, this is a solution that could – and should – be implemented in businesses everywhere at all times to help curb the spread of any contagious diseases.