How Machine Learning and Machine Vision Can Benefit the Medical Device Manufacturing Process.

Machine learning (ML) technologies have become key tools in the performance and administration of medical devices, including the production of clinical evaluation reports (CERs) manufacturers must regularly submit to stay compliant. 

Device manufacturers also use ML capabilities to add powerful features and introduce new service models to capture recurring revenue streams and strengthen relationships with clients.

The introduction of tiny machine learning – known as tinyML – takes this even further by allowing manufacturers to make ML-powered medical devices smaller and more compact. ML and big data tools are also used to track the performance of medical devices and uncover trends around device failures.

But there’s another way machine learning – in particular, machine vision techniques – can improve the effectiveness of medical devices: During the manufacturing process itself.

How machine vision improves medical device manufacturing

Machine vision is often used for inspection and quality assurance (QA) tasks during manufacturing – an especially vital role given the strict compliance requirements of most device makers. 

First, though, a note on the difference between machine vision and computer vision. Machine vision systems always include a camera to view an image, and are often used in tandem with computer vision algorithms to make inferences about those images. They’re commonly used in manufacturing settings to detect product flaws.

On the other hand, computer vision can extract data from images without other systems’ assistance and read images from various sources (including video and 3D).

But machine vision for medical devices isn’t just about meeting compliance standards. Indeed, the improved speed, consistency, and accuracy of machine vision-enabled inspection algorithms – which, unlike human employees, never get tired or need a vacation – have the potential to save companies millions of dollars. 

Many machine vision systems are so precise they can detect imperfections the width of a human hair, for example. They’ve also proven adept at determining whether such defects are functional or cosmetic, whether a component is symmetrical, and checking for required stamps or markings.

Because every manufacturer (and device) is different, ML and machine vision tools are applied to different tasks and parts of the production process depending on requirements and the existence of bottlenecks.

The challenges – and promise – of machine vision for medical device inspections

The main challenge of using cameras and machine vision systems for QA has traditionally been their ability to identify accurate pass/fail criteria. Partly this has stemmed from the hardware used in such systems in the past, argue Frederick Gertz and Gilbert Fluetsch of SHL Medical, a manufacturer of auto and pen injectors.

These hardware limitations range from the camera’s resolution to the lighting quality when capturing images (although the advent of LED lighting, in different colors for different applications, has partially alleviated the latter issue). 

“Complicated algorithms,” they write in ONdrugDelivery, a pharmaceutical industry publication, “usually hundreds or thousands of lines of code, had to be programmed for an inspection of a component, a measurement of a critical dimension or any other application. The code would often rely on pixel counts of the camera, which in turn was dependent on the camera’s resolution and the above-mentioned light source. 

“Hundreds of hours of testing would follow the coding process and, even then, the traditional inspection systems would not provide solutions that could be trusted 100 percent.”

Scalability problems related to so-called “traditional” computer vision systems for medical device QA also include defining the important areas of images, usually through a detailed report by a subject matter expert (SME). Machine vision engineers then use this SME feedback to design rules for the algorithm to follow when scanning images and determining quality, typically a very time-consuming process.

Gertz, Fluetsch, and their colleagues at SHL Medical say they’ve developed a better method that combines machine vision with deep learning through the TensorFlow neural framework.

SHL set up a convolutional neural network of nine layers, including three for convolution and three for pooling, creating a network of nearly 240 million trainable parameters. “Using an unimpressive amount of computational resources, including a readily available laptop, the training over a data set of about 240 images took approximately 30 minutes,” the authors write.

At that point, the authors say their new system reached 100 percent accuracy on both their training and test datasets.

Gauging the accuracy of machine vision inspections

So how does one efficiently gauge the accuracy of a machine vision system for medical device inspections – especially when imperfections can be microscopic?

One technique developed by machine vision engineers is to overlay a heat map on the component, indicating which areas the algorithm has identified as important. An example of heat maps on two similar but separate components is below.

BTW: The component on the right has a defect – can you spot it?

A heat map of two components that shows the most important regions for machine scanning. The image on the right contains an imperfection. Image courtesy ONdrugDelivery.

Hint: It’s on the left-hand side of the image on the right.

CapeStart powers machine learning for medical devices

CapeStart’s machine learning engineers work with medical device manufacturers every day, from developing natural language processing-aided CERs to supplying companies with prebuilt ML models and ML training datasets. 

Contact us today to learn how we can help you innovate on every step of your AI and business optimization journey.