The The Transform Technology Summits begin October 13 with Low-Code / No Code: Enabling Enterprise Agility. Register now!
Deep learning can detect abnormal chest x-rays with an accuracy that matches that of professional radiologists, according to a new paper by a team of Google AI researchers published in the peer-reviewed scientific journal Nature.
The deep learning system can help radiologists prioritize chest x-rays, and it can also serve as a first-response tool in emergency situations where experienced radiologists are not available. The results show that while deep learning isn’t about to replace radiologists, it can help boost their productivity at a time when the world faces a severe shortage of medical experts.
The article also shows how successful the AI research community has been in creating processes that can reduce the risks of deep learning models and create work that can be built on more in the future.
Checking for abnormal chest x-rays
Advances in AI-based medical imaging analysis are undeniable. There are now dozens of deep learning systems for medical imaging that have received official approval from the FDA and other regulatory agencies around the world.
But the problem with most of these models is that they were trained for a very limited task, such as finding traces of specific disease and conditions on x-ray images. Therefore, they will be useful only in cases where the radiologist knows what to look for.
But radiologists don’t necessarily start by looking for a particular disease. And building a system capable of detecting all possible diseases is extremely difficult, if not impossible.
“[The] wide range of CXRs possible [chest x-rays] anomalies make it impossible to detect all possible conditions by building multiple separate systems, each of which detects one or more pre-specified conditions, ”Google AI researchers write in their article.
Their solution was to create a deep learning system that detects whether a chest CT scan is normal or contains clinically actionable results. Defining the problem domain for deep learning systems involves balancing specificity and generalizability. At one end of the spectrum are deep learning models that can perform very restricted tasks (e.g. detecting pneumonia or fractures) at the cost of not generalizing to other tasks (e.g. detecting tuberculosis. ). And on the other side, there are systems that answer a more general question (for example, is this x-ray normal or does it need a closer look?) But cannot resolve any issues. more specific problems.
The intuition of the Google researchers was that the detection of anomalies can have a big impact on the work of radiologists, even if the trained model did not report specific diseases.
“A reliable AI system to distinguish normal from abnormal CXRs can contribute to rapid patient assessment and management,” the researchers write.
For example, such a system can help deprioritize or exclude normal cases, which can speed up the clinical process.
Although Google researchers did not provide specific details on the model they used, the document mentions EfficientNet, a family of convolutional neural networks (CNN) which are renowned for achieving state-of-the-art accuracy on computer vision tasks at a fraction of the computational costs of other models.
B7, the model used for X-ray anomaly detection, is the largest of the EfficientNet family and is made up of 813 layers and 66 million parameters (although the researchers likely adjusted the architecture to suit their application ). Interestingly, the researchers didn’t use Google’s TPU processors and used 10 Tesla V100 GPUs to train the model.
Avoid unnecessary biases in the deep learning model
Perhaps the most interesting part of Google’s project is the hard work that went into preparing the training and testing dataset. Deep learning engineers are often faced with the challenge of their models to detect the bad biases hidden in their training data. For example, in one case, a deep learning system for skin cancer detection had mistakenly learned to detect the presence of ruler marks on the skin. In other cases, models may become sensitive to irrelevant factors, such as the brand of equipment used to capture the images. And more importantly, it is important that a trained model can maintain its accuracy in different populations.
To ensure that problematic biases did not creep into the model, the researchers used six independent datasets for training and testing.
The deep learning model was trained on over 250,000 x-rays from five hospitals in India. The examples were tagged as “normal” or “abnormal” based on the information extracted from the results report.
The model was then evaluated with new chest x-rays obtained from hospitals in India, China and the United States to ensure that it was generalized to different regions.
The test data also contained x-rays for two diseases that were not included in the training data set, tuberculosis and Covid-19, to verify model performance on invisible diseases.
The accuracy of the labels in the dataset was independently reviewed and confirmed by three radiologists.
The researchers made the labels publicly available to aid future research on deep learning models for radiology. “To facilitate the continued development of AI models for chest radiography, we are releasing our abnormal vs. normal labels from 3 radiologists (2,430 labels out of 810 images) for the publicly available CXR-14 test set. We believe this will be useful for future work as the quality of the labels is of paramount importance for any AI study in healthcare, ”the researchers write.
Raise the radiologist with deep learning
Radiology has had a checkered history with deep learning.
In 2016, deep learning pioneer Geoffrey Hinton said, “I think if you’re working as a radiologist you’re like the coyote that’s already on the edge of the cliff but hasn’t lowered your feet yet. eyes, so he doesn’t yet realize that there is no ground beneath him. People should stop training radiologists now. It is quite obvious that in five years, deep learning will do better than radiologists, because it will gain a lot more experience – it could take ten years, but we already have a lot of radiologists. “
But five years later, AI isn’t about to kick radiologists out of their jobs. In fact, there is still a severe shortage of radiologists around the world, even though the number of radiologists has increased. And the work of a radiologist involves much more than looking at x-rays.
In their paper, the Google researchers note that their deep learning model was able to detect abnormal x-rays with comparable and, in some cases, better accuracy than human radiologists. However, they also point out that the real benefit of this system is when it is used to improve the productivity of radiologists.
To assess the effectiveness of the deep learning system, the researchers tested it in two simulated scenarios, in which the model helped a radiologist either by helping to prioritize which scans were found to be abnormal or by excluding scans. which were found to be normal. In both cases, the combination of deep learning and the radiologist resulted in a significant improvement in turnaround time.
“Whether deployed in a relatively healthy outpatient practice or in an unusually busy hospital or outpatient setting, such a system could help prioritize abnormal CXRs for expedited radiologic interpretation,” the researchers write.
Ben Dickson is a software engineer and founder of TechTalks. He writes about technology, business and politics.
This story originally appeared on Bdtech pourparlers.com. Copyright 2021
VentureBeat’s mission is to be a digital public place for technical decision-makers to learn about transformative technology and conduct transactions. Our site provides essential information on data technologies and strategies to guide you in managing your organizations. We invite you to become a member of our community, to access:
- up-to-date information on the topics that interest you
- our newsletters
- Closed thought leader content and discounted access to our popular events, such as Transform 2021: Learn more
- networking features, and more