7 AI-Powered Ways to Improve Data Quality in Medical Imaging Research

The quality of medical image datasets (as with image datasets in any other field) directly affects the accuracy of machine learning models.

This is even more important in the healthcare sector, as the quality of large medical image datasets for diagnostic and medical AI (artificial intelligence) or deep learning models can be a matter of life and death for patients.

As clinical research teams know, the complexity, formats, and layers of information in medicine are greater and more intricate than in non-medical images and videos. This is where the need for artificial intelligence, machine learning (ML) and deep learning algorithms comes in, with the aim of understanding, interpreting and learning from annotated medical image datasets.

In this article, we will discuss the challenges of creating training datasets from medical images and videos (especially in the field of radiology) and share recommendations for creating the highest quality training datasets.

What is a medical image dataset?

A medical image dataset can include a wide range of medical images or videos. Medical images and videos come from a variety of sources, including microscopic examinations, radiology, CT scans, MRIs, ultrasounds, X-rays, and many others.

Medical image analysis is a complex field. It involves obtaining training data and applying ML, AI, or deep learning algorithms to understand the content and context of images, videos, and health information in order to identify patterns and expand our understanding of diseases and health conditions. Some of the most common sources of medical image data are images and videos from magnetic resonance imaging (MRI) machines and X-rays.

It all starts with creating accurate training data from large-scale medical image datasets, which requires a sufficient sample size. The accuracy of an ML model directly correlates with the quality and statistically relevant number of annotated images or videos on which the algorithm is trained.

How are medical image datasets used in machine learning?

A medical image dataset is created, annotated, labelled, and fed into machine learning (ML) models and other AI algorithms to help medical professionals solve their problems. The ultimate goal is to solve medical problems, using data sets and ML models to help teams of clinical researchers, nurses, doctors, and other medical professionals make more accurate diagnoses of diseases.

To achieve this ultimate goal, it is often helpful to have multiple data sets for training the ML model and a sufficient sample size. For example, a set of patients who may potentially have health problems and diseases (in particular, cancer) and a set of healthy people. ML and AI models are more effective when they can be trained to identify diseases, illnesses, and tumours.

When annotating and labelling large-scale medical image datasets, it is particularly useful to have images with metadata and clinical reports. The more information that can be fed into the ML model, the more accurately it will be able to solve problems. Of course, this means that medical image datasets require large amounts of data to be processed, and ML models really need data.

Why is it important for machine learning to have high-quality datasets of medical images?

Annotation and labelling work takes time, and clinical research teams need to obtain datasets of the highest possible quality. Quality control is an integral part of this process, especially when project results and model accuracy are so important.

Ideally, to reduce the risk of bias, high-quality data should come from a variety of devices and platforms, using images or videos from as many ethnic groups as possible. Datasets should include images and videos of both healthy and sick patients.

Quality directly affects the results of machine learning models. Therefore, the more accurate and diverse the range of images and annotations applied to them, the higher the likelihood that the model will achieve a level of effectiveness that justifies the financial investment in the project.

Annotators can create more accurate training data if they have the right tools, such as an AI-based tool that helps healthcare institutions and companies cope with the great challenges of computer vision in healthcare. Clinical research teams need a platform that simplifies collaboration between teams of annotators, medical professionals, and machine learning engineers.

What are the consequences of transferring a machine learning model with a ‘bad’ data set?

Transferring a low-quality, poorly cleaned (cleaning raw data is an integral part of this process), inaccurately labelled and annotated data set to a machine learning model is a waste of time.

It will negatively affect the model's results and performance, potentially devaluing the entire project and forcing clinical research teams to either start over or redo large parts of the project. Such problems require additional time and money, especially when processing large data sets.

The quality of a large data set is extremely important. A low-quality data set can result in the model not learning anything from the data due to insufficient relevant material for training.

Or, if the model is trained on a medical data set that is not diverse enough, it will produce a skewed result. Model bias can manifest itself in many different ways. It can be biased towards men or women or in favour of certain ethnic groups. The model may also mistakenly identify sick people as healthy and healthy people as sick. Hence the importance of a statistically large sample size in the data set.

‘Bad’ data can take many forms. The task of annotators and labelers is to ensure that clinical trial teams and ML engineers have access to the highest quality data with accurate annotations and labels, as well as strict quality control.

What are the most common problems with medical image datasets?

One common problem is that machine learning models cannot read image datasets.

Medical institutions sell large data sets for medical imaging research and ML-related projects. When this happens, images may be delivered without the required model diversity or with important clinical metadata such as biopsy reports removed. Or healthcare institutions simply sell data sets in bulk, without the technical ability to filter them for the images and videos they need.

However, an equally common problem is that medical data still contains personally identifiable information about patients: names, insurance details, and addresses. Due to healthcare regulatory requirements and data protection laws (e.g., FDA or EU regulations), every image annotation project must be extremely careful to cleanse data sets of anything that could identify patients and violate their privacy.

Another problem is the use of old models of medical devices with low image and video resolution.

Common challenges in creating a medical image dataset

To create and launch a project for annotating and labelling a medical image dataset, it is necessary to overcome some of the most common challenges:

• Where do we get the data from? Will it come from internal sources (like a healthcare facility or medical organisation that uses its own data sets), public sources, or will we buy it from healthcare facilities?
• Who will annotate and label the data set (e.g., in-house staff or external service providers)? Keep in mind that the time of a radiologist or other specialist is too valuable. Such work requires a reliable contractor.
• Where and how will we store medical image data?
• How will raw data be extracted for annotation and labelling?
• How will medical image data be transferred? Datasets typically contain hundreds of thousands of images, videos, and medical metadata. They cannot simply be stored on cloud servers. They cannot be zipped and attached to an email. Medical data requires high levels of encryption and, in some cases, armed security.
• Are you getting all the data you need to train the model? Remember that you need a wide enough range of images to eliminate inaccuracies and biases.
• How can you validate and filter huge amounts of images, videos, and medical imaging data?
• How long can you store the data? Regulatory authorities may limit the storage period to three years.
• If the data is annotated and labelled in another country, what are the data protection laws there, and can you do this legally? How can you guarantee that the data is transferred and stored securely?
• How can you implement effective quality control throughout the annotation process to ensure that the model receives the highest quality and most accurate data?

All these questions need to be answered before starting a medical image dataset annotation project. Only after completing the annotation and labelling of images or videos can you begin training a machine learning model to solve specific problems and challenges of the project.

Clinical research teams can improve the quality and accuracy of medical image datasets in seven ways.

7 ways clinical research teams can improve the quality and accuracy of medical image datasets

1. Obtain the right data in the right volume

Before embarking on any computer vision project, you need to obtain suitable data that is of sufficiently high quality and in quantities sufficient for statistical weighting. As mentioned above, quality is extremely important, as it can directly influence the results of ML models in a positive or negative way.

Before ordering medical image datasets, project managers need to coordinate with machine learning, data science, and clinical research teams. This will help overcome the challenges of obtaining ‘bad’ data or the fact that annotation teams will have to filter thousands of irrelevant or low-quality images and videos when annotating training data, which affects the money and time spent.

2. Addressing regulatory and compliance issues when annotating data sets

Regulatory and compliance issues must be addressed before purchasing or extracting data sets from internal sources or external suppliers.

Project managers and ML teams must ensure that data sets comply with FDA, EU, HIPAA, and any other data protection laws.

Regulatory issues affect data storage, access, and transport, the time required for the project, and ensuring sufficient anonymity of images and videos (they must not contain any identifiers of specific patients). Otherwise, you risk violating the law, which will result in substantial fines and even the risk of data leakage, especially when working with third-party annotation services.

3. Provide annotation teams with powerful AI-based tools specialising in medical image datasets

Annotating medical images for machine learning models requires accuracy, efficiency, high quality and security.

With powerful AI-powered image annotation tools, medical annotators and specialists can save hours of work and generate more accurately labelled medical images. Give your annotation teams access to the tools they need to transform data sets into training data that can be used by AI, ML, or deep learning models.

4. Ensure ease of transfer and use of medical image data sets in machine learning models

Clinical data must be delivered and transferred in a format that is easy to parse, annotate, port, and, after annotation, quickly and efficiently transfer to the ML model. Having the right tools helps, as annotators and ML teams will be able to annotate images and videos in their native format, such as DICOM and NIfTI.

When searching for reference data for medical data sets, visualisation methods and medical image segmentation also play a role. Providing deep learning algorithms with a statistically broad range of high-quality images, along with anonymised health information, dimensions (in the case of DICOM images) and biomedical image data, will yield the results that ML teams and project managers are striving for.

5. Provide clinical research and ML teams with sufficient resources to review large volumes of image data

Review resources are an issue that project managers need to consider when there are large volumes of images or videos in the data set. Do your teams have enough annotators and ML devices to review the data? Can you increase resources so that review capacity does not become a bottleneck for the project?

6. Overcoming storage and transfer challenges

As mentioned above, you will also have to overcome storage and transfer challenges. Medical image datasets often consist of hundreds or thousands of terabytes, which cannot simply be sent by email. Project managers need to ensure end-to-end security and efficiency in the purchase or extraction, cleaning, storage, and transfer of medical data.

7. Use automation and other tools in the annotation process

When annotating thousands of medical images or videos, you need automation and other tools to help your team of annotators. Make sure they have the right tools to process large amounts of medical image data so that, regardless of the quantity and quality of the data, you can be confident that they are working efficiently and cost-effectively.