MRI-IQ-ETK: Digital MRI Test Phantoms and Automated Image Quality Evaluation Toolkit for Assessing MRI Image Reconstruction Methods

Summary

This repository contains the open-source Python code for the paper titled "Evaluating Machine Learning-Based MRI Reconstruction Using Digital Image Quality Phantoms". It consists of:

1. Digital phantom creation (digital_phantom.py): creating 3 types of phantoms in k-space: disk, resolution, low-contrast phantom
2. Metrics evaluation (evaluation_metrics.py): geometric accuracy, intensity uniformity, percentage ghosting, sharpness, SNR, high contrast resolution, and low contrast detectability.

Start Here

1. Install python 3.11.3 or any version greater

   (Note: the code was tested on python 3.11.3 and 3.11.5 by the authors but the user can check the compatibility with a lower version of python.)

2. Clone this repository and navigate to its root directory

   git clone https://github.com/DIDSR/mr-recon-eval-core.git

3. Install the required dependencies in a python enviroment

   pip install numpy matplotlib scikit-image scipy

Demo

Run python demo.py.

Expected Output

Geometric accuracy (maximum percentage error): 0.0003
Intensity uniformity: 94.2322
Percentage ghosting: 0.0009
Sharpness fwhm: 1.5268 Sharpness slope: 0.3233
SNR dual: 11.3576
High contrast resolution:
         number of resolved vertical lines in the upper resolution block:  4
         number of resolved horizontal lines in the lower resolution block:  4
Calculating low contrast detectability measurement (may take a while) ...
Low contrast detectability (number of complete spokes):  9

The exact values will be different in each run due to random noise in the generated phantom images. The evaluation result and the generated phantom images will be saved to an ouptutfolder.

How to Cite

• Paper Tan F, Delfino JG, Zeng R. Evaluating Machine Learning-Based MRI Reconstruction Using Digital Image Quality Phantoms. Bioengineering. 2024; 11(6):614. https://doi.org/10.3390/bioengineering11060614

• Regulatory Science Tool (RST) U.S. Food and Drug Administration. (2026). MRI-IQ-ETK: Digital MRI Test Phantoms and Automated Image Quality Evaluation Toolkit for Assessing MRI Image Reconstruction Methods (RST26MD03.01). https://cdrh-rst.fda.gov/mri-iq-etk-digital-mri-test-phantoms-and-automated-image-quality-evaluation-toolkit-assessing-mri

Disclaimer

About the Software This software and documentation (the "Software") were developed at the Food and Drug Administration (FDA) by employees of the Federal Government in the course of their official duties. Pursuant to Title 17, Section 105 of the United States Code, this work is not subject to copyright protection and is in the public domain. Permission is hereby granted, free of charge, to any person obtaining a copy of the Software, to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, or sell copies of the Software or derivatives, and to permit persons to whom the Software is furnished to do so. FDA assumes no responsibility whatsoever for use by other parties of the Software, its source code, documentation or compiled executables, and makes no guarantees, expressed or implied, about its quality, reliability, or any other characteristic. Further, use of this code in no way implies endorsement by the FDA or confers any advantage in regulatory decisions. Although this software can be redistributed and/or modified freely, we ask that any derivative works bear some notice that they are derived from it, and any modified versions bear some notice that they have been modified.

About the Catalog of Regulatory Science Tools The enclosed tool is part of the Catalog of Regulatory Science Tools, which provides a peer-reviewed resource for stakeholders to use where standards and qualified Medical Device Development Tools (MDDTs) do not yet exist. These tools do not replace FDA-recognized standards or MDDTs. This catalog collates a variety of regulatory science tools that the FDA's Center for Devices and Radiological Health's (CDRH) Office of Science and Engineering Labs (OSEL) developed. These tools use the most innovative science to support medical device development and patient access to safe and effective medical devices. If you are considering using a tool from this catalog in your marketing submissions, note that these tools have not been qualified as Medical Device Development Tools and the FDA has not evaluated the suitability of these tools within any specific context of use. You may request feedback or meetings for medical device submissions as part of the Q-Submission Program. For more information about the Catalog of Regulatory Science Tools, email RST_CDRH@fda.hhs.gov.

Contacts

Rongping Zeng, rongping.zeng@fda.hhs.gov Fei Tan, fei.tan@fda.hhs.gov