NeuroInterp: A method for facilitating neuroimaging research on cerebral malaria

The NeuroInterp instrument.

Development of the NeuroInterp instrument began in 2008, when, as part of a long-standing NIH-funded study of pediatric cerebral malaria, brain MRI acquisition became part of the standard evaluation of comatose patients admitted to the pediatric research ward at Queen Elizabeth Central Hospital in Blantyre, Malawi.¹

CT and autopsy studies of cerebral malaria suggested a number of likely radiographic findings, including acute edema, chronic atrophy, inflammatory lesions, and ischemia.^2,3 To develop appropriate items for inclusion in the NeuroInterp platform for pediatric cerebral malaria findings, 2 years of narrative-based findings were summarized and distilled into discrete items for structured entry. Two fellowship-trained radiologists, one in neuroradiology (M.J.P.) and one in MRI (S.D.K.), read all studies. The radiologists reviewed all MRIs from the research ward and collaboratively developed a systematic scoring procedure for the potential cerebral malaria cases that included graded measures of cerebral edema, periventricular white matter changes, cortical abnormalities, and brainstem changes, among others.⁴ These included variables denoting anatomical localization and characterization of specific MRI pulse sequence characteristics, such as presence or absence of T2 abnormalities and/or diffusion-weighted imaging changes. The variables for each feature were created to permit unambiguous and mutually exclusive options (e.g., “Were there T2 signal abnormalities present within the supratentorial cortex? [yes/no]”). Equivocation options of “not applicable” were also included for most features. Some features cannot be endorsed with confidence. To address the ambivalence, we included a field to capture the reviewer confidence on the presence or absence of the feature in question (<25%, 25%–50%, 50%–75%, >75%). Fields were also created to capture the change in a clinical feature over the course of serial readings in the same patient. Fields capturing longitudinal data were given the following answer options: progressed, remained the same, or regressed. Importantly, this was an iterative process whereby new insights into relevant findings led to additional items for inclusion in NeuroInterp and/or further delineation of existing variables until item saturation when no additional new findings were routinely identified.

The NeuroInterp application.

To begin the data entry application, we generated metadata to describe each of the fields required by the instrument. The metadata as a set contain the caption or description, the field or variable name, the data type (number or text), and the answer options (e.g., 1 = yes; 2 = no). The data entry forms for NeuroInterp are built on a proprietary research information management system called RIX. This platform generates hypertext markup language (similar to Web pages) forms based on the metadata with fields required by the instrument. Each data entry form is dedicated to one radiologist's evaluation of one neuroradiologic interpretation for one patient. The metadata can be used to generate forms to capture structured data as guided by the instrument.

The NeuroInterp workflow.

To begin the workflow, radiologists assemble the files for the study, launch the NeuroInterp form, find or create a patient, review the files, and enter their findings as prompted by the form, which follows the instrument paradigm. Periodically, an adjudicating radiologist views a report showing which patients have 2 readings and await a discordance review (see figures 1 and 2). From the report, a patient is selected and the adjudicator (in collaboration or solo) makes a judgment for a final answer for each discordant field.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1 Workflow for NeuroInterp

MSU = Michigan State University; PACS = picture archiving and communication system.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2 NeuroInterp work screen for reading and adjudication

Discordance algorithms for identifying cases for adjudication were developed to compare endorsed values from 2 readers and identify discordant values. When a field has identical values for each field, the value is the automatic final answer. Fields that capture an unambiguous endorsement of the presence or absence of a specific feature allow no discordance between readers. Fields that capture the reader's confidence in an unambiguous endorsement have greater tolerance for discordance. For these fields, the workflow allows the “worse case” value as the automatic final answer. Fields that capture continuous measures such as volume also have a higher tolerance for discordance. For these fields, the average value between the 2 endorsed answers becomes the automatic final answer unless the values exceed the accepted variability and are therefore noted to be discordant. When discordance is detected between answers for fields with no or limited tolerance, the adjudicating user must choose a final answer upon review of the neuroradiologic images/files/studies. Open-ended text fields require a review of both readers' evaluations to enter a final adjudicated answer.

NeuroInterp—The final product.

The NeuroInterp instrument guides structured evaluation of scaled quantitative assessments of neurologic structures. The NeuroInterp application has a Web-based data entry form providing data validation upon entry and the workflow ensures that each neuroradiologic study is evaluated twice before applying a question-specific discordance algorithm and a discrepancy adjudication utility. NeuroInterp has been used to collect readings from 338 patients with adjudication required for approximately 15% of variables, with most circumstances of adjudication related to disagreement on variables detailing degrees of abnormality or involvement.

To date, NeuroInterp has facilitated the systematic study of the radiologic findings of cerebral malaria survivors with neurologic sequelae⁵ as well as a study of normative brain MRI findings in a representative community-based sample of Malawian children.⁶ NeuroInterp offers a feasible approach to the challenges of radiologic data acquisition and management, especially for conditions with little prior descriptive data available. We are presently enhancing NeuroInterp further with instructional images to facilitate reader training and improve interreader reliability. Plans are also under way to transition from propriety software to open source software, which should allow more broad use of this data management tool in cerebral malaria imaging research.