to study search History

Prospective multicenter observational study of an integrated Artificial Intelligence system with live monitoring

Organizational Data

DRKS-ID:
DRKS00027322
Recruitment Status:
Recruiting ongoing
Date of registration in DRKS:
2022-03-07
Last update in DRKS:
2023-12-01
Registration type:
Retrospective

Acronym/abbreviation of the study

PRAIM

URL of the study

No Entry

Brief summary in lay language

In collaboration with German screening units, a prospective observational study will commence in early 2022 to investigate the effectiveness, safety, and productivity of an AI application and workflow to support breast cancer screening physicians under real-world conditions. The PRAIM study (PRospective, multicenter observational study of an integrated AI system with live monitoring to support breast cancer screening) is led by Prof. Dr. med. Alexander Katalinic (University Hospital Schleswig-Holstein) in collaboration with Vara (MX Healthcare GmbH). PRAIM is a non-interventional, observational, controlled, clinical follow-up study of a CE-certified medical product (Vara) designed to display mammograms and pre-classify findings to assist users in their reporting routine. Women will continue to undergo routine breast cancer screening as usual, consistent with the screening guidelines and independent of the research objective being pursued in this study. No study monitoring visits are planned. No (randomized) allocation of mammography studies to AI or control is performed. The source population for this study will be asymptomatic women presenting to the German national breast screening program for biennial screening with digital mammography at participating breast cancer screening units. The study will aim to prospectively follow approximately at least 400,000 screening-age women, regardless of breast cancer diagnosis and screening history. The study consists of three parts: (I) Assessing the association between AI use and breast cancer detection rate (CDR) and recall rate (RR), the two primary endpoints of the study; (II) Longitudinal measurement of AI impact on CDR, RR, and reader sensitivity/specificity; and (III) Assessing the relationship between AI use and interval cancer rate and breast cancer screening program sensitivity. The primary endpoints will be evaluated in the groups of all participants deemed eligible at the time of inclusion. Secondary endpoints are related to study part (II) and (III). Primary and secondary endpoints are compared between women whose studies are read with Live AI, meaning one or both readers use AI support and the Vara user interface, and women from the same screening units whose studies are read under Shadow mode, in which Vara is evaluated in the background without interacting with readers (control group). The Shadow mode group will be supplemented with data from a retrospective historical screening cohort, composed of studies examined by means of mammography and double-reading without AI support.

Brief summary in scientific language

PRAIM is a non-interventional, observational, clinical follow-up study of a CE-certified medical product (Vara) designed to display mammograms and pre-classify findings to assist users in their reporting routine. Women will continue to undergo routine breast cancer screening as usual, consistent with the screening guidelines and independent of the research objective being pursued in this study. No study monitoring visits are planned. No (randomized) allocation of mammography studies to AI or control is performed. The source population for this study will be asymptomatic women presenting to the German national breast screening program for biennial screening with digital mammography at participating breast cancer screening units. The study will aim to prospectively follow approximately at least 400,000 screening-age women, regardless of breast cancer diagnosis and screening history. PRIMARY OBJECTIVE: The purpose of this observational study is to investigate the real-world effectiveness, safety, and productivity of an AI application and workflow to support breast cancer screening radiologists ("Live AI"); and to compare this to standard guideline-concordant screening without AI assistance ("Shadow mode"). It is hypothesized that AI application is not inferior to the standard screening process. MAIN OBJECTIVES: 1. To assess the safety and effectiveness of the AI algorithm, in terms of the screen-detected cancer detection rate and recall rate for studies read with and without AI support 2. To assess improvements in user performance (in terms of diagnostic measures) over time, and understand the factors that have the greatest influence on improvement in order to understand the most optimized setting for AI 3. To assess productivity in terms of number of exams sent to consensus for studies read with and without AI; and resource utilization and cost-effectiveness in terms of recall rate, diagnostic imaging rate, and biopsy rate for studies read with and without AI support EXPLORATORY OBJECTIVE (where available): To assess the interval cancer rate and overall program sensitivity for studies read with or without AI support if data becomes available as per standard of care, made possible through the routine data linkages between screening units and local cancer registries

Health condition or problem studied

ICD10:
C50 - Malignant neoplasm of breast
ICD10:
D05 - Carcinoma in situ of breast
Healthy volunteers:
Yes

Interventions, Observational Groups

Arm 1:
Live AI: Studies read in a double-reader setting, where one or both involved readers use Vara software for workflow and AI predictions.
Arm 2:
Shadow mode: Studies read as usual in a double-reader setting, without support from the Vara user interface or AI. Studies will also have AI predictions made in the background, but these will not be reported to the user so as to not influence screening practices.

Endpoints

Primary outcome:
I. Screen-detected cancer rate Defined as the number of true-positive examinations divided by the total number of screening mammograms, per 1000 women screened. True-positive mammogram is a positive mammogram (BI-RADS assessment ≥3) followed by the biopsy-confirmed diagnosis of breast cancer within the timeframe as defined by the mammography screening program. II. Recall rate Defined as the number of surveillance imaging examinations with BI-RADS assessment ≥3 at consensus conference and recalled for further imaging, per 1000 women screened.
Secondary outcome:
1. Number of studies sent to consensus conference per 1000 women screened 2. Biopsy recommendation rate: number of diagnostic imaging studies with BI-RADS assessment 4 or 5 per 1000 women screened 3. Biopsy rate: number of women with diagnostic biopsies per 1000 women screened 4. False positive rate: a. Number of negative diagnostic imaging studies, per 1000 women screened b. Number of negative diagnostic biopsies, per 1000 women screened 5. AI metrics: a. Number of times safety net is triggered, per 1000 women b. Number of times safety net is triggered and concordant with user prediction, per 1000 women c. Number of times safety net is triggered, shown and waived, per 1000 women d. Number of times safety net is triggered, shown and accepted, per 1000 women e. Number of times safety net is triggered, shown and accepted, resulting in malignant biopsy, per 1000 women f. Number of triaged negative studies, per 1000 women 6. Interval cancer metrics (where possible according Exploratory objective). a. Number of interval cancer diagnoses within 24 months after a normal screening mammogram, per 1000 women b. Number of times safety net is triggered and waived, and resulted in an interval cancer, per 1000 women c. Number of times study was marked as "normal" by AI and resulted in interval cancer diagnosis, per 1000 women d. AI model scores for subsequent interval cancer diagnoses 7. Screening sensitivity (where possible according to Exploratory objective). Defined as the number of screen-detected cancers, over the total number of cancers detected during 24-month period 8. Screening specificity (where possible according to Exploratory objective). Defined as the number of true negatives, over the total number of negatives. 9. Reader sensitivity. Defined as the number of cases a reader recommended to consensus conference during the first and second read, over the total number of screen-detected cancers. 10. Reader specificity. Defined as the number of cases a reader did not recommend to consensus conference, over the total number of normal cases in the screening round. 11. Reader cancer detection rate. Defined as the number of true-positive examinations found by an individual reader divided by the total number of screening mammograms read by an individual reader, per 1000 women screened.

Study Design

Purpose:
Screening
Retrospective/prospective:
Prospective
Study type:
Non-interventional
Longitudinal/cross-sectional:
Longitudinal study
Study type non-interventional:
Epidemiological study, Patient Registry

Recruitment

Recruitment Status:
Recruiting ongoing
Reason if recruiting stopped or withdrawn:
No Entry

Recruitment Locations

Recruitment countries:
  • Germany
Number of study centers:
Multicenter study
Recruitment location(s):
  • University medical center Universitätsklinikum Schleswig-Holstein, Campus Lübeck Lübeck

Recruitment period and number of participants

Planned study start date:
2022-03-01
Actual study start date:
2021-07-01
Planned study completion date:
2024-06-30
Actual Study Completion Date:
No Entry
Target Sample Size:
400000
Final Sample Size:
No Entry

Inclusion Criteria

Sex:
Female
Minimum Age:
50 Years
Maximum Age:
69 Years
Additional Inclusion Criteria:
Attending biennial breast cancer screening at a screening unit participating in the German national breast cancer screening program.

Exclusion Criteria

None

Addresses

Primary Sponsor

Vara (MX Healthcare GmbH)
Max-Urich-Strasse 3
13355 Berlin
Germany
Telephone:
No Entry
Fax:
No Entry
Contact per E-Mail
URL:
No Entry
Investigator Sponsored/Initiated Trial (IST/IIT):
No

Contact for Scientific Queries

Direktor, Institut für Sozialmedizin und Epidemiologie, Universitätsklinikum Schleswig-Holstein
Prof. Dr. med. Alexander Katalinic
Campus Lübeck, Ratzeburger Allee 160
23538 Lübeck
Germany
Telephone:
+49 451 5005 1200
Fax:
No Entry
Contact per E-Mail
URL:
No Entry

Contact for Public Queries

Country Manager Germany, MX Healthcare GmbH (Vara)
Fridtjof Storde
Max-Urich-Strasse 3
13355 Berlin
Germany
Telephone:
+49 151 2084 2538
Fax:
No Entry
Contact per E-Mail
URL:
No Entry

Principal Investigator

Direktor, Institut für Sozialmedizin und Epidemiologie, Universitätsklinikum Schleswig-Holstein
Prof. Dr. med. Alexander Katalinic
Campus Lübeck, Ratzeburger Allee 160
23538 Lübeck
Germany
Telephone:
+49 451 5005 1200
Fax:
No Entry
Contact per E-Mail
URL:
No Entry

Sources of Monetary or Material Support

Commercial (pharmaceutical industry, medical engineering industry, etc.)

MX Healthcare GmbH (Vara)
Max-Urich-Strasse 3
13355 Berlin
Germany
Telephone:
No Entry
Fax:
No Entry
Contact per E-Mail
URL:
No Entry

Ethics Committee

Address Ethics Committee

Ethik-Kommission der Med. Fakultät der Universität zu Lübeck
Ratzeburger Allee 160
23538 Lübeck
Germany
Telephone:
+49-451-5004639
Fax:
+49-451-5003026
Contact per E-Mail
URL:
No Entry

Vote of leading Ethics Committee

Date of ethics committee application:
2022-02-10
Ethics committee number:
22-043
Vote of the Ethics Committee:
Approved
Date of the vote:
2022-03-03

Further identification numbers

Other primary registry ID:
No Entry
EudraCT Number:
No Entry
UTN (Universal Trial Number):
U1111-1274-6086
EUDAMED Number:
No Entry

IPD - Individual Participant Data

Do you plan to make participant-related data (IPD) available to other researchers in an anonymized form?:
Yes
IPD Sharing Plan:
Anonymized participant-related data will be shared to anyone who wishes access to the data immediately after publication. There will be one row for each screening case. Results of the manuscript will be reproducible. Plan is to make data available via https://datadryad.org/.

Study protocol and other study documents

Study protocols:
No Entry
Study abstract:
No Entry
Other study documents:
No Entry
Background literature:
No Entry
Related DRKS studies:
No Entry

Publication of study results

Planned publication:
No Entry
Publikationen/Studienergebnisse:
No Entry
Date of first publication of study results:
No Entry
DRKS entry published for the first time with results:
No Entry

Basic reporting

Basic Reporting / Results tables:
No Entry
Brief summary of results:
No Entry