Monitoring and Evaluating AI: Challenges and Practical Implications

Virtual 2020 SIIM-ACR Data Science Summit
June 23, 2020 | Virtual Meeting


Agenda

Session 1: Regulatory Clearance and Approval of AI Models

12–1 PM


15 Minute Panel Discussion — Moderator: Bibb Allen Jr., MD, FACR

Session 2: Science of Evaluation: Bias, Fairness, Brittleness and Explainability

1–2 PM


15 Minute Panel Discussion — Moderator: Jayashree Kalpathy-Cramer, PhD

Session 3: ACR Data Science Institute's AI-LAB

2–2:30 PM

Session 4: Training, Validation and Generalizability: Lessons Learned

2:30–3:15 PM


15 Minute Panel Discussion — Moderator: Howard Chen, MD, MBA

Session 5: Evaluating Performance and Monitoring Algorithms

3:15–4:15 PM


15 Minute Panel Discussion — Moderator: Keith J. Dreyer, DO, PhD, FACR

Session 6: ACR Data Science Institute's AI-LAB Pilot Sites

4:14–5:00 pm


15 Minute Panel Discussion with AI-LAB Pilot Sites  Laura Coombs, PhD

Earn 5.0 CME or Category A Credit*

Physician Accreditation Statement:
The American College of Radiology is accredited by the Accreditation Council for Continuing Medical Education (ACCME) to provide continuing medical education for physicians.

Credit Designation Statement:
The American College of Radiology designates this live activity for a maximum of 5.0 AMA PRA Category 1 Credits™. Physicians should claim only the credit commensurate with their participation in the activity.

Technologist Accreditation Statement:
The American College of Radiology (ACR) is approved by the American Registry of Radiologic Technologists (ARRT) as a Recognized Continuing Education Evaluation Mechanism (RCEEM) to sponsor and/or review Continuing Education programs for Radiologic Technologists and Radiation Therapists.

Technologist Credit Designation Statement:
The American College of Radiology designates this educational activity as meeting the criteria for up to 5.0 Category A Credit hours of the ARRT.

For information about the accreditation of this program, please contact the ACR at [email protected].

Earn 6.5 CME or Category A Credit*

Physician Accreditation Statement:
The American College of Radiology is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians.

Credit Designation Statement:
The American College of Radiology designates this live activity for a maximum of 6.5 AMA PRA Category 1 Credits™. Physicians should claim only the credit commensurate with their participation in the activity.

Technologist Accreditation Statement:
The American College of Radiology (ACR) is approved by the American Registry of Radiologic Technologists (ARRT) as a Recognized Continuing Education Evaluation Mechanism (RCEEM) to sponsor and/or review Continuing Education programs for Radiologic Technologists and Radiation Therapists.

Technologist Credit Designation Statement:
The American College of Radiology designates this educational activity as meeting the criteria for up to 6.5 Category A Credit hours of the ARRT.

For information about the accreditation of this program, please contact the ACR at [email protected].

KEYNOTE SESSION: Challenges and Opportunities in Machine Reading of Radiology Exams

Machine learning methods are enabling computers to interpret unstructured document content and perform real-world tasks. Our keynote will highlight the latest trends in deep learning models that utilize imaging, free text, and structured data to advance early diagnosis, treatment, and disease prevention.

Speaker: Regina Barzilay, PhD, Professor, Department of Electrical Engineering and Computer Science and Member, Computer Science and Artificial Intelligence Laboratory at the Massachusetts Institute of Technology (MIT)

SESSION 1: Current state of AI in Practice: Diverse Perspectives and Panel Discussion

Artificial intelligence in the health care industry is gaining traction. Hear from those with experience using AI in clinical care and hospital operations about the issues that will impact you most.

  • Insight from the perspective of the radiologist, informatics/IT department, business leader and patient.
  • How to begin using AI
  • Looking for increased efficiency in operations and incorporating AI into workflows

Moderator: Amy Kotsenas, MD, ACR DSI Council Steering Committee Liaison and Associate Professor of Radiology at Mayo Clinic, Rochester

Presenters:
Melissa A. Davis, MD, MBA, Section Chief, Emergency Radiology, Yale University
Daniel Blezek, PhD, Associate Consultant, Department of Radiology, Mayo Clinic
David Andrews, Patient Advocate
Christoph Wald, MD, PhD, MBA, FACR, Chairman, Department of Radiology, Lahey Hospital & Medical Center, Professor of Radiology, Tufts University Medical School

KEYNOTE SESSION: Challenges and Opportunities in Machine Reading of Radiology Exams

Machine learning methods are enabling computers to interpret unstructured document content and perform real-world tasks. Our keynote will highlight the latest trends in deep learning models that utilize imaging, free text, and structured data to advance early diagnosis, treatment, and disease prevention.

Speaker: Regina Barzilay, PhD, Professor, Department of Electrical Engineering and Computer Science and Member, Computer Science and Artificial Intelligence Laboratory at the Massachusetts Institute of Technology (MIT)

SESSION 1: Current state of AI in Practice: Diverse Perspectives and Panel Discussion

Artificial intelligence in the health care industry is gaining traction. Hear from those with experience using AI in clinical care and hospital operations about the issues that will impact you most.

  • Insight from the perspective of the radiologist, informatics/IT department, business leader and patient.
  • How to begin using AI
  • Looking for increased efficiency in operations and incorporating AI into workflows

Moderator: Amy Kotsenas, MD, ACR DSI Council Steering Committee Liaison and Associate Professor of Radiology at Mayo Clinic, Rochester

Presenters:
Melissa A. Davis, MD, MBA, Section Chief, Emergency Radiology, Yale University
Daniel Blezek, PhD, Associate Consultant, Department of Radiology, Mayo Clinic
David Andrews, Patient Advocate
Christoph Wald, MD, PhD, MBA, FACR, Chairman, Department of Radiology, Lahey Hospital & Medical Center, Professor of Radiology, Tufts University Medical School

KEYNOTE SESSION: Challenges and Opportunities in Machine Reading of Radiology Exams

Machine learning methods are enabling computers to interpret unstructured document content and perform real-world tasks. Our keynote will highlight the latest trends in deep learning models that utilize imaging, free text, and structured data to advance early diagnosis, treatment, and disease prevention.

Speaker: Regina Barzilay, PhD, Professor, Department of Electrical Engineering and Computer Science and Member, Computer Science and Artificial Intelligence Laboratory at the Massachusetts Institute of Technology (MIT)

SESSION 1: Current state of AI in Practice: Diverse Perspectives and Panel Discussion

Artificial intelligence in the health care industry is gaining traction. Hear from those with experience using AI in clinical care and hospital operations about the issues that will impact you most.

  • Insight from the perspective of the radiologist, informatics/IT department, business leader and patient.
  • How to begin using AI
  • Looking for increased efficiency in operations and incorporating AI into workflows

Moderator: Amy Kotsenas, MD, ACR DSI Council Steering Committee Liaison and Associate Professor of Radiology at Mayo Clinic, Rochester

Presenters:
Melissa A. Davis, MD, MBA, Section Chief, Emergency Radiology, Yale University
Daniel Blezek, PhD, Associate Consultant, Department of Radiology, Mayo Clinic
David Andrews, Patient Advocate
Christoph Wald, MD, PhD, MBA, FACR, Chairman, Department of Radiology, Lahey Hospital & Medical Center, Professor of Radiology, Tufts University Medical School

KEYNOTE SESSION: Challenges and Opportunities in Machine Reading of Radiology Exams

Machine learning methods are enabling computers to interpret unstructured document content and perform real-world tasks. Our keynote will highlight the latest trends in deep learning models that utilize imaging, free text, and structured data to advance early diagnosis, treatment, and disease prevention.

Speaker: Regina Barzilay, PhD, Professor, Department of Electrical Engineering and Computer Science and Member, Computer Science and Artificial Intelligence Laboratory at the Massachusetts Institute of Technology (MIT)

SESSION 1: Current state of AI in Practice: Diverse Perspectives and Panel Discussion

Artificial intelligence in the health care industry is gaining traction. Hear from those with experience using AI in clinical care and hospital operations about the issues that will impact you most.

  • Insight from the perspective of the radiologist, informatics/IT department, business leader and patient.
  • How to begin using AI
  • Looking for increased efficiency in operations and incorporating AI into workflows

Moderator: Amy Kotsenas, MD, ACR DSI Council Steering Committee Liaison and Associate Professor of Radiology at Mayo Clinic, Rochester

Presenters:
Melissa A. Davis, MD, MBA, Section Chief, Emergency Radiology, Yale University
Daniel Blezek, PhD, Associate Consultant, Department of Radiology, Mayo Clinic
David Andrews, Patient Advocate
Christoph Wald, MD, PhD, MBA, FACR, Chairman, Department of Radiology, Lahey Hospital & Medical Center, Professor of Radiology, Tufts University Medical School

KEYNOTE SESSION: Challenges and Opportunities in Machine Reading of Radiology Exams

Machine learning methods are enabling computers to interpret unstructured document content and perform real-world tasks. Our keynote will highlight the latest trends in deep learning models that utilize imaging, free text, and structured data to advance early diagnosis, treatment, and disease prevention.

Speaker: Regina Barzilay, PhD, Professor, Department of Electrical Engineering and Computer Science and Member, Computer Science and Artificial Intelligence Laboratory at the Massachusetts Institute of Technology (MIT)

SESSION 1: Current state of AI in Practice: Diverse Perspectives and Panel Discussion

Artificial intelligence in the health care industry is gaining traction. Hear from those with experience using AI in clinical care and hospital operations about the issues that will impact you most.

  • Insight from the perspective of the radiologist, informatics/IT department, business leader and patient.
  • How to begin using AI
  • Looking for increased efficiency in operations and incorporating AI into workflows

Moderator: Amy Kotsenas, MD, ACR DSI Council Steering Committee Liaison and Associate Professor of Radiology at Mayo Clinic, Rochester

Presenters:
Melissa A. Davis, MD, MBA, Section Chief, Emergency Radiology, Yale University
Daniel Blezek, PhD, Associate Consultant, Department of Radiology, Mayo Clinic
David Andrews, Patient Advocate
Christoph Wald, MD, PhD, MBA, FACR, Chairman, Department of Radiology, Lahey Hospital & Medical Center, Professor of Radiology, Tufts University Medical School

KEYNOTE SESSION: Challenges and Opportunities in Machine Reading of Radiology Exams

Machine learning methods are enabling computers to interpret unstructured document content and perform real-world tasks. Our keynote will highlight the latest trends in deep learning models that utilize imaging, free text, and structured data to advance early diagnosis, treatment, and disease prevention.

Speaker: Regina Barzilay, PhD, Professor, Department of Electrical Engineering and Computer Science and Member, Computer Science and Artificial Intelligence Laboratory at the Massachusetts Institute of Technology (MIT)

SESSION 1: Current state of AI in Practice: Diverse Perspectives and Panel Discussion

Artificial intelligence in the health care industry is gaining traction. Hear from those with experience using AI in clinical care and hospital operations about the issues that will impact you most.

  • Insight from the perspective of the radiologist, informatics/IT department, business leader and patient.
  • How to begin using AI
  • Looking for increased efficiency in operations and incorporating AI into workflows

Moderator: Amy Kotsenas, MD, ACR DSI Council Steering Committee Liaison and Associate Professor of Radiology at Mayo Clinic, Rochester

Presenters:
Melissa A. Davis, MD, MBA, Section Chief, Emergency Radiology, Yale University
Daniel Blezek, PhD, Associate Consultant, Department of Radiology, Mayo Clinic
David Andrews, Patient Advocate
Christoph Wald, MD, PhD, MBA, FACR, Chairman, Department of Radiology, Lahey Hospital & Medical Center, Professor of Radiology, Tufts University Medical School