Do@m

Hello Everyone. I am Bibb Allen. I am a community practice radiologist in Birmingham, Alabama at Grandview Medical Center. I am the Chief Medical Officer for the American College of Radiology Data Science Institute. #RadAIchat #ACRDSI #ACRInformatics #RadAIchat

A1 AI models are brittle which means they may not perform as expected outside of the environment where they were trained. This is true for all industries – not just healthcare. Algorithms may work beautifully and then fail at unexpected moments. #RadAIchat #ACRDSI #ACRInformatics

AI models are brittle which means they may not perform as expected outside of the environment where they were trained. This is true for all industries – not just healthcare. Algorithms may work beautifully and then fail at unexpected moments. #RadAIchat #ACRDSI #ACRInformatics

In radiology, brittleness may be caused by variations in equipment, imaging protocols & heterogeneous patient populations. Models may understand output from some but not all devices. Need multisite validation prior to deployment in clinical workflow. #RadAIchat #ACRInformatics

At last year's ACR DSI Data Science Summit, Woojin Kim (@woorinrad) advocated for using data sets enriched with abnormal and challenging cases when evaluating AI models – especially for disease processes that have low prevalence #RadAIchat #RadAIchat

Woojin also said --- You wouldn't buy a pair of shows without trying them on first, would you? You should try AI models at your institution before purchasing to make sure they work in you setting #RadAIchat

Lay press also noticed concerns our safety and bias in FDA cleared algorithms. #RadAIchat

Check out Casey Ross's Stat+ article Casey Ross: “The result is a dearth of information on whether AI products will ...trigger unintended consequences, such as an increase in incorrect diagnoses, unnecessary treatment, or an exacerbation of racial disparities.” #RadAIchat

Of course, AI does not have to be perfect to be useful. See article in Lancet Digital Health. thelancet.com/journals/landi… Generalizability is important but even models that fail occasionally can assist radiologists provide better care for our patients. #RadAIchat

Curt would say AI + humans = best result. #RadAIchat

ACR DSI is also working toward centralized data sets with NIH, RSNA, AAPM and the University of Chicago through the Medical Imaging and Data Resource Center (MIDRIC) project. midrc.org #RadAIchat

We can centralize data sets to be used for validation similar to a multicenter clinical trial but lots of challenges – HIPPA, data use agreements, etc. Or, how cool would it be if we could send the models to multiple sites and let the patient data stay on premises? #RadAIchat

FDA recognizes need for adaptive AI, but challenged to regulate so models stay locked. #RadAIchat

T5 - not only need agreement with radiologists but also exam metadata (equipment, protocol, patient info) so we can understand where models fail. #RadAIchat

Tools for evaluating and comparing AI models will be part of the ACR AILAB ailab.acr.org #ACRDSI #ACRInformatics American College of Radiology #RadAIchat#RadAIchat

Radiology organizations collaborating to promote leadership training for radiologists. #AUR2015

Dr Saini discussing "relevant" #PQRS measurements for improving patient outcomes #ARRS15 Mass General Imaging ARRS

DRAGON DICTATION软件用户许可协议

Glad to be following our state chapter AL Academy Radiology