In a randomized clinical trial including 50 physicians, the use of an LLM did not significantly enhance diagnostic reasoning performance compared with the availability of only conventional resources.
Fifty physicians (26 attendings, 24 residents; median years in practice, 3 [IQR, 2-8]) participated virtually as well as at 1 in-person site. The median diagnostic reasoning score per case was 76% (IQR, 66%-87%) for the LLM group and 74% (IQR, 63%-84%) for the conventional resources-only group, with an adjusted difference of 2 percentage points (95% CI, ?4 to 8 percentage points; P?=?.60). The median time spent per case for the LLM group was 519 (IQR, 371-668) seconds, compared with 565 (IQR, 456-788) seconds for the conventional resources group, with a time difference of ?82 (95% CI, ?195 to 31; P?=?.20) seconds. The LLM alone scored 16 percentage points (95% CI, 2-30 percentage points; P?=?.03) higher than the conventional resources group.
https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2825395
