Gaps Between Expectations and Reality for an Intraoperative AI Intervention

A recent article conducted a multisite qualitative implementation assessment of the Operating Room Black Box, focusing on gaps between what surgeons expected an intraoperative AI–based program would deliver and what participants experienced after implementation. This was the first multisite implementation assessment to examine expectation–experience gaps for this intervention across multiple centers.

The study relied on semistructured interviews conducted at 3 large academic centers that had implemented the AI intervention. Interview participants included 30 surgeons and 17 implementation leaders, capturing both end-user and operational perspectives on how the program functioned in practice. The stated purpose was to identify where expectations of the technology misaligned with lived experience and actual program deliverables, using interview-based accounts rather than performance benchmarking. Surgeons and implementation leaders contributed perspectives intended to clarify where the intervention’s promise did—and did not—match what stakeholders could access or use.

Across interviews, participants identified four major themes that the article reports as central areas of misalignment between expectations and program deliverables. First, interviewees described that the AI model required additional training to be usable, indicating that anticipated ease of adoption was not consistently reflected in real-world use. Second, they reported difficulty and time burden in accessing data on surgical cases, portraying data retrieval as more labor-intensive than expected. Third, participants described limited ability of the program to predict postoperative complications, suggesting that anticipated predictive utility was not always realized. Fourth, interviewees reported few academic deliverables, contrasting expectations for scholarly outputs with what was produced. Together, these themes were presented as the main domains in which expectations diverged from experience.

Alongside these operational gaps, the authors also reported heterogeneous surgeon attitudes toward the technology: most surgeons held neutral views, a substantial minority expressed positive views, and a small proportion expressed negative views. Difficulty accessing case data was highlighted as a prominent operational burden, with retrieval described as time consuming and not always straightforward for users who expected more readily available information. The article characterizes these attitudes and frictions as coexisting features of implementation rather than a single uniform experience across stakeholders. In their concluding interpretation, the authors state that successfully implementing AI-based interventions may require deliberate work to minimize gaps between what surgeons expect from such interventions and what the interventions can deliver.

Key Takeaways:

• At 3 large academic centers, the study interviewed 30 surgeons and 17 implementation leaders to document where expectations of an intraoperative AI intervention misaligned with experienced deliverables.
• Interviewees reported four misalignment themes spanning usability-related training needs, burdens of accessing case data, limits in predicting postoperative complications, and a low volume of academic deliverables.
• The authors concluded that successfully implementing AI-based interventions may require deliberate efforts to minimize gaps between what surgeons expect from the interventions and what they can deliver.