Reflection machines: increasing meaningful human control over Decision Support Systems

• Main Authors: Cornelissen, N.A.J (Author), Haselager, W.F.G (Author), Schraffenberger, H.K (Author), van Eerdt, R.J.M (Author)
• Format: Article
• Language: English
• Published: Springer Science and Business Media B.V. 2022
• Subjects: AI ethic; AI ethics; Decision making; Decision support systems; Decision Support Systems; Decisions makings; Domain Knowledge; Final decision; Human control; Human expert; Human machine interaction; Human-machine interaction; Machine decisions; Meaningful human control; On-machines; Philosophical aspects; Responsibility gap
• Online Access: View Fulltext in Publisher

 Rapid developments in Artificial Intelligence are leading to an increasing human reliance on machine decision making. Even in collaborative efforts with Decision Support Systems (DSSs), where a human expert is expected to make the final decisions, it can be hard to keep the expert actively involved throughout the decision process. DSSs suggest their own solutions and thus invite passive decision making. To keep humans actively ‘on’ the decision-making loop and counter overreliance on machines, we propose a ‘reflection machine’ (RM). This system asks users questions about their decision strategy and thereby prompts them to evaluate their own decisions critically. We discuss what forms RMs can take and present a proof-of-concept implementation of a RM that can produce feedback on users’ decisions in the medical and law domains. We show that the prototype requires very little domain knowledge to create reasonably intelligent critiquing questions. With this prototype, we demonstrate the technical feasibility to develop RMs and hope to pave the way for future research into their effectiveness and value. © 2022, The Author(s).