Assessment and refinement of an intelligent tutor for complex decision making

Abstract

We tested the effectiveness of a prototype simulation-based training procedure designed to support decision making in complex dynamic environments. An intelligent tutor was designed to monitor trainees’ use of over-simplifying heuristics throughout three scenarios and to discourage their use in a timely manner. Two test scenarios were used to assess training effectiveness: One involved a transparent problem structure while the other was partly opaque. Results showed a 34% decrease in the average use of the four heuristics monitored. However, goal attainment did not significantly improve in the test scenarios (heuristics were not replaced by better strategies). The training had a positive impact on participants’ ability to anticipate system behavior in the high transparency scenario but a negative impact in the low transparency scenario. We conclude that intelligent tutors should not only detract from using ineffective heuristics but also prescribe effective context relevant heuristics to overcome the “wall of complexity”.


Back to Table of Contents