Towards evaluation of systems thinking interventions: A case study

The use of system dynamics and systems thinking in organizations is increasing dramatically. However, little is known about the efficacy of these interventions in enhancing organizational effectiveness or productivity. With few exceptions, the relationship between the use of systems thinking and organizational performance remains the province of anecdote rather than rigorous follow-up research. Evaluation helps identify the tools and techniques that can be used successfully, the situations in which they work, and their limitations. This article reports a follow-up evaluation of a well known systems thinking intervention designed to improve quality and performance in the claims adjusting unit of a U.S. insurance firm. Through a questionnaire, interviews, and extensive review of company records, it has been established that the intervention succeeded in changing the mental models and behavior of key managers. New policies consistent with model recommendations were implemented. Evidence of performance improvement is mixed, however. A number of confounding variables limit the possibility of assessing the business impact of the intervention. These limitations could have been lessened if the original intervention had been designed with rigorous evaluation as a goal.