The following is a ‘thought paper’ which was shared with us through our LinkedIn group by Carsten Engel:
Peter Dahler-Larsen in 2012 published a very interesting book “The Evaluation Society” (1). It describes, how our present society has traits from three sociohistoric epochs: The modern society, the reflexively modern society, and the audit society.
The thoughts in his book are still very relevant and provide us with a lens through which we may better understand some of the things happening around us. In particular, some aspects of the audit society are causing resentment and resistance among healthcare professionals.
These are exaggerated standardization and data collection in the belief that this will control quality and risk, measure fixation ie mistaking an indicator of quality for real quality, and a reduction of human activities from including, what Arendt calls labor, work, and action, to only including labor and work (labor = what we have to do of functional and biological necessity. Work = what we do to achieve certain goals. Action = what we do to realize our full creative potential) (p194).
A little bit more on Dahler-Larsen’s analysis.
In the modern society it is assumed that the world can be understood and changed based on knowledge. Rationality and autonomy are key concepts. Things can be fixed. Evaluation is producing knowledge that is not questioned, is generalizable, and can be used to guide further progress. Evaluation is thus part of the foundation for prudent decision making.
Eventually it is realized that complex systems are not as predictable and describable as simpler systems. Therefore there are limits to the way in which we can understand and manage by linear, rational thinking. This leads to the epoch of the reflexively modern society, giving rise to the concept of the learning organization.
Contingency and adaptability are key concepts. The idea of evaluation fits well into a reflexive mode, but attention to the contingent also renders evaluation a flavor of subjectivity and unpredictability. And evaluation can become something done, because it is expected as a demonstration of reflexivity, rather than to increase knowledge or support prudent decision.
As a reaction a neorigoristic audit society emerges, offering alternative ways of making sense of and managing the world. It is characterized by constant verification and checking. A defensive view of quality prevails: “Risk should be avoided and nothing should fall below standards. At the same time, an offensive view of quality is sacrificed” (p190).
Characteristic for the audit society are evaluation machines: “mandatory procedures for automated and detailed surveillance that give an overview of organizational activities by means of documentation and intense data concentration” (p176). Evaluation machines can be seen as “a form of ‘assisted sense making’. They offer interpretive keys that draw attention, define discourse, and orient actions in certain direction” (p204). They “define what is central in work.
Evaluation criteria help determine what actors should strive to accomplish in a given activity” (p206). This provides the typical organization of this epoch, the institutionalized organization, with “important metaphorical understandings, values, and organizational recipes and routines…that give legitimacy” (p39). But it also leads to measure fixation and replacement of action with labor and work, which in turn fosters de – professionalization, demotivation and cynicism. And much energy may be diverted into feeding the evaluation machines.
Even if I (and Dahler-Larsen) emphasize a number of negative aspects of the audit society, I do not think that the intention is that we should see the drift towards this epoch as an entirely bad thing that should always be combatted. There is a place for verification and checking by data collection. In some domains a defensive view on risk is clearly warranted.
Making sense of a complex world by organizational rituals can make things work, much in the same ways that most decisions in our daily work are not knowledge-based, based in a thorough rational analysis of the situation and outcomes of possible actions, but rule-based, based in heuristics, or value-based in what is seen as the proper and right thing to do.
I think that the perceived need for control is a response – albeit probably not the most appropriate response – to the realization that proper functioning of a complex system is not ensured by competent (“trustworthy”) individual actors. So I conclude that the audit society way is a way of making sense of and managing the world, but not THE way. Problems will come, if it over dominates.
Hospital accreditation has encountered resistance in Denmark, and the resistance has been expressed in a way that makes it obvious that accreditation has been perceived as a highly bureaucratic evaluation machine; it will probably be replaced by another evaluation machine, based on a limited set of numerical performance indicators, in the hope that this machine will turn out to be more lean and manageable.
There are reasons for the negative view of accreditation that some people hold; no doubt some over dimensioned evaluation machines have been operating to ensure that a hospital can demonstrate compliance with standards. But in my view, this is not by necessity and is actually a misunderstanding. In my opinion, accreditation offers a way of replacing continuous checking with trust, based in an assessment of, whether the system “has, what it takes”. I am fully aware that our knowledge of what this might be is far from what we would desire. Play best friv games on the site friv games now!
For the quality community, I propose some ideas related to quality and risk management to bring us out of a too unbalanced reliance on the audit society way of thinking:
- Quality cannot be entirely captured in a dataset
- Indicators can provide us with a warning that something may need attention, but in most cases there is a need to understand, what lies behind the indicator value, before any judgment can be made. We need to take what Dixon-Woods and others have called a problem-sensing, not a comfort-seeking approach to data (2).
- Data from quality surveillance in general cannot prove that future quality will be adequate, but should be designed to signal, if there may be problems.
- Risk cannot be managed solely by a defensive approach (manuals and controls). Look out for, what Hollnagel and others call “Safety II” (3).
- We should balance accountability (based on performance data) with trust, based in assurance that the system and the individual actors “have, what it takes”.
- We should accept that the world is not totally predictable and controllable. An undesired outcome is not always proof that a person (as in a blame-and-shame culture) or a system (as in much of current patient safety thinking) failed in a predictable and therefore preventable way.
- Evaluation is an important and valuable activity, but it cannot give an answer to everything. In Dahler-Larsen’s words: Evaluators should be modest.
To me a major value of Dahler-Larsen’s book is that it makes us see and understand something that hitherto was hidden to us, in this case an underlying trend to a drift towards domination of the audit society. This is the first step towards doing better.
(1) Peter Dahler-Larsen. The Evaluation Society. Stanford University Press, 2012.
(2) Dixon-Woods M, Baker R, Charles K et al. Culture and behaviour in the English National Health Service: over-view of lessons from a large multimethod study. BMJ Qual Saf 2013; 23: 106-115.
(3) Erik Hollnagel. Safety-I and Safety–II: The Past and Future of Safety Management. Ashgate, 2014.
