In the 1920s, a series of experiments were conducted at Hawthorne Works, a Western Electric telephone factory just outside Chicago, to study the effects of lighting on worker productivity. The researchers found that improved lighting increased manufacturing output – but only until the study ended, when productivity reverted to its previous level, even though the new lighting persisted. The researchers concluded that it was the act of being studied, not the lighting, that made workers increase their productivity.
The “Hawthorne effect,” as it is now known, has been well-documented in social science: individuals, typically research subjects, actively change their behavior when they know they are being observed and monitored. The effect goes beyond productivity. It has been found in many different contexts, from improved hand hygiene among health care workers being studied to increased voter turnout when people’s voter preferences are being assessed.
In a recent study, published in JAMA Internal Medicine, we uncovered a unique form of the Hawthorne effect in hospitals, with important consequences for patients. We analyzed how unannounced hospital inspections affect a hospital’s patient outcomes. These inspections are done by the Joint Commission, a U.S. health care regulatory organization that ensures hospitals comply with patient safety standards. To maintain accreditation, hospitals are required to undergo these unannounced, week-long inspections every 18 to 36 months. Officials assess the hospital’s adherence to a number of quality and safety processes, including hand hygiene, infection control protocols, and proper documentation, to name a few.
The stakes for hospitals are high – loss of accreditation in the review process can harm a hospital’s reputation or even presage closure. So hospital staff are keenly aware of their behavior during the inspection period. Some observers have called these visits “Code J” and noted that hospitals mobilize tremendous resources to deal with these visits. Indeed, two of us (Jena and Barnett) work at large teaching hospitals in Boston and have observed several Joint Commission visits: the inspection week is stressful for hospital executives and staff, starting with a flurry of emails to hospital employees making them aware that a Joint Commission inspection is underway. Despite the level of attention these visits get, their immediate effect on physician behavior and patient outcomes has not been explored.
We obtained Joint Commission inspection dates for 1,984 U.S. hospitals during 2008-2012, and we matched those dates to hospitalization data for more than 1.7 million Medicare beneficiaries. We compared the outcomes of patients admitted to the hospital during an inspection week against patients admitted to the same hospital in the weeks immediately preceding or following the inspection.
We found that Medicare patients who were admitted to a hospital during a Joint Commission visit had slightly lower mortality than patients admitted during our non-inspection weeks. The effects were most pronounced in large teaching hospitals, where patients admitted during an inspection week had lower 30-day mortality (5.9%) than patients in the surrounding weeks (6.4%), a statistically significant 6% relative decrease. Mortality rates in the weeks before and after the inspection were similar; in other words, the mortality drop during inspection weeks was short-lived. Although observational studies such as ours cannot establish causal relationships, the natural experiment study design that we employed suggests that Joint Commission visits led to mortality declines for patients admitted during inspection weeks.
While a 6% reduction in mortality may seem modest, it is meaningful in aggregate. According to our estimates, it means that approximately 3,600 fewer Medicare beneficiaries would die each year, or approximately 10 fewer per day, if the mortality rates observed during inspection weeks prevailed the rest of the year.
A natural question would be whether patients hospitalized during inspection and non-inspection periods were different. We found that the volume of hospitalizations was nearly identical between inspection and non-inspection weeks, and the characteristics of patients and their reasons for hospitalization were also similar. This should be expected since inspections were unannounced.
At least two explanations for why we’re seeing this inspection effect are possible. First, physicians and nurses may modify their specific behaviors that the Joint Commission inspectors measure, such as hand hygiene and infection control practices. We found no evidence, however, that hospital-acquired infection rates fell during inspection weeks, which would suggest that improved infection control practices during inspection weren’t behind our findings. We also found that other adverse safety events, like pressure ulcers and pulmonary embolism after surgery, did not decrease during inspection weeks, further suggesting that our findings weren’t due to better adherence to the specific protocols that the Joint Commission measures.
Another possibility is that doctors were just more focused during inspection weeks. Heightened scrutiny during visits may raise clinicians’ awareness of certain operational deficiencies, and improve their focus, attention, and vigilance – all of which would lead to better patient care. For instance, being more attentive may produce more careful clinical documentation, which begets stronger communication across staff and ultimately higher quality treatment. Similarly, the presence of inspectors may reduce time spent by hospital staff on non-work related activities that may distract them from patient care – we call this the “Facebook effect.”
Our findings do not imply that the Joint Commission should visit hospitals more often. Inspection visits are incredibly stressful, and we initially wondered if we’d findworse patient outcomes because clinicians may be distracted by inspectors. Rather, our findings suggest that focus, attention, and clinical vigilance can have a significant impact on patient care. This is well known. One reason hospitals have tried to reduce resident physician work hours is because physicians are not as alert 20 hours into a shift as they are in the first two hours. Studies of antibiotic prescribing, for example, have demonstrated that physicians are more likely to inappropriately prescribe antibiotics later in the day, presumably due to increased fatigue.
How might our findings be used to improve patient care?
First, hospitals could analyze their own clinical data to observe which aspects of their normal day-to-day operations change most dramatically to meet inspection standards. For example, are there reductions in inappropriate medication prescribing during inspection weeks? Is there evidence that hospital infections fall? If so, is it because handwashing increases? Identifying these changes may offer opportunities to improve care and patient outcomes.
Second, to the extent that heightened clinical vigilance and attention are behind our observed findings, efforts could be made to build a work environment that enhances these conditions. For example, periods of the day when the majority of clinical decisions are made – typically the morning during patients rounds – could be dedicated “disruption”-free periods, in which pages, calls, and other interruptions to doctors’ and nurses’ workflow are minimized. Creating more quiet work spaces that facilitate focus on clinical tasks may also be effective.
The “Hawthorne effect” is real. Identifying what behaviors change when they’re being monitored, and how they affect patient outcomes, may provide useful insights into how the quality of hospital care can be improved.
Source:
https://hbr.org/2017/03/when-clinicians-know-theyre-being-watched-patients-fare-better
Speak Your Mind