Background

Survey of patients exiting health facilities is a common way to assess consultation practices. It is, however, unclear to what extent health professionals may change their practices when they are aware of such interviews taking place, possibly paying more attention to following recommended practices. This so-called Hawthorne effect could have important consequences for interpreting research and programme monitoring, but has rarely been assessed.

Methods

A three-arm cluster-randomised trial of interventions to improve adherence to guidelines for the use of anti-malarial drugs was conducted in Tanzania. Patient interviews were conducted outside health facilities on two randomly-selected days per week. Health workers also routinely documented consultations in their ledgers. The Hawthorne effect was investigated by comparing routine data according to whether exit interviews had been conducted on three key indicators of malaria care. Adjusted logistic mixed-effects models were used, taking into account the dependencies within health facilities and calendar days.

Results

Routine data were collected on 19,579 consultations in 18 facilities. The odds of having a malaria rapid diagnostic test (RDT) result reported were 11 % higher on days when exit surveys were conducted (adjusted odds ratio 95 % CI: 0.98-1.26, p = 0.097), 17 % lower for prescribing an anti-malarial drug to patients with a negative RDT result (0.56-1.23, p = 0.343), and 27 % lower for prescribing an anti-malarial when no RDT result was reported (0.53-1.00, p = 0.052). The effect varied with time, with a U-shaped association over the study period (p < 0.001). We also observed a higher number of consultations recorded on days when exit-interviews were conducted (adjusted mean difference = 2.03, p < 0.001).

Conclusions

Although modest, there was some suggestion of better practice by health professionals on days when exit interviews were conducted. Researchers should be aware of the potential Hawthorne effect, and take into account assessment methods when generalising findings to the ‘real word’ setting. This effect is, however, likely to be context dependent, and further controlled evaluation across different settings should be conducted.