To evaluate the interobserver reliability of inspectors from the Dutch Healthcare Inspectorate in the system for phased supervision of nursing homes and to study the association between the type of evaluation and the presence or absence of argumentation accompanying these evaluations.

Descriptive, quantitative, retrospective.

A total of 4914 evaluations with accompanying argumentation by 26 inspectors from the Dutch Healthcare Inspectorate were collected from 182 supervision reports and analyzed. These evaluations were made in 2005 and 2006 on 25 criteria specified for reliable care in nursing homes. To compare inspectors and their evaluations of different types of nursing homes, data were statistically corrected for institutional features using analysis of covariance.

There were statistically significant differences in evaluation between the inspectors. After a correction for institutional features these differences remain significant for 14 of 25 criteria. The presence of argumentation for an evaluation appeared to depend on the inspector as well as on the evaluation given.

Interobserver disagreement plays a role in the second phase of phased supervision of nursing homes. The results of this research will be used for further fine-tuning of the system of supervision by the Dutch Healthcare Inspectorate.