Consensus on interprofessional collaboration in hospitals: statistical agreement of ratings from ethnographic fieldwork and measurement scales

RATIONALE: Few methods are available for analysing psychometric properties of combined qualitative and quantitative data. While conventional reliability of measures - meaning reproducibility or consistency - may not be meaningful in small-N research, in some health services studies agreement on perceptions arising from data generated by fieldwork and quantitative measures can be examined to good effect. METHODS: We studied interprofessional collaboration (IPC) in seven hospitals. An ethnographer shadowed and conducted interviews with regulated health professionals in medicine wards. Concurrently, nurses completed the nurse-doctor relations subscale of the Nursing Work Index (NWI-NDRS) and a new measurement scale for IPC with doctors in the domains of communication, accommodation, and isolation. After fieldwork, the ethnographer rank-ordered hospital sites on IPC from 1 to 7 based on interpretation of the qualitative data. Mean-scale scores were calculated for hospital sites and converted to ranks similarly. The Tinsley-Weiss T-index (Tinsley & Weiss, 1975) for agreement among rank orderings was calculated for dyadic combinations of fieldwork and measurement ranks. RESULTS: Perfect agreement was obtained for the most liberal agreement definitions considered - differences of two rank positions - involving qualitative data agreement with IPC subscales for accommodation and isolation. Defining agreement as a difference of 1 rank at most, the T-index was 0.77 for agreement between fieldworker and IPC accommodation and the same for NWI-NDRS and IPC isolation. CONCLUSION: Qualitative data from fieldwork rankings were substantially in accord with the contemporary IPC scales, less so with the NWI-NDRS. Qualitative data appear to be useful as an additional approach to confirming the validity of quantitative scale data in measuring a complex interpersonal relational construct.