Improving the validity of facility management research by use of interrater agreement

In research methodology, epistemology is concerned with the question how humans generate knowledge. In facility management (FM) research, for instance, it deals with the evaluation criteria such as validity and reliability by which researchers discriminate good knowledge from bad. The objective of this paper is to add to the scholarly methodological aspects in FM research. The paper takes a postpositivist stance and pre-supposes that scholars are able to discover what happens in FM through the categorization and scientific measurement of affective responses. It applies a method by which scholars are able to develop good knowledge and by which talented bachelor students are involved in FM research.
In this study 26 semi-structured interviews were conducted at nine different organizations in the Netherlands. Interviews, which focused on office environments and productivity, were conducted in pairs by Honours students. This paper reports on methodological issues of this study. Data collection and analysis by different researchers revealed serious threats to validity and reliability. Consequently an interrater agreement (IRA), measuring the degree of agreement between raters, was introduced to reveal and overcome differences in interpretations.
In this paper the difficulties of achieving good agreement were considered. Adjustment between raters and clear demarcation of constructs are necessary. A synopsis of usage and reporting of qualitative interview approaches is shown.