Are we there yet? Thematic analysis, NLP and Machine Learning for academic research: a comparative review

Thematic analysis is a well-established technique for qualitative analysis which is covered in traditional research methods training. The objective of thematic analysis is to elicit themes and significant topics from discursive data such as free style discussions and semi-structured or unstructured interviews or comments. The approach is laborious and time consuming and requires a significant input from researchers for identifying and coding the themes although software tools such as NVIVO and E-MRTQ can aid with results presentation. Recent developments in Machine Learning (ML) and Natural Language Processing (NLP) have boosted interest in text analytics and its applications to social science research. For example, automatic topic identification using ML NLP offers valuable insights in social media analytics. However, machine learning techniques conventionally rely on large data sets to enable the algorithm to elicit themes. More recent research efforts have turned to the performance of machine learning approaches with smaller data sets. This study aims to compare and contrast the effectiveness of Machine Learning NLP vs human generated themes using NVIVO as tool for eliciting themes from academic literature review in the context of service operations management research. Results indicate that the ML NLP approach has the potential to automatically detect research themes even with small data sets, although the algorithm requires significant calibration. The implications for researchers and PhD students are also significant as it suggest that the inclusion of machine ML NLP tools in the training curriculum may be beneficial.