1 GUIDELINES FOR THE CRITICAL REVIEW OF QUALITATIVE PUBLICATIONS These guidelines have been developed to help you to critically evaluate research articles. They are designed to help you to actively and critically read a piece of research and to make a judgement as to whether the research was appropriate and well conducted. Not all research published in journals is good research. Being able to decide what is good and what is not is a skill that you will require both academically and professionally. You may not be able to answer all these questions when reviewing a paper - they are a guide only. 1) Area of study and clarity of research focus What is the topic or area of study? Is the area relevant or appropriate? What literature/sources of information (e.g. research studies, theory, opinions of community groups etc.) have been reviewed and is this balanced relating to different aspects of the study? Are the methods and conclusions of the studies cited, constructively criticised and the references given? Is there a clear aim/research question being addressed? Are objectives given to inform the research? Is the study seeking to interpret the actions and/or subjective experiences of participants so justifying qualitative methodology? 2) The study design and methodology Has the data been collected in a way that addresses the research question and has the setting for data collection been justified? Is it clear what methods have been used for data collection (e.g. focus group, semi-structured interview)? Is there some discussion as to why this method(s) was chosen over others and is it appropriate to the area under investigation? Is the methodology guided by theoretical considerations and are these clearly stated? Are the methods explicit (e.g. for the interview method, is there an indication of how many interviews were conducted, was a topic guide/interview schedule used etc)? Were methods modified (e.g. following a pilot study) during the research and if so, what modifications were made and have they been explained? Is the format of data clear (e.g. tape recordings, videos, notes etc) and is there some justification for ending data collection? Who constitutes the sample. How were the participants selected (e.g. convenience, purposive, etc.)? Is there any explanation as to why the participants who were selected were the most appropriate to provide the type of knowledge sought by the study? Is there any discussion provided about recruitment (e.g. why some people chose not to take part, sample size, number of groups, time frame for data collection etc)? Does the study give adequate consideration of the relationship between the researcher and participants (e.g. role of the researcher and their influence in formulating interview questions) ? Has there been a response to events during the study; did this result in any changes to the planned study and have the implications of any changes been considered? Have ethical issues been taken into consideration? Is there sufficient detail in terms of how the research was explained to participants for ethical standards to be assessed? Was approval sought Maria Maynard, Leeds Beckett University 2015 With acknowledgement to Katie Peck, Pinki Sahota and Helen White. 2 from an ethics committee? Are ethical issues raised by the study given adequate consideration (e.g. confidentiality, informed consent)? 3) The findings of the study - analysis Does the study describe the analysis process in sufficient depth? For example are themes derived from the data and if so, is it explained how these have been developed so the reader can appreciate what analysis has been carried out. What are the main findings and does the study go beyond reporting accounts to 'draw-out' meaning? Does this relate to what the study intended to do? To what extent does the study take account of contradictory data i.e. data that does not fit with the main themes (sometimes called 'deviant' or 'idiosyncratic' data)? Is sufficient data presented to support the findings and have labels been included with quotes indicating anonymised individuals? Are the themes adequately explained and fully defined? Has the analysis been informed by further data collection to fully develop themes? At what point did analysis end and was any justification given for this? Have appropriate principles been applied to evaluate how rigorous the study is (e.g. triangulation, respondent validation, 'thick' description, detailed methodology, evidence of ethical practice)? How has the data been analysed (e.g. recorded and transcribed verbatim) and by whom? Do the researchers critically examine their own role ('reflexivity'), and how that may have introduced potential bias or influence during the analysis and selection of data for presentation? 4) Explaining the findings Are the findings explicit and is there adequate discussion of the evidence for and against the interpretations made? Are the interpretations credible and adequately informed by the research and are other interpretations possible, if so what? Have the findings been discussed in relation to the original research questions and the contribution to existing knowledge or understanding (e.g. current practice, policy, research based literature, and theory). Does the study identify new areas where research is necessary? How can the findings be transferred to other populations or are there other ways in which the research can be used? Come to a conclusion as to whether the research findings have been appropriately theoretically generalised. 5) The publication/audience Are sources of funding/support acknowledged? Has any conflict of interest been declared and if so, how might this have influenced the study? Is the research published in an appropriate journal in terms of its target audience? Is this a peer reviewed journal of quality and impact? Maria Maynard, Leeds Beckett University 2015 With acknowledgement to Katie Peck, Pinki Sahota and Helen White.