SAMA at the Health Economics Study Group (HESG) Summer Conference 2023
Background
As part of work package 6 (health economics) we have been feasibility testing questionnaires which are typically used in health economics research to measure quality of life in adolescents. These measures have generally been developed and validated in high-income countries (HICs), but their use in low- and middle-income countries (LMICs) is limited. As such, it is important for us to know whether these instruments are feasible and valid for use with Indian adolescents before conducting full economic evaluations of school-based interventions in India, such as those delivered by SAMA.
In our work, we focus on two measures designed to capture adolescent quality of life – the EQ-5D-Y and CHU-9D. To assess the feasibility and validity of these measures, we conducted 10 cognitive (‘think-aloud’) interviews with adolescents in schools during my visit to India in January 2023 and have since distributed written surveys with the two measures to over 160 adolescents from three waitlist schools. Analysing the data produced from these interviews and surveys has been the main focus of our work in the past few months. In April, we submitted the abstract of our working paper ‘Feasibility Testing Health Economic Methods for School-Based Mental Health Interventions in India’ to the Health Economics Study Group (HESG) summer conference being held at the University of Oxford. This abstract was accepted, and, after a fair amount of frantic work, a full draft of the paper was produced for the conference by the end of May, ready for distribution to attendees ahead of the conference on 21-23 June. The conference was attended by myself and Hareth, as well as another one of our colleagues from the University of Birmingham.
Summary of our paper
The aim of our paper was to assess the feasibility, acceptability, and content validity of the EQ-5D-Y and the CHU-9D when used to evaluate the school-based mental health interventions delivered by SAMA. Thus far, two sources of data have been generated for this purpose: i) 10 think-aloud interviews with follow-up discussions conducted by myself and Muthu with adolescents in January 2023; ii) 164 pre-intervention surveys containing the two measures and open-ended follow-up questions collected from individuals in waitlist schools before SAMA commenced. Further data for the purpose of assessing the properties of the two measures will be generated by the post-intervention surveys which are due to be distributed towards the end of August.
We analysed the think-aloud interviews in two ways. Firstly, the think-aloud portion of the interviews were divided into individual segments pertaining to each item on the EQ-5D-Y and CHU-9D, with an error-scoring coding framework then applied to each segment to identify completion challenges exhibited by the participants. These challenges related to five potential sources of error: i) Translation, ii) Comprehension, iii) Retrieval, iv) Judgement, and v) Response. Second, we analysed both the think-aloud portion and follow-up discussions using a thematic framework developed to assess areas pertaining to content validity, feasibility, and acceptability. For the written surveys, we conducted quantitative analysis to calculate completion rates and assess the distribution of responses to the two measures, before applying our thematic framework developed from the interviews to the open-ended follow-up questions.
The analysis of the think-aloud interviews identified a number of completion challenges for both measures. However, error rates were higher for the EQ-5D-Y than the CHU-9D. Key issues for the EQ-5D-Y pertained to the translation and comprehension of the item ‘Having pain or discomfort’, and the wording of some items failing to induce participants to think about their health ‘today’. Completion challenges were also more evident on the EQ-5D-Y in the written surveys, with a lower proportion of adolescents completing the full measure. The qualitative analysis of both the interviews and the surveys indicated that the CHU-9D was the more preferred measure by participants for reasons including having more response options and more relevant content. However, some participants indicated that neither measure covered all important aspects of their quality of life, suggesting more is needed to capture influences from external factors such as social interactions, education, and material circumstances.
Overall, our findings suggest that researchers might prioritise the CHU-9D as an instrument for measuring QoL in school-going Indian adolescents. However, the measure was not without limitations, and careful attention should be paid to understanding country-specific factors that pose challenges to its usage in economic evaluations. Furthermore, we would expect the feasibility of both measures to be improved with some small language modifications.
HESG Oxford 2023
HESG was founded in 1972 and is the oldest organisation of its type in health economics. HESG conferences (or ‘meetings’) take place biannually (winter and summer), with this summer’s conference hosted at The University of Oxford’s Keble College (pictured). It was attended by over 100 academics from numerous UK and international universities, as well as individuals from various health economics consultancies. In a departure from a more traditional conference format, accepted papers are distributed to conference attendees who may then choose a paper which they would like to be the discussant for. The discussant then presents a 20-minute summary and discussion of their chosen paper, with the author of the paper then providing clarifications and responses to initial points of discussion raised. The discussion of the paper is then opened to the room for a further 30 minutes.
The discussant for our paper was Maria Kathia Cardenas Garcia Santillan from the University of Oxford Health Economics Research Centre (HERC). Unfortunately, due to a technical issue she was unable to present their slides to the audience, however she was still able to provide a nice summary of the paper and some useful starting points for the discussion. The paper seemed well-received by the audience, and the subsequent discussion brought forward some interesting ideas and reflections both on the current paper and for future research directions. Individuals showed particular interest in the challenges we faced in conducting think-aloud interviews in both English and Kannada, especially regarding the need for Muthu to provide real-time translation to me during the interview. Some suggestions were made regarding aligning our analysis to recommended guidelines on assessing content validity of questionnaires, ensuring that we considered our reflexivity in the qualitative research process, and ways in which we can utilise our results to ensure that EuroQoL implement our suggested adaptions to the EQ-5D-Y for use in this context. Some participants also indicated that they were planning on conducting similar studies with adolescents in other contexts including Peru and Ghana, and asked if they could contact me to discuss this with them.
Over 50 papers were discussed across the three days, covering a broad range of topics in health economics. Some particular highlights included a paper on developing a capability wellbeing measure for children and adolescents, which may prove to be a useful tool to take forward in the future when evaluating SAMA and other-school based interventions, as well as several papers on the impact of COVID-19 on various health outcome measures across different populations. On the Thursday evening, a banquet dinner was provided in Keble College’s dining hall (pictured) which provided an excellent opportunity to talk to a range of individuals working in the field of health economics and further spread the word on the work being delivered by SAMA.
Final reflections
This was a great first experience for me in attending and academic conference, and one I hope to repeat again soon. It was really encouraging that people seemed to be so interested in our work and the wider SAMA project, and I gained some valuable feedback on the work we have done so far. As always, I extend my thanks to the wider SAMA team for their contributions to our paper. I am looking forward to continuing with our work on costing SAMA and with the collection and analysis of the post-intervention surveys.
