Data collection for health economics
Background on work package 6 of Project SAMA
Work Package 6 comprises of the health economics work for Project SAMA. Health economics is concerned with how we should allocate scarce health care resources. Health economists can evaluate new health interventions using economic evaluations where the costs and outcomes of one or more health interventions are measured, with this information then utilised to calculate the cost effectiveness of a health intervention.
We can measure health outcomes in a number of ways for an economic evaluation. One approach is to utilise quality-adjusted life years (QALYs), a metric that combines health-related quality of life (HRQoL) and length of life (in years). One QALY is equivalent to one year in full health. There are a number of patient reported outcome measures (PROMs) that can be used to measure HRQoL. Some of these have been designed for use with children and adolescents. However, whilst these have been validated in high-income and Western settings, they have not been used before for measuring QoL amongst Indian school-going adolescents. Thus, one of the aims of WP6 is to feasibility test and assess the appropriateness of using two of these measures, the EQ-5D-Y and CHU-9D in this setting. The EQ-5D-Y consists of two parts. The first is a descriptive system assessing quality of life in five dimensions, with respondents choosing the response that best described them from three levels. The second part consists of a visual analogue scale (VAS) for assessing self-perceived overall health, with respondents indicating what they believe their health level to be on a scale of 0 to 100. The EQ-5D-Y is available in English and in Kannada. The CHU-9D just consists of a descriptive system, although it measures quality of life across nine dimensions with responses on five levels. The CHU-9D is available in English, but a Kannada translation was not available. Members of the NIMHANS team therefore produced an inhouse translation for use in this study.
We use the cognitive interview technique called ‘think-aloud’ interviews (Willis, 2005) as the primary method for feasibility testing the two measures. With this technique, the adolescents are presented with the questionnaires and asked to verbalise their thought process as they complete them. This allows us to ascertain any problems they may encounter regarding their comprehension, retrieval, judgement, and response difficulties. The transcripts from the interview can then be analysed to identify any issues which threaten the validity of the measures. Additional feasibility testing of the measures will also be conducted through administering the questionnaires pre- and post-intervention in waitlist school adolescents. Not only will this provide an indication of the impact that SAMA had on the adolescent’s QoL, but it will also enable us to further identify any completion challenges by analysis the proportion of missing responses and to ascertain the presence of ceiling effects.
The second aim of WP6 is to cost the SAMA intervention. Costing a health intervention involves three steps: i) identification of resource use; ii) measurement of resource use; and iii) valuation of resource use. Two approaches can be utilised for costing – a ‘top-down’ or ‘bottom-up’ (micro) approach. The former derives costs from overall prices for an intervention, whereas the latter identifies, measures, and values individual activities linked to the intervention and then aggregates these up. For WP6, we are adopting a micro costing approach. This will allow us to not only cost SAMA in its current form, but also explore the potential costs that would be incurred from scaling-up SAMA.
The final aim of WP6 is to utilise the outcome and cost data we collect to conduct an exploratory partial economic evaluation, known as a cost-consequence analysis. With this, we will present the costs and outcomes in a disaggregated manner to help provide preliminary evidence on possible the cost-effectiveness of SAMA to decision makers. By conducting this feasibility study, we can also help to identify any challenges in implementing traditional health economic methods to this setting, allowing for adjustments to be made if necessary in future full trials of SAMA or other related interventions.
Data collection in India
My visit to India had two key aims. The first was to conduct the think-aloud interviews with adolescents in schools to feasibility test the EQ-5D-Y and CHU-9D. The second was to discuss the information I require for the micro-costing of SAMA with members of the NIMHANS team and the potential avenues for upscaling SAMA with government officials. Throughout the trip I was accompanied by Muthu to the schools and to meet the officials.
Muthu and I conducted one-to-one think-aloud interviews with 10 adolescents across three schools over two days. On the first day, we interviewed five adolescent girls at a private school in Bangarpet, Kolar. On the second day, we interviewed three boys and two girls at government schools located in Hoskote, Bangalore. The interviews began with us explaining to the participants what the purpose of the interview was and the tasks we would be asking them to complete. Subsequently, we presented the participants with the EQ-5D-Y questionnaire, asking them to complete it whilst verbalising their thought process. The adolescents were then asked to repeat this task with the CHU-9D questionnaire. On the first day, the EQ-5D-Y was provided in English and the CHU-9D in both English and Kannada. On the second day, a Kannada version of the EQ-5D-Y was instead provided, although this had some translation mistakes which were quickly picked up on by the adolescents! We had originally planned to conduct five interviews in English and five in Kannada, however given the purpose of the think-aloud technique is to ascertain any problems encountered in the participant’s response process by getting them to verbalise their thoughts, we agreed that we would get more out of the interviews by encouraging the adolescents to think-aloud in Kannada. Muthu would then translate what was being said to me during the interview where necessary. Following the completion of the two questionnaires, we then asked the adolescents some follow-up questions whereby we probed them on any difficulties they may have encountered and asked them if they felt anything was missing from the questionnaires which they felt was important to their quality of life.
Despite some of the initial language difficulties, after the first couple of interviews we had refined the interview process to work smoothly with the necessary translating. We did have to deviate somewhat from the traditional method of conducting a think-aloud interview, whereby the interviewers remain silent whilst the questionnaires are being completed, in order for Muthu to translate what was being said to me in real time. Some of the participants also needed some extra encouragement to continue thinking aloud whilst they were completing the questionnaires. However, overall, the interviews went very well. The adolescents provided useful thoughts and insights on the questionnaires, despite some of them being understandably nervous at the start of the interviews. We are now in the process of translating and transcribing the interviews for our analysis. The next steps will then be to separate the transcribed interviews in to 15 segments reflecting each of the dimensions of the EQ-5D-Y and CHU-9D and the EQ-5D-Y VAS, and then analyse each segment using a coding framework for error identification based on the survey response model (Tourangeau et al., 2000).
Muthu and I also met with two government officials to discuss avenues for upscaling SAMA for future delivery to more schools in Karnataka. These discussions primarily focussed on availability of funding. We received some encouragement in this regard, and Muthu and the rest of the NIMHANS team will explore this area further. It was also very useful to meet Prachi, who is involved in another school-based intervention, the SEHER Project, which is currently being upscaled in Goa. I plan to have further discussions with her on how we can apply lessons from SEHER with regards to upscaling to SAMA. Visiting the schools and meeting with the NIMHANS team was also useful in providing me with a better understanding of the resources that are being utilised to implement SAMA, with the identification of these resources being the first step in micro-costing the programme. The next steps are then to measure and value these resources, with Muthu advising me that this information will be available from members of the NIMHANS team. We are now in the process of collating this information. When finalised, we will be able to provide unit costs for each resource use and indicate the cost per pupil and cost per school of delivering SAMA.
Further data collection
Further data collection will take place with around 50 adolescent participants from the waitlist schools. We plan to administer a pre- and post-intervention quality of life survey in these schools. These surveys have been designed and will be distributed by members of the NIMHANS team. Both surveys will ask the participants to complete the EQ-5D-Y and CHU-9D measures. This will allow us to measure change in quality of life after the adolescents have received the SAMA intervention, which will then be used in the cost-consequence analysis. The pre-intervention survey will also ask some follow-up questions on how the adolescents found the two measures. These questions are similar to those used in the follow-up part of the think-aloud interviews and will therefore provide further data on the feasibility of using these measures. The post-intervention questionnaire instead will ask the adolescents some questions on how they felt SAMA impacted their quality of life, general health, and mental health and wellbeing, as well as providing them with an opportunity to make any further comments on the impact the SAMA programme had on them.
References:
Tourangeau, R., Rips, L., Rasinski, K. (2000) The Psychology of Survey Response. Cambridge University Press, New York.
Willis, G.B. (2005) Cognitive Interviewing: A Tool for Improving Questionnaire Design. Thousand Oaks, CA:Sage.