[MUSIC PLAYING] We are the Association for Child and Adolescent Mental Health, or ACAMH for short. And this is ACAMH Learn. Welcome to Mind the Kids, a podcast series dedicated to exploring the latest advancements in child and adolescent mental health research and practise. I'm Clara, an academic clinical fellow in child psychiatry, and I'm passionate about understanding and addressing the diverse mental health challenges faced by young people globally. And in this series, I'm joined by renowned researchers and clinicians from around the world to discuss their cutting edge research, innovative interventions, and best practises in child and adolescent mental health. Today, I have the pleasure of receiving Dr. Karolin Krause, a senior researcher based at the Centre for Research in Epidemiology and Statistics at the Université of Paris Cité. Today, we're going to be discussing her paper measuring life impact of youth mental health difficulties as coping umbrella review of 80 instruments that was published at the JCPP. It's an honour to have you here, Dr. Krause. If you could start by telling us a bit about yourself and your research journey so far. Sure. Thanks so much for having me, Clara. It's lovely to be able to chat with you today. So as for my journey, I started out with training in sociology, actually in Germany and in London, and I spent three years working in the private sector as a programme evaluator evaluating aid programmes at the time for the Department for International Development. And in that role, we would always start any evaluation with sitting down with all programme stakeholders and trying to figure out together what were the key outcomes that a programme should achieve and that the evaluation should focus on. So at the end of those three years, I had the great opportunity to do a PhD in psychology at University College London, at the Anna Freud Centre under the supervision of Professor Miranda Wolpert. And I was going to work on outcomes measurement in child mental health. And as I came in with what in sociology, we call the look of the stranger, I was just surprised to see that the kinds of outcome priority negotiations that I was used to from my previous work were not at all taking place in at least not in youth depression research, which was what I focused on, but rather for the past two or three decades, people had been measuring depressive symptoms, often as the only key outcome to measure in a clinical trial. And that became the topic of my PhD. I did mixed methods research with young people, caregivers, and also clinicians to see how they thought of what was a good outcome of treatment for youth depression, and how that related to what was actually measured in the literature. And I found that there was quite a strong mismatch between this often quite unidimensional focus on depressive symptoms in the literature and the priorities of young people and caregivers. And I think maybe we can talk about that a little bit more. Also, I found that there was a lot of heterogeneity in how depressive symptoms were measured. Everyone was measuring them, but not at all in the same way. These two insights in the last year of my PhD led me to be a research fellow with the International Consortium for Health Outcomes Measurement, or ICHOM, where I was working with a working group of researchers, clinicians, young people, and caregivers. And we try to identify a core set of outcomes and outcome measurement instruments that everyone should use ideally in youth depression clinical care. Both at the same time, help us agree on a slightly broader set of outcomes that matter to everyone, and to try and measure them in a somewhat harmonised way. In that work, functioning in daily life came out as a core important outcome, but we struggled to identify a measurement instrument-- a good measurement instrument for that outcome domain. And that led me to my postdoc at the Centre for Addiction and Mental Health in Toronto, where I looked at measurement challenges specifically related to functioning and what I call life impact. And that is where the scoping umbrella review started that we're going to talk about today. And maybe just about where I work now. So I'm now a clinically epidemiologist and measurement scientist at the CRESS in Paris, and I currently work as a co-PI on a big Wellcome-funded research project that aims to develop a platform for evidence synthesis of measurement properties of patient reported outcome measures. Wow. That's a fantastic journey. And I love how one thing led to the other and, yeah, I just had like everything started with insights that you had working in policy and realising which is absolutely true. As a clinician, I think in clinical practise, we definitely focus a lot on symptoms, but actually, as you said, for patients and for carers like life impact, which I'm sure you're going to talk to us more about it and what it means. But yeah, life impact or what we're calling payment. It's a much more important construct. And speaking of that, so there's been a lot of discussion recently around mental health symptoms in children and young people, how they are increasing and what actual impact those symptoms have on a young person's functioning. And I think your paper address is a really important point in this whole scenario as you said, how we're even measuring those outcomes. And yeah, can you tell us a bit more about that and also the specific gaps your paper was trying to address? Yes, of course. So I think sometimes outcomes measurement can appear a bit like a technicality that you have to deal with when you do interesting research. But it is essential to evidence-based mental health practise, because the evidence that we want to use to make clinical decisions comes from outcomes measured in research. If we want mental health care to be person-centred, then we need outcomes measurement in research to be reasonably person-centred and to capture outcomes that matter not just to researchers, but also to all those involved in clinical decision making further down the line, that is young people, caregivers, and clinicians. And yes, so we've already spoken about it a little bit. But in my PhD, I found that depressive symptoms were important to young people, caregivers, and, of course, to clinicians. But almost just as important is the ability-- or is developing the ability to cope with symptoms. Some young people were quite articulate in saying that, especially in depression, symptoms often come back. It's a recurrent disorder oftentimes. And so it's good when they disappear, but it's even better to be prepared and equipped for what to do when they return. So coping was a big issue. And then functioning and quality of life, maybe especially for those with more severe depressive symptoms, where really there was quite tangible impairment in daily life. And so I've talked about the ICHOM core outcome set initiative that I was part of and where we identified functioning as a core outcome. But almost in parallel, it was also identified as really important by other initiatives. I was a co-investigator of the inroads initiative that was led by Dr. Suneeta Monga at the Hospital for Sick Children in Toronto. Another core outcome set initiative for youth depression, specifically for trials. And in the inroads initiative, functioning and quality of life also emerge as very important outcomes. And thirdly, the Wellcome Trust and NIMH have issued some measurement recommendations around common measures that those funded by them might want to adopt to rally the field around a smaller set of measures that we would all use. And they are some symptom measures. And then the WHODAS as a functioning measure for adults. But what is interesting is that across these three initiatives, everyone struggled to identify an established, self-reported youth functioning or quality of life measure that we could comfortably recommend as the one to use where we know the evidence is strong. The item content corresponds to what we thought should be captured within those constructs. And so I think there's a real gap there between knowing that functioning quality of life and broader life impact are important, but not knowing how we should really be measuring them. That makes total sense, and I think your review tried to fill that gap right by-- well, it's an umbrella review and you kind of reviewed all reviews looking at routine outcome measures in depth assessed this so-called impairment. So you looked at quality of life measures, well-being measures and you've touched on that. You said that actually the Wellcome is trying to encourage clinical trials in depression to use the same measures because there are so many different measures. And I think one thing that really struck me reading your paper was that you've identified 80 different instruments trying to measure life impact. So I thought this number was incredibly high. Were you expecting those many measures? I know you've worked previously in the area in your PhD, so maybe we're already aware of the heterogeneity and the number of instruments in the field. But yeah, I was just wondering. Yes, that's a great question. So I had seen fragmentation and measures used, multitudes of measures used. I already knew that we had over 200 depression measures quite famously developed in the field, but I was still surprised that we identified 80 different measurement instruments for this broader construct of life impact, and especially for functioning and quality of life, where we had, I think, 35 and 32 each. There were fewer well-being measures. And what was interesting also was how different they were. So some are self-report measures, some are parent or proxy report measures, other are clinician reported, some are short, some are long Some were developed in mental health, some were developed in physical health or in a general population. So that was in part why we wanted to start with quite broad scoping umbrella review that would compare these different what we call design characteristics to really map out this field and these many different instruments that we have available. That makes total sense. And you've touched on that briefly when you were talking about the co-production work that you did during your PhD. But I suppose one of your favourite findings also confirmed that actually, symptom severity and life impact don't always move along. And I think I suppose we see that in the clinic as well, like some young people that score really high on symptom measures, but actually they have relatively little impairment while others experience the opposite. So it's really important to focus on life impairment as opposed to quantifying symptoms. And do you think clinicians and researchers sometimes focus too much on symptoms alone? Yes, I completely agree with your observation there. And I think you've already said something important about why life impact is important to consider separately. A young person might have considerable symptom burden and yet not feel so impaired. And they may not actually have the same treatment needs as a young person that scores lower on a symptom measure, but that feels much more empowered. And where the family agrees that they face quite a lot of barriers in their daily life. And within a person-centred approach to care, the care plan that will be made for these two young people might be different. And that's why the DSM-5, for example, requires that symptoms are above a certain threshold, but also there is a presence of impairment or distress in order to warrant a diagnosis. So the importance of capturing both dimensions is already recognised there. You had asked previously about challenges with outcomes measurement. One challenge specifically for measuring functioning and symptoms is that they are often conflated together in the same measurement instrument. And when that happens, we struggle to really unpack the relationship between the two dimensions. We then struggle to look longitudinally what comes first and what improves first over the course of treatment. Does improvement in functioning lag behind improvement in symptoms or not actually what response to treatment, which of the two dimensions? So I think to unpack this, it's critical that we have really dedicated measurement instruments that capture either one or the other. 100%. And just to add my two cents, I think-- at the moment, I work in an inpatient unit. So that means we often see people when they are most acutely unwell. And because there is this ethos which is great to try to treat people in the community for as most as we can, we end up not seeing them for such a long period of time. We end up seeing them for two weeks, maybe four weeks if it's a more complex case. But we only see them for that very kind of cross-sectional window of time. And actually, as you said, sometimes it might be that the functional recovery lags behind symptom recovery. And it might be that the clinicians are not necessarily going to see it in an inpatient admission, which I think is that's why it's so important to have routine outcome measurement across health services because then you can just follow up these people consistently. And you can have like-- yeah, basically, you can have good measurements of how they are doing. And we can know for sure if that treatment or that intervention is being effective or not. Of course, outcome measures are also really important inpatient psychiatry. I'm not saying there, but I think it's really important to have that longitudinal follow up. There is another aspect where I wonder what you make of it, Clara, with your experience from the clinic. I had the chance of being a co-author on a network analysis led by Dylan Johnson at the Centre for Addiction and Mental Health. I think the protocol for that just got published or accepted for publication. This network analysis looks at how functioning sits relative to symptom clusters for common disorders that we see in intake samples in youth mental health clinical settings. And Dylan showed that functioning sits very centrally between these different symptom clusters, which makes sense because different clinical presentations can be associated with functional impairment. And so if we think of cross diagnostic approach to assessment and possibly outcome tracking, given that we often have multimorbidity in young people we see in clinical settings, especially when they're more severe, then it makes even more sense to have a dedicated approach to capturing functioning. And then we can also capture the different symptom clusters. And the other thing, speaking of symptoms and of things that don't fit into clear cut categories, one thing I found really interesting in your paper was that more than a quarter of the instruments you analysed, they were classified differently across reviews. So for example, some measures that were labelled as quality of life by a research team, they were labelled as well-being by another. How do you think this happened like in terms of how can different groups look at the same questionnaire and reach different conclusions about what it is measuring? And I mean, your current project is about that, but can you tell us a bit more how moving forward we can ensure consistency when using the same measures? That's a great question. So yes, I think the initial problem is that neither of these three constructs has an established consensus definition that everybody adheres to especially quality of life has been defined in various different ways. Well-being also has several strands of defining it, and so that is already complicated because sometimes functioning is considered a nested element of quality of life, for example, as the objective part. And I put this in quotation marks, the kind of objective part of quality of life. And sometimes quality of life is conceived in a much more subjective way, so that it starts to resemble well-being. So already the definition borders between the three constructs are porous, and that doesn't help. So I think part of the confusion comes from there. So in the review, we have picked the most widely used or most widely cited self-report measures developed specifically for children or youth. And we looked at the development quality, the kind of development methodology for those instruments. And we used some of the methodology developed by the COSMIN framework. Although we did not do a formal COSMIN appraisal. That's important to note because that is more substantial. So we looked at the developmental quality and a key aspect of that is to define the conceptual framework and provide a definition of the construct to measure, and then to work with the target respondents to make sure that the content of the items that make up the measure actually represents that construct. And we saw that that was not always done in the development of the measures that we were looking at. Neither were they always was the construct always clearly defined at the outset. Often, there was not a really formal conceptual framework. And oftentimes, young people and caregivers were not involved qualitatively or it wasn't clear whether they had been involved in defining the items that would make up the measure. And I think that lack of conceptual clarity and of gaps in the development of a measure contribute to issues and uncertainties around its content validity. What the content actually is, and if that content captures the target construct. And then that leads us to what are called jingle-jangle fallacies, which we see quite often in the field, where a jingle fallacy is when we assume that two measures assess the same construct because they have the same name, or they are said to measure the same thing. But when you look at the actual content, they do not assess the same thing. And similarly, a jangle fallacy where we think that two measures capture something differently because they have different labels or use different terms. And yet, actually the content overlaps a lot. And that happens sometimes with these kinds of measures, with quality of life functioning and well-being measures. And so I think that contributes to this situation we're seeing, where some research teams may interpret the same measure differently than others. Something that we're working on is a content analysis and item content analysis of quality of life functioning and well-being measures to unpack a little more, how much overlap is there in the items and what are the distinct dimensions for each of the constructs. I think that psychometric work is so important because as you said, especially measures that were developed a long time ago and they might have been developed without input from patients and without input from people with lived experience and parents and carers, et cetera. And also, as you said, sometimes the content validity is not there. So yeah, I think that's going to be really, really helpful. And speaking of gaps and things that are not there, the other finding that I was also-- well, there were lots of findings because as Karolin's paper is a review of 80 studies, but we're just highlighting a few in this podcast today. But yeah, the finding I wanted to bring to attention was that there was no specific instrument that was validated in the 19 to the 24-year-old population. And it just occurred to me that this is a particular strategic and key age range, because usually, that's the age that people are transitioning from chem services to adult services. In England, it's even earlier, the transition at 17. But I don't know how it is in France. But yeah, I think it is just such an important age. And in terms of epidemiology, it's also an age range that we know that the incidence of depression, for example, in young girls increases around that age considerably. So yeah, as an epidemiologist, this results surprised you or you were already expecting it and how did you make sense of it. Yes, it's a really interesting question, I think this one about the young adults. I was not deeply surprised because we had already seen this gap in the work with ICHOM for the core outcome set, and we had already identified it as a gap. In terms of why that is, yes, it's interesting how this age group just somehow gets lost. I think clinically, they sometimes get lost in terms of their transition journey. Sometimes they have to transition early, even though not all of the youth in that age range might be ready to go into adult services in a much more autonomous manner, be much less supported than they would be in CAMHS. And in other systems, they're able to transition later. There's a colleague at the Centre for Addiction and Mental Health in Toronto, Dr. Kristin Cleverley, who works specifically on the transition of this age group and what scaffolding could be put in place. How could they be accompanied to achieve that transition. So I think clinically, that's very interesting. And from a measurement science perspective, most measures are either validated up to 18, or they are validated for adults. I don't think I know of any measure, maybe with very few exceptions that was designed specifically for young adults or even explicitly validated or adapted for young adults. It's very rare. I think in part, it has to do with how we work as researchers. When you're interested in how well a tool performs, say, you're doing a systematic review, you will either end at 18 or you do an adult review. Because if you do up to 24, then you will have to go through all the adult literature. Your search will bring up so many more studies, most of which are not relevant. And then for this age range, 19 to 24, we often have opportunistic studies that involved university samples, first year university students just because they were there. So it's interesting these different barriers to researching this group more properly. And for me, in measurement science as a field, that should be a priority that we become interested, more interested in how a measure functions differently in different age groups and not just child versus adult, but that we are a bit more fine grained than that and specifically become more interested in this age group where brain development is not complete typically at 19 years. And there are, as you said, there are so many changes in their lives in that period and challenges. So yeah, I think it's critical to be more interested in measurement in that. You mentioned the brain development. I was going to say that as well that there is-- I know the evidence is not 100% agreeing on the age cutoff, but there are some brain development scientists, like Professor Sarah-Jayne Blakemore in Cambridge that support the notion that actually prefrontal cortex development goes up until age 25. So we know there is robust evidence to support that at least until early 20s, that there's still development going on. And I think, as you said, it's such a key age with lots of transitions going to university, getting your first job, moving out of your parents' house usually. And it's odd because I worked in an adult service. And here in the UK, the age cutoff is 18. So you can't be admitted to a CAMHS inpatient unit if you are 18. So we have this kind of unit that admits 18 to 65. And it is just a bit sometimes weird because, as you said, it's just very different because some 18-year-olds-- obviously every person is different, but some 18-year-olds definitely still require much more support and don't have the level of independence that you would expect of a 30-year-old adult. So it's kind of-- yeah, it feels odd to have these people-- all these people in the same environment with very different developmental needs. Speaking of clinicians and things that are important to clinicians are lots of clinicians will recognise the situation where a parent recognises there is a lot of impact as a result of, for example, their child's depression. While maybe the young person doesn't realise or doesn't recognise this impact or even the other situation. A young person may say that they are having a lot of internalising symptoms, but the parent does not see it. And if you have multi-informant measures you sometimes get discordant results. So your review found that the measures often used multiple informants for measures assessing functioning, whereas well-being and quality of life measures, they rely much more heavily on self-report. So why do you think they were like structured this way? And do you think we should all move towards more like move informant based measures. Yeah, as an epidemiologist, I'm super curious to hear your thoughts. Yes, indeed. We found that the functioning measures, there was much more variety in terms of informants than with regards to the other two domains. And I think that comes from a kind of legacy where functioning and impairment in daily life was seen as something that could be objectively observed and best ascertained not by the young person themselves, but by an adult observer, either the clinician or the parent or caregiver. And so we have-- I think to extent that is true. The parent certainly has a view on how well the young person functions in quotation marks, functions in the home environment. And we have teacher rated measures as well in terms of how the person functions at school and functioning. There are definitions of functioning as how well does the young person comply with role expectations for that specific age and in that cultural context. And so, of course, someone's role performance can be observed by others. So I think it makes sense that there would be report from other informants for functioning given this definition. But young people certainly have a very important own experience of how well they can function or not. And I think it's absolutely critical that we assess that subjective notion and that it complements the judgement of others. So I don't think, 2026, there is an excuse not to assess functioning also with regard to self-report. And for the other two constructs, it's a bit different because they are more subjective and more experienced. I think it's almost a research question to what extent is it valuable and does it add value to have the caregiver assess a young person's quality of life and well-being. Or can we just rely on the young person's own notion given that it is an inherently subjective construct. Yeah, I think you're right. Especially because oftentimes, multi-informant measures, they are more time costly and sometimes they require more people to do the interviews with parents. And like sometimes I'm just thinking like about the doorbell, for example, the doorbell doesn't measure quality of life specifically, but it measures. It is like a multi-informant interview and uses also sometimes even school reports. Yeah, I think, yeah, that's a research question in itself, what's the added value of having multi-informants for things such as quality of life and well-being. I think it's interesting because I think this is something that also evolved with the notion that actually, we need to put young people at the centre of research because like reading old studies, like reading studies from the '80s sometimes, those classical studies, especially for younger children like that usually that 8, 11 to 16 age bracket. Lots of the times, it's just the parent report. You don't have a young person's report. But I think fortunately, the field, as you said, is moved forward. And now really puts a really important emphasis on co-production and valuing young people's voices, which is really important. I agree. And maybe just to add, I think one other interesting question that is directly related is maybe more interesting to capture not the parents' perspective on the young person's quality of life and well-being, but the parent's perspective on their own quality of life and well-being. That might add a more valid and interesting dimension. Because, of course, the young person-- how well the young person is would have an impact on how well everyone else is in a family system. And so maybe then that would be a more interesting complimentary assessment. Yeah, 100%. And I think this is also another really interesting development, how some child development researchers are now much more focused on the whole system around the child. So I have a colleague, she's just about to defend her PhD, actually. And her PhD is dedicated to looking at the impact of mental health conditions in the siblings. So how having a sibling with a severe mental health condition has an impact on your quality of life and well-being. And as you said, I think it's more interesting sometimes maybe to look at how it's impacting the parent's quality of life because that's going to have a ripple effect on the whole family, which, yeah, it's really important. Yeah. And well, speaking of future research and future directions, I just wanted to say a huge thank, Karolin, for being here with us today and for such an interesting discussion. And I wanted to ask you, looking ahead, where would you like to see the measurement science field going to next? Yes, thank you, Clara, for having me. It's been so great discussing with you about these topics, which are obviously which I'm very passionate about. So it's lovely to have this space to chat about them in terms of where measurement in mental health is going, that is really something that we're thinking about in the Wellcome funded project that I'm a co-PI of. The PI is Astrid Chevance, who is a colleague of mine at the CRESS at Université of Paris Cité. And in this project, we're trying to think about how measurement practise in mental health can itself become more evidence-based. Because while we are using-- I mean, specifically interested in patient reported outcome measures in that project while we are all increasingly using those measures to collect evidence which should then inform clinical decision-making, it's not so established yet that we should also use evidence to select the problems we use in our research. So we've been doing a qualitative survey of 43 researchers around the world. And what we see is that not always have researchers great confidence that they can find-- that they can identify all the measurement instruments available to assess a given construct and then that they can easily find all the relevant research on the psychometric qualities or the measurement properties of those instruments. And so selecting a measure for a research project becomes a huge task if you want to do it well and to appraise all your options. And then sometimes what is just much more feasible is to go with a measure that you've used before, or that the lab has used before, or that is widely cited. And I think that puts our field in the situation that we're currently in, where we have many different measures. People don't quite agree on which ones to use. And there remains a lot of uncertainty about how well each of these measures would perform in a given use context, because we always have a specific population with specific language requirements in a given setting. So what we're trying to think about in this Wellcome funded project is how could we develop evidence-based measurement practise and measure selection in our field and give researchers resources that can help us be more evidence-based in our selection of measurement instruments. Very much looking forward to seeing the evidence synthesis platform when it's out. I think that's going to be a huge game changer. And yeah, we didn't even touch upon this. But another really interesting thing that Karolin's paper did was to look at geographical locations. A hugely important consideration is the cultural context and populations are different. So was that measure validated in that specific population or was it developed with that specific population, which also brings on the regional inequalities thing? But yeah, that's a whole other podcast. But yeah, I think that's going to be so, so important when it comes out. Well, I'll be happy to come back anytime to chat more with you about these topic. But we would love to have you. Thank you so much again for a lovely discussion. And I think, yeah, we could all see your passion for this topic shining through and it is so, so important. And as you said, I think lots of people, they look at outcome measures in the title and they think, oh, that's technical, but actually, it's how where-- it's the whole kind of-- when you see a clinical trial being published and this clinical trial reports a positive result, this positive result is based on an outcome measure. So if we're using a flawed outcome measure, that result might be not valid at all, basically. That's how important I think your work is. So thank you so much. Thank you, Clara. Thank you so much. Thanks. And thank you for the lesson. [MUSIC PLAYING]