We are The Association for Child and Adolescent Mental Health, or ACAMH for short. And this is ACAMH Learn. Welcome to Mind the Kids Podcast from The Association for Child and Adolescent Mental Health, known as ACAMH. Today, the key question is, what do reward learning and anhedonia tell us about how young people respond to rewards as they move through adolescence? I'm Mark Tebbs. I'm your host for today. I spent my whole career working in mental health from frontline service delivery to Director of Mental Health Commissioning. I'm currently a chief executive of charity, and I'm a trustee of a mental health organisation and also a career coach. I'm really pleased to be hosting these podcasts. It gives me the opportunity to speak to some of the leading clinicians and academics in the forefront of children and young people's mental health. So if you're a clinician, if you're an educator, a parent, or just curious about mental health, I hope these discussions leave you with some fresh insights and practical tools to help. Don't forget to subscribe where you get your podcasts from and also on your YouTube channel. Just search A-C-A-M-H, and you'll find us across all socials. Today, I'm joined by Dr. Holly Sullivan-Toole, who is based in the University of Minnesota and working out of the Masonic Institute for the Developing Brain. We'll be discussing her recent research published in ACAMH's Journal of Child Psychology and Psychiatry, about reward learning, anhedonia, and how young people update their expectations based on rewards and losses. We'll also explore what the findings may tell us about adolescent development, and what might happen when reward processing is blunted. So let's get started. Welcome, Holly. It's an absolute pleasure to be speaking with you today. Yeah. Nice to be here, Mark. Thanks so much for having me. I'm really excited to talk about this work. Brilliant. We're looking forward to the podcast. So we'll start with a little bit of background. Just set the scene a little bit. So who did you collaborate with on the paper? Yeah, it was a great team. The second author here, Jeremy Haynes, he took the lead in developing the computational model specifically for this task with our other computational collaborator, Nathaniel Haines, who had developed the original computational model for the Iowa Gambling Task. This work was out of the lab of Dr. Thomas Olino at Temple University, who led this big longitudinal development study looking at adolescent risk for depression. As part of the preparation for the podcast, I was reading a little bit about reward learning and anhedonia. So could you just tell us what those are and why they're so important for your study? Yeah. So, broadly, anhedonia is just blunted or decreased response to rewards in the environment. So not enjoying, for example, food, or time with friends, or not looking forward to it. And anhedonia is this transdiagnostic symptom that is found across multiple disorders. It's found in depression. It's found in some instances of psychosis, in social anxiety. And it's actually a pretty serious symptom. It's associated with worse outcomes, mental health outcomes long term. It's pretty resistant to the current treatments we have. And it's predictive of the onset of depression, for example. And so it's a really important symptom that we need to understand better. Officially, in the DSM, it's defined as a lack of interest or a lack of pleasure in normally rewarding activities. But researchers are really digging deep into this concept, and I think we're really still in the process of understanding exactly which components of reward processing are affected in anhedonia. So is it primarily interest in rewards and motivation for rewards, or is it enjoying the rewards once you get them? Animal models and some emerging work in humans have shown these are different things. And you might have one of these processes could be really intact. You might enjoy rewards once you get them. You enjoy eating the ice cream, but you just don't have any motivation to go out. And you don't want to put your boots on and walk out in the cold and go to the store and get the ice cream. So there's these different processes that make up this big idea of reward responsivity, and a really important one that hasn't been studied nearly as much is, how do we learn about rewards? Again, if someone just puts the ice cream in front of you, you might enjoy it, but are you learning, when I hang out with these friends, it's an ice cream party, and when I hang out with these friends, it's a broccoli party? I don't know. So, how we actually update our expectations to seek out rewards in the environment, we don't know a lot about that. And we're really just trying to figure out which of these processes are most impacted in anhedonia. And so it's like a really exciting time to be working in this area. Brilliant. So as I understand it, there's almost like two core parts to the study. There's the looking about how reward-specific learning changes during adolescence and then the impact of risk of depression on that process. What was like known about young people learning from rewards during adolescence beforehand? And what were you hoping to learn from this study? There's a considerable body of evidence showing that reward responsivity is increased during adolescence. There's evidence of heightened brain activation in reward regions when adolescents are anticipating a reward or receiving a reward. We generally see more activation in adolescents than children or adults. And there's evidence of greater behavioural reward seeking and risk taking in adolescence. And this corresponds to what we know from animal models, showing that the dopamine system, which is known to support different aspects of reward responsivity, particularly like incentive motivation, we know that the dopamine system undergoes substantial development in adolescence. So we know just generally that reward responsivity seems to be heightened. But in terms of reward learning specifically, less is known, particularly when it comes to different components of reward learning, like learning from rewards versus learning from punishments. And the findings that we do have from the existing studies are kind of mixed. And I think what complicates the story is that the studies that have been done have used really different learning tasks. And we know that the best way to learn from rewards is actually really dependent on the specific environment or the specific demands of the task. So, for example, updating your expectations or learning a lot when you get a reward, that might help you be successful in one environment or one behavioural task, but in another task or another environment, it might hurt you. Like if you're updating too strongly when you get rewards. And maybe if that updating for rewards sort of outweighs the amount that you're updating when you encounter a loss. So maybe in some environments, learning too strongly from rewards can be a bad thing. But what does seem to be emerging from these sort of mixed findings is that adolescents seem to be like tuning their reward learning in ways that support performing tasks better. So I was really expecting to learn something very global, about the ways adolescents learn from rewards in a global sense. But as I've come to understand that the components of reward learning support optimal task performance on different tasks in different ways, and as I've thought more about the specific tasks that we've used, I began to realise there's just a lot to learn about reward learning, and it's complicated. But I do think our study made a contribution towards understanding how reward learning may become really fine tuned across adolescents in a way that supports mature functioning. Great stuff. So we'll get into a little bit more of that details and the kind of results and implications in a moment. I think it'd be really good just to set the scene a little bit more, particularly about how you went about the study. So who took part? What was the kind of age range of the kind of people in the study? So our participants were drawn from Dr. Tom Olino's Temple Adolescent Development Study. We call it TADS. And the study is based out of Temple University in Philadelphia. TADS was a longitudinal study of development of reward function in adolescents and the role of reward system development in relation to emerging psychopathology. The study started with kids aged 9 to 14, and we followed them for three years. And the data set's really rich. It contains a lot of measures, but a major focus of the study was the role of the developing reward system in the emergence of depression. So we excluded kids who had parents with a history of bipolar disorder or psychotic spectrum disorder. And we excluded kids with learning disabilities, neurological illnesses, developmental disabilities like autism spectrum, just because that wasn't really the focus of the study. So I specifically analysed data using the Iowa Gambling Task, and we had about just over 200 kids who completed this task at baseline at the beginning of the study. And then we followed them for about three years. We got the kids to complete the task five times, spaced about nine months apart. And by the end of the study, our kids were 17. So we have this data set covering a really large span of adolescents from about 9 to 17 years old. And since the study was based out of Temple University, which is located in North Philadelphia, we were able to recruit a really racially and ethnically diverse sample, which I think is really cool. So out of the 79% of the kids who reported their race, about 59% reported a racial minority status. So it's a really cool data set I think. Yeah, amazing. And then so you mentioned the Iowa Gambling Task. So what was that? Tell us what the kids were doing. Sure. The original Iowa Gambling Task is this game where participants see four decks of cards on the screen, like a virtual card game, and they choose a card from a deck. Deck A, deck B, C, and then they get monetary rewards and losses based on their choices. And the participants aren't really given any information about the outcomes associated with the different cards, but they can just learn by sampling. Two of the decks are good and lead to net monetary wins, and two of the decks are bad and lead to net losses. But in the original version, as a researcher, you can't really tease apart whether people are making their choices because they like the rewards or because they dislike the punishments. So we use this kind of different version that allows us to experimentally break apart approaching rewards and avoiding punishments. We call it the Play or Pass IGT. Kids still see all four decks of cards from the original task. But instead of them freely choosing, we just give them a deck. We say, like, here, on this trial, you get deck B, and do you want to play on it and see whether you get those rewards or losses, or do you want to pass on it and just nothing happens and you get a new choice? And it's a pretty cool task. We can look at how much they choose to play on the good decks, and that's a measure of approaching rewards. Or we can look at how much they choose to pass on the bad decks, and that's a measure of avoiding punishment. So just as a researcher, it's a really cool behavioural task. For the participants, it's a hard game, but I think it can be frustrating because you're winning money, and you think you're doing well, and then you get hit with a big monetary loss. So in the original task, it was random. You could choose between the four different decks, but you gave them one deck. How did that change what you were learning from the experiment? Yeah. So I mean, this task, what we're calling the Play or Pass IGT was specifically designed to give you those separate measurements for approaching rewards and avoiding punishments, which is really cool. You can't separate those things in the original task. We know these are separate systems. So they're really cool measures to get. So what did you find? What were the changes that you saw as the young people got older? When we just looked at what we can observe directly from the task, when we looked at the choices our participants made in the game, we see a subtle increase in adolescents playing from the good decks or more approaching reward as they get older. And we see a subtle decrease in them passing on the bad decks or avoiding punishments as they get older. But those separate effects that this game was designed to tease apart, those separate effects weren't statistically significant. So we really only saw the effect of kids. We did see an effect of kids just generally playing the game better as they got older, but we only really see that effect when we put approaching reward and avoiding punishment together. And then that was significant when we look at this measure that we can directly observe. So as the young people get older, their approach to the game gets more sophisticated, and that they both avoid punishments and seek rewards. Yeah. Definitely. Yeah, we're definitely seeing that. OK. Right. And the computational model in, which parts of learning actually changed around adolescence? DR. HOLLY SULLIVAN-TOOLE: We were really interested in looking at, like I said, reward and punishment separately. And the cool thing about the model is it gives us another way to look at that, which was handy since we didn't see those effects in purely based on the choices that they made. But our model lets us break apart the different processes in reward learning. So the reinforcement learning model that we used, it was developed specifically for this POP IGT game. It's developed by a group led by my colleague, Jeremy Haynes. We call it the Play or Pass Outcome Representation Learning Model or the POPORL model, and it allows us to break down participants' choice behaviour that we can directly observe. We can count the choices. It lets us break that into component parts, and specifically it gives us a measure of reward learning rate and punishment learning rate. And this tells us how much people are like updating in reaction to each little reward or each little punishment. The model also gives us a measure of win-loss frequency sensitivity, and that tells us how much people update their expectations based on how often they get a reward. So not the size of the reward, not the magnitude of the reward, but just how frequently it's occurring. And then, finally, we get a measure go bias, and that's just how often they generally choose to play. How often are they hitting play versus pass? So of these four measures that our model gives us, it's pretty cool. We specifically saw significant developmental change across adolescence in the two measures that are related to reward processing specifically. So we saw developmental changes in reward learning rate and in the win frequency effect, but not in punishment learning rate and not in go bias. But what did you find about reward learning rates? DR. HOLLY SULLIVAN-TOOLE: We found that reward learning rate decreased in our task across adolescents. And so that might seem sort of surprising because I've said earlier that we see enhanced reward responsivity in adolescence, and yet we found developmental decreases in reward learning rates. And I think that's because reward learning rate is this really nuanced construct. It's basically a measure of how volatile learning from rewards is. And I think when you use that word volatile, it makes a lot more sense, what reward learning rate is. It's basically how dramatically a person updates their expectations based on each little reward that they get. So we might be tempted to think more learning is better. We want higher reward learning rates, like the more the better. But in general, we actually want to see learning rates that are somewhere in the middle. We want to see people who are updating not too much and not too little, but just right, that sweet spot. Also, the optimal learning rate really depends on the environment or the demands of the task. Here in the Iowa Gambling Task, you get better performance when you have a relatively lower reward learning rate. And that's because rewards are pretty distracting in this game. If you're just sort of only paying attention to, oh, I got $100, I got $100, I got $100, and you don't pay attention to, oh, I just got hit with $1,000 loss, you're not going to do well in the task. So on the surface, it might seem like these developmental decreases in reward learning rates, like contradicting what we know about the reward system being amped up in adolescence, but it's actually consistent with the optimization of reward learning across adolescents. So the decrease is actually being more appropriate and sensitive to the task demands in an optimal way. Yeah, exactly. Yeah. OK. Amazing so the other part of your study was about the impact of risk of depression. Should we talk a little bit about that now? Is there anything else you wanted to say about the normative adolescent learning? DR. HOLLY SULLIVAN-TOOLE: We also found that the effect of wind frequency sensitivity was increasing across adolescence. It's a pretty interesting measure. It's sensitivity to how often a reward occurs, but it's ignoring the size of the reward. And again, that measure is really dependent on the environment or the task demands. So specifically in this task, it can be adaptive to ignore the magnitude of rewards, because, for example, our deck A gives you these high magnitude rewards but small, frequent losses, and it sort of leaves you with a net loss. And specifically in this task, it's better to have higher sensitivity to the frequency of rewards. And that is what we're seeing in our adolescents as they get older. So again, we're seeing something that suggests they're tuning their reward, learning for optimal performance in the game. So the learning is both in terms of the frequency and the magnitude of the learning. So the adolescents are having to evaluate these kind of combined effects and what the overall optimal strategy is. Yeah. And I think ignore features of the environment that may not be informative for optimal performance, so ignoring the size of the reward because the frequency in this case is really informative. Yeah. The other bit was the impact of risk of depression. So why did you want to include that within the study? So that's my core interest in this reward learning stuff is, how do these processes go awry? And how does that confer risk for depression? So we specifically looked at whether the kids' moms had a history of depression, because parental history of depression is a really robust risk factor for offspring going on to develop depression. Dads are important, too, but mum's history of depression appears to be like a particularly strong risk factor. And the literature suggests that one of the mechanisms by which a mum having depression and then a kid going on to develop depression, one of those mechanisms might be through passing on blunted reward processing or just not responding to rewards. So basically, our hypothesis here was that high-risk youth with the moms who have ever been depressed, that they would show reduced learning from rewards compared to the kids with the moms who hadn't been depressed. So we just wanted to see whether this blunted reward processing is something we see in the kids, with the moms with depression. So how was maternal depression defined? Was it clinical levels? Just wondered how you found those mothers for the study. Yeah. So this was using data again from Tom Olino's TAD study. Tom's a clinician and some other people in the lab are clinicians. I am not a clinician. So we had moms come into the lab and complete the SCID. It's a structured clinical interview for DSM disorders. And this was conducted with trained clinicians. And we counted the mum as having a history of depression if they had ever experienced major depressive disorder or persistent depressive disorder. I think that one used to be called dysthymia, but it's a low grade chronic depression. But if they had experienced either of those at any time in their life, we counted that as history of depression. And again, just the reason we looked at mum's depression was because that's a really robust factor in offspring going on to develop depression. So the moms might not have been in treatment or medication. It was just through the assessment. Yeah. We certainly didn't recruit based on any clinical characteristics of the parents. We just recruited a broad sample of kids. And then we had their moms come in. And as part of the study, the moms completed this clinical interview. And just through base rates in the population, we got a fairly large enough sample of kids whose moms endorsed having had depression at some point in their life. I don't think we actually got any information about whether they were in treatment currently or anything like that. Reward processing is something we see in the kids with the moms with depression. OK. Could you give us an example of blunted reward processing? What would that look like in a real-world context? The symptom of anhedonia is just blunted reward processing. It's defined as a lack of interest or pleasure in rewards. So it might look not responding to a text message from a friend or not being excited about going out to a social event, or not being interested in getting good grades or not enjoying achieving a goal or something like that. And I guess from a learning perspective, that behaviour doesn't get reinforced. And therefore, it might not happen. That engagement in activity, for example, might not happen because it's just not as rewarding. Yeah, exactly. I mean, I think that's the idea behind behavioural activation therapy, where they try to increase their engagement with rewarding activities because, if you're not seeking it out, then you don't get that reinforcement and then it becomes a cycle. Yeah, sure. And so what differences did you see in the reward learning for young people with that family history of depression? So we did have the hypothesis that reward learning measures specifically would be blunted or decreased in the kids with moms with depression. And that's what we saw. So we found lower reward learning rates again, just meaning that they're updating their expectations less when they get a reward. It's not really changing how they estimate the value for the next time as much in the kids who are high risk for depression. And again, while it's generally better to have a lower learning rate on this task, you also don't want a learning rate that's too low because you need some responsivity to rewards. And what we're seeing here is consistent with youth who are at high risk for depression just showing blunted responsivity to rewards. What was the size of that effect? Was it a very strong effect? This is a tough question. No. I would say it does not look like a strong effect, but I'm not totally sure how much that is a feature of just the measure itself. Learning rate is-- it's a reactivity to each tiny little reward in the environment. And they're also these latent measures. Mathematically, they're like bounded to be between 0 and 1. I don't know. So effect size is an ongoing thing we're thinking about. OK. I guess, as you were speaking, I was thinking even a small effect size in a cumulative way would be a massive from a life perspective could be absolutely significant. Yeah. That's how I've been thinking about it, too. If you're just going around in your environment, and even if you're just a little bit dampened, but it's happening all day long that that would have a big effect. Yeah. OK. So I'm just thinking about the implications really of the paper. So your paper suggests that these changes may reflect learning becoming more optimised during adolescence. So what can we safely take from that idea? And what do we need to be kind of careful about in terms of overinterpreting the results? Again, this is a really hard question. So it's really hard, I think, to take findings from any specific task and make big sweeping generalisations about the real-world adolescent reward responding. It's hard to say also what optimal learning from rewards in the real world looks like. I think it would depend on goals and preferences for different types of rewards and the need for different types of reward. If you're hungry, food is more rewarding than if you're full. So I think what we can learn from these tasks is somewhat limited in relation to real-world stuff. On the other hand, these laboratory tasks, these behavioural tasks are really controlled. It's really easy to define what optimal performance is, whereas in life, it's a little hard. Depends on the individual. What's great about these tasks is we can define how different measures support optimal task performance. It's very defined. Optimal task performance is making more money on this task. But I think it's also important to remember that these tasks are very specific. And the way that these measures support success is very specific to the task. So I have to be really careful about extrapolating our findings from any specific task to make these generalisations. But at the same time, these tasks that we're using, because they do give us so much control over the environment and definition of the parameters, I think it has a lot to tell us. I think what we're going to need to do over time is to use a bunch of different learning tasks and get these reward measures out of different tasks and put them together to get an overall picture of how adolescents are responding in different environments. Yeah, that makes absolute sense. I guess you're able to control the variables, but I guess you need more variables to be able to get the generalizability. Yeah, I think so. I think a really cool new direction that people are going, and I would love to go, is sampling from people's real lives, asking them in real time, using their phones what they're expecting from a situation, then how much they enjoyed the situation. And you can apply these models to people's real lives as well as laboratory tasks. And so I think that has some potential to move us into telling us a little more about real-world reward responding. OK, yeah. Very interesting. Was there any kind of implications or further research that you want to look at that's kind of been simulated from that part of the study? Yeah. Let's see. I mean, we did find blunted reward learning in the kids with a maternal risk for depression. We know from other studies that, if the mum is depressed, there's a higher risk for depression in the kids. But we actually don't have the data. 17 is a little bit young to really see a full emergence of depression typically. So it'd be really great to follow these kids or in another study to see, specifically, if blunted reward learning does in fact predict the onset of depression. And we don't really know that. Instead, we're just seeing kids that we think have a higher risk for depression are showing this effect. But we don't know if this effect then predicts depression itself. So it sounds quite early then, but potential for future research. Yeah, I think so. Cool. And have you got other research in the pipeline? Have you got other studies that you're starting or looking at? Yeah. So in the same data set, we're looking at the effects of childhood trauma, different aspects. Childhood trauma questionnaire, so physical neglect, emotional neglect, physical and emotional abuse, and how they impact these reward learning measures. It's very preliminary. I think we will find some pretty interesting stuff with this. Here at the University of Minnesota, I'm getting into neural systems that support these behavioural phenotypes. Specifically with Dr. Bart Larson here at Minnesota, we're looking at brain iron. And we know that brain iron supports dopamine synthesis. And there's this dopamine reward connection. So we're looking at how we can look at brain iron from just fMRI data that's just lying around. Lots of studies have fMRI data. And we can extract this brain iron measure. And we're looking at how that relates to the different reward learning parameters. Is it related to learning from rewards? Is it related to learning from punishments? Is it related to just a general tendency to push go? These are exciting studies that I want to see the answers to. Brilliant. Excellent. So look, we're coming to the end of the podcast. I wonder whether there's a final take-home message you'd like to leave the listeners with. Yeah. I think this is a really complicating world. That's what makes it all so exciting. I think it's really important to have these very rich ways of characterising behaviour. Using these models lets us not just look at, did people choose the rewarding things more, but it lets us look at really specific aspects of how those choices got made. Was it reward informing it? Was it punishment informing it? And I think, when we get these really rich behavioural phenotypes, we can start to connect them to specific neural mechanisms. And I think that's really exciting because we can start to think about mechanisms that underlie mental illness. And that points towards intervention targets. Brilliant. Thank you, Holly. Thank you so much for your time. It's been lovely speaking to you. Yeah. Lovely speaking to you too. Thanks so much for having me. Great stuff. I really enjoyed this chat with Holly and reflecting on just how complex reward learning actually is, and how important it is to look beyond those reward ideas and understand those underlying processes. I think it's also a reminder about how these laboratory task can tell us a lot, but also, as Holly really described well, how careful we need to be about generalising them. The work still really opens up some exciting questions about how we might better understand reward processing, mental health risk and the developmental pathways that may shape well-being over time. Plus, I got to learn a new word, anhedonia. So please don't forget to subscribe wherever you get your podcasts from. Also, like on the YouTube channel, search, ACAMH, and you'll find us across all of those kind of socials. So join me next week, and I'll be speaking to Emma Butler as we discover if sensory processing difficulties in infancy are predictive of child mental health. [INSTRUMENTAL MUSIC]