Childhood externalizing, internalizing and comorbid problems: distinguishing young adults who think about suicide from those who attempt suicide

Abstract Background While childhood externalizing, internalizing and comorbid problems have been associated with suicidal risk, little is known about their specific associations with suicidal ideation and attempts. We examined associations between childhood externalizing, internalizing and comorbid problems and suicidal ideation (without attempts) and attempts by early adulthood, in males and females. Method Participants were from the Quebec Longitudinal Study of Kindergarten Children, a population-based study of kindergarteners in Quebec from 1986 to 1988 and followed-up until 2005. We captured the co-development of teacher-rated externalizing and internalizing problems at age 6–12 using multitrajectories. Using the Diagnostic Interview Schedule administered at age 15 and 22, we identified individuals (1) who never experienced suicidal ideation/attempts, (2) experienced suicidal ideation but never attempted suicide and (3) attempted suicide. Results The identified profiles were no/low problems (45%), externalizing (29%), internalizing (11%) and comorbid problems (13%). After adjusting for socioeconomic and familial characteristics, children with externalizing (OR 2.00, CI 1.39–2.88), internalizing (OR 2.34, CI 1.51–3.64) and comorbid (OR 3.29, CI 2.05–5.29) problems were at higher risk of attempting suicide (v. non-suicidal) by age 22 than those with low/no problems. Females with comorbid problems were at higher risk of attempting suicide than females with one problem. Childhood problems were not associated with suicidal ideation. Externalizing (OR 2.01, CI 1.29–3.12) and comorbid problems (OR 2.28, CI 1.29–4.03) distinguished individuals who attempted suicide from those who thought about suicide without attempting. Conclusion Childhood externalizing problems alone or combined with internalizing problems were associated with suicide attempts, but not ideation (without attempts), suggesting that these problems confer a specific risk for suicide attempts.


Introduction
Youth have the highest rates of suicidal ideation and suicide attempts across the lifespan (Nock et al., 2008), with life time prevalence of 10-12% for suicidal ideation and 4-7% for suicide attempts (Nock et al., 2013;Orri et al., 2020). Suicide attempts represent a major public health concern, as they are associated with a range of long-term consequences such as mental and physical health problems, socioeconomic adversities and death by suicide (Geoffroy, Orri, Girard, Perret, & Turecki, 2020;Goldman-Mellor et al., 2014;Mars et al., 2014). To orient preventative efforts in youth, a better understanding of the modifiable childhood factors that are associated with suicidal ideation and attempts is crucial.
First, it is unclear whether externalizing, internalizing, and comorbid problems are distinctly associated with suicidal ideation and suicide attempts, as most existing studies have combined ideation and attempts, thus failing to examine mutually exclusive categories (Shin et al., 2009;Sourander, Helstelä, Haavisto, & Bergroth, 2001). Moreover, to our knowledge, no studies have examined whether childhood externalizing and internalizing problems distinguish those who think about suicide without making an attempt from those who attempt it. This is important as a significant proportion of individuals who are thinking about suicide will not attempt in a near future (Mars et al., 2019b). According to the ideation to action framework, the development of suicidal ideation and progression from ideation to attempts are distinct processes with distinct predictors and mechanisms (Klonsky & May, 2014). Several studies support this framework and have identified specific risk factors for either ideation or attempts (Mars et al., 2019a(Mars et al., , 2019bMay & Klonsky, 2016;Orri et al., 2020). As such, it is crucial to better understand whether childhood externalizing, and internalizing problems are differentially associated with suicidal ideation and attempts.
Second, though co-occurring externalizing and internalizing problems are common (Gilliom & Shaw, 2004) and have been associated with worse mental health outcomes, such as depression (Winsper, Bilgin, & Wolke, 2019) and substance abuse (Brook, Lee, Finch, & Brook, 2015), than either externalizing and internalizing problems alone, only two longitudinal studies have documented associations of comorbid externalizing and internalizing problems with suicide attempts (Brezo et al., 2008;Sourander et al., 2009). While children presenting with externalizing and internalizing problems were found to be at greater risk of attempting suicide by early adulthood than those with no problems, these two studies did not assess whether comorbid problems (a) were more strongly associated with suicidal ideation (in the absence of attempts) or attempts than externalizing or internalizing problems alone and (b) did not distinguish those who thought about suicide without attempting it from those who attempted (Brezo et al., 2008;Sourander et al., 2009). Additionally, the co-occurrence between externalizing and internalizing problems is better captured using repeated behavioral assessments throughout childhood and modelled using latent class growth analyses, such as group-based multitrajectories (Wiggins, Mitchell, Hyde, & Monk, 2015;Winsper et al., 2019).
Third, while females are more likely to experience suicidal ideation and suicide attempts than males (Cha et al., 2018), findings on sex differences in the associations between externalizing and internalizing problems and suicidal risk have been scarce and inconsistent (Brezo et al., 2008;Sourander et al., 2009). For example, one study found that comorbid externalizing and internalizing problems were associated with suicide attempts and death by suicide in males but not females (Sourander et al., 2009), while another study found associations of comorbidity with suicide attempts in females but not males (Brezo et al., 2008). Given the mixed state of the literature, there is a need to clarify sex differences in the association between childhood externalizing and internalizing problems and suicidal risk.
Using data from a large population-based cohort of children followed from 6 to 22 years of age, our aims were to capture the co-development of externalizing and internalizing problems over time with group-based multitrajectories to (a) test their associations with suicidal ideation and suicide attempts by early adulthood, and specifically to test whether comorbidity carries an additional risk with respect to externalizing or internalizing problems alone, (b) examine sex differences in their associations with suicide outcomes, and (c) examine whether childhood problems distinguish young adults who think about suicide without attempting it from those who attempted suicide.

Participants
Participants were from the Quebec Longitudinal Study of Kindergarten Children (QLSKC), a population-based sample of children attending kindergarten in French-speaking schools in the province of Québec in the 1986-1987 and 1987-1988 school years and followed up until 29 years. The cohort was selected by a random sampling procedure stratified by both administrative region and school board size; see cohort profile in Rouquette et al. (2014). The cohort comprised sub-samples of (a) 2000 children representative of Quebec kindergarteners and (b) 1017 children, who, in kindergarten, scored at the ⩾80 th percentile on the disruptive behaviors scale from the Social Behavior Questionnaire (SBQ) (Tremblay et al., 1991). QLSKC was approved by the University of Montreal Ethics Board and Statistics Canada. Written informed consent was obtained from children's parents prior to participation.

Measures
Teacher reports of externalizing and internalizing problems from 6 to 12 years old School teachers rated children on the SBQ (Tremblay et al., 1991) at ages 6,7,8,9,10,11, and 12 via mail-in questionnaires. Given that teacher raters differed each year, the yearly assessments were independent. The SBQ has good predictive validity for various outcomes such as educational attainment, employment earnings and psychiatric problems (Brezo et al., 2008;Pingault et al., 2014;Rouquette et al., 2018;Vergunst et al., 2019) and showed good psychometric properties (Tremblay, Desmarais-Gervais, Gagnon, & Charlebois, 1987). Items were rated on a three-point scale according to the frequency of the behaviors, with 0 indicating never applies; 1, applies sometimes; and 2, frequently applies. At each time point, we derived externalizing (13 items e.g. 'destroys own or others' belongings', 'fights with other children'; α = 0.89-0.93) and internalizing (five items e.g. 'tends to be fearful or afraid of new things or new situations', 'cries easily'; α = 0.61-0.76) problems scales with confirmatory factor analysis. Derived factor scores were used as a measure of externalizing and internalizing problems latent constructs (higher scores indicating higher problems; see online Supplementary Tables S2  and S3).

Psychological Medicine
Suicidal ideation and suicide attempts by age 22 Suicidal ideation and suicide attempts were assessed at 15 and 22 years using the Diagnostic Interview Schedule for Children (Breton, Bergeron, Valla, Berthiaume, & St-Georges, 1998) (self/ parental-report) and the Diagnostic Interview Schedule for Adults (self-report) (Robins, Helzer, Ratcliff, & Seyfried, 1982). Participants were asked by a trained research assistant visiting their home: 'have you/your child thought seriously about killing yourself/themselves?' and 'have you/your child tried to kill yourself/themselves?'. Either parental or self-report of items administered during adolescence were sufficient in classifying participants as having ideated or attempted suicide.
For the current study, we created a three-category outcome: (1) participants who never experienced suicidal ideation or attempted suicide, labelled non-suicidal, (2) participants who have experienced suicidal ideation but never attempted suicide, labelled as suicidal ideation, (3) participants who have experienced suicidal ideation and attempted suicide, labelled as suicide attempts.
Child and family characteristics at age 6 as potential confounders Potential confounding factors were assessed at 6 years old using questionnaires administered to mothers. These were sex (male v. female), maternal age at childbirth (<20 v. ⩾21 years), family structure (two parents v. single-parent), maternal education (years of schooling), maternal occupational prestige (measured with the occupational prestige scale (Blishen, Carroll, & Moore, 1987)) and positive and negative parenting attitudes (assessed with the Emotional Climate for Children Scale (Falender & Mehrabian, 1980); see online Supplementary Tables S4 and S5).

Statistical analyses
First, we modelled the co-development of teacher-rated externalizing and internalizing problems from ages 6 to 12 using multitrajectory modelling (i.e. parallel processes latent growth curve analysis) in Mplus (version 7.3). This method estimates the co-development of these problems using mixture models and robust maximum likelihood estimation (Nagin, 2005). The result of this analysis allowed the identification of different profiles defined by the joint development of externalizing and internalizing problems across seven timepoints in childhood (from ages 6 to 12). To select the best-fitting model, we estimated a series of models including one to five groups based on participants having at least one data point (n = 3017). These models were compared using the Bayesian information criterion (BIC), the bootstrapped likelihood ratio (BLRT), entropy, theoretical relevance and class size (>10% of the sample retained at smaller class size).
Second, we conducted multinomial regression using R software (version 4.0.5) to examine associations between the identified trajectories and the suicide outcome: non-suicidal, suicidal ideation and suicide attempts. The non-suicidal category was used as the reference group in the main analysis. To test whether there were differences in associations by sex, we included an interaction term between sex and the trajectory group for suicidal ideation and suicide attempts (v. non-suicidal). When a significant interaction was observed, stratified analyses by sex were conducted. Models were then adjusted for child and family characteristics assessed at 6 years old that were associated with the identified problem trajectories compared to low/no problem ( p < 0.10). Further, we tested whether the externalizing and internalizing trajectory groups further distinguish between suicidal ideation and suicide attempts. This was done by using suicidal ideation as the reference group. Missing data in the covariates were handled using multiple imputations, in which the models were estimated across 50 imputed datasets and the results pooled.

Results
The original sample included 3017 participants; main analyses were based on 2143 participants with information on lifetime suicidal ideation and suicide attempts (see online Supplementary Table S1 for comparisons between included and excluded participants; n = 1429 participants were from the representative sample and n = 705 from the disruptive sample); sub-sample membership was controlled in all analyses. Included and excluded participants did not differ in terms of most child and family characteristics except sex (males were more likely to be excluded) and childhood emotional and behavioral problems. Figure 1 depicts multitrajectories of externalizing and internalizing problems according to the four-group model, based on the maximum available sample of 3017. Although the BIC and BLRT pointed towards a five-group model, a closer examination of the four-and five-group models indicated that entropy, theoretical relevance and class size (minimum class size, 10%) favored a four-group model (see online Supplementary Table S6). The four identified group-based multitrajectories were: no/low problems, characterized by low internalizing and low externalizing problems (n = 1369, 45.35%); internalizing, characterized by high internalizing and low externalizing problems (n = 354, 11.74%); externalizing, characterized by low internalizing and moderate externalizing problems (n = 882, 29.23%) and; comorbid, characterized by moderate and increasing internalizing, and high externalizing problems (n = 413, 13.65%).
Child and family characteristics by externalizing and internalizing trajectory group are presented in Table 1. Children presenting with externalizing (64.2%) or comorbid problems (82.3%) were more likely to be male, while children with internalizing problems were more likely to be female (59.3%). All child and family characteristics were associated with either externalizing and/or internalizing problems, except for positive parenting attitudes. We therefore adjusted for maternal education and occupational prestige, maternal age at birth, family structure and negative parenting attitudes in the regression analyses. Notably, children with comorbid problems were exposed to the highest level of psychosocial adversities (e.g. non-intact family: 39.7%; and young mother: 17.7%).
In this sample, 14.8% of individuals endorsed having experienced suicidal ideation but never attempted suicide, while 9.7% said they had attempted suicide by 22 years of age. The prevalence of suicidal ideation and suicide attempts was higher in females (17.5% for ideation and 12.9% for attempts) than in males (12.1% for ideation and 6.3% for attempts) ( Table 2).
Associations between childhood externalizing, internalizing and comorbid problem from 6 to 12 years old and suicidal ideation and suicide attempts by 22 years old For suicide attempts (reference category = non-suicidal), significant associations with childhood externalizing and internalizing problems alone or in co-occurrence were observed in the whole sample (Table 3). In fully adjusted analyses, males and females with externalizing (OR 2.00, CI 1.39-2.88) and internalizing (OR 2.34, CI 1.51-3.64) problems were at higher risk for attempting suicide by early adulthood compared to those with low/no problems. A significant interaction between the comorbid group and child sex ( p = 0.028) suggested that comorbid problems were differentially associated with suicide attempts in females and males. In analyses stratified by sex, the odds ratio for the association between comorbid problems (v. no/low problems) and suicide attempts was 5.40 (CI 2.79-10.46; p ⩽ 0.001) in females and 1.99 (CI 0.97-4.08; p = 0.059) in males. Additionally, females with comorbid problems were at higher risk for attempting   Supplementary Table S7).

Discussion
Our study captured the co-development of externalizing and internalizing problems throughout childhood and investigated their specific associations with lifetime suicidal ideation (without attempts) and suicide attempts by early adulthood. We also examined whether these problems distinguished those who have thought about suicide without attempting it from those who have attempted suicide. In this large cohort, we identified 54.6% of children with elevated emotional and behavioral problems: 29.2% with externalizing, 11.7% with internalizing, and 13.7% with comorbid problems. The odds of attempting suicide in adulthood were two to three times higher for children with externalizing, internalizing or comorbid problems than for those with no/ low problems. Females with comorbid problems had the highest risk of attempting suicide than any other group. Conversely, childhood externalizing, internalizing and comorbid problems were not associated with suicidal ideation (without attempt). Of note, externalizing with or without internalizing problems distinguished those who have attempted suicide from those who have thought about suicide without attempting it. Comparable to our findings, most studies examining the co-development of externalizing and internalizing problems over the course of childhood and adolescence using group-based multitrajectories have identified four groups of children displaying either externalizing and internalizing problems, both, or neither (Brook et al., 2015;Duprey, Oshri, & Liu, 2019;Hannigan et al., 2018;Ip, Jester, Sameroff, & Olson, 2019;Winsper et al., 2019). Notably, using the Avon Longitudinal Study of Parents and Children cohort, the identified group-based multitrajectories of children of 4−9.5 years old in a recent study were similar to Note. Adjusted model controlled for sex, maternal education, occupational prestige, age at birth, family structure and negative parenting attitudes.
those derived by our team in terms of the number, shape and size of trajectories (Winsper et al., 2019). In addition, the prevalence of children with comorbid problems in this cohort (13.7%) fits within the range of 6.3-15.0% that has been identified by prior studies (Brook et al., 2015;Duprey et al., 2019;Hannigan et al., 2018;Ip et al., 2019;Winsper et al., 2019). These children displayed the highest levels of externalizing and moderate-increasing levels of internalizing problems, beginning to incline from 6 years old and continuing steadily up to 12 years old, where they reached the most severe levels of problems in each dimension compared to the other groups. Of note, children presenting with comorbid problems were more likely to be male and to be exposed to the highest levels of adversity (e.g. non-intact family, young mother). This is in line with prior large-sample studies showing that comorbidity in childhood was more prevalent in males and linked to having experienced the highest level of psychosocial problems such as family hardship, and living in a single-parent or low-income household (Duprey et al., 2019;Hannigan et al., 2018;Wiggins et al., 2015;Winsper et al., 2019). As reported in prior systematic reviews and meta-analyses (Soto-Sanz et al., 2019;Witte, Gauthier, Huang, Ribeiro, & Franklin, 2018) childhood externalizing and internalizing problems have been found to be associated with suicide attempts. In their meta-analysis, using suicide attempts by early adulthood as an outcome, Soto-Sanz et al. (2019) reported a moderate effect size for externalizing and large effect for internalizing problems, suggesting that internalizing problems are more strongly associated with suicide attempts. In our study, the odds ratio for externalizing and internalizing problems and suicide attempts were nearly of the same strength (2 and 2.34, respectively). This may, however, be due to methodological differences, as we separated children with comorbid problems from those presenting with only one dimension, while Soto-Sanz et al. (2019) did not. The current study extends existing knowledge by identifying children with comorbid externalizing and internalizing problems while tracking the development of these problems over time.
Of note, we found associations of childhood externalizing and internalizing problems with suicide attempts, but not suicide ideation (in the absence of attempts), suggesting that childhood problems may confer specific risk for later suicide attempts. In the same line, externalizing problems alone or in co-occurrence with internalizing problems were associated with elevated risk of suicide attempts. These results provide support for the ideation to action framework suggesting that specific risk factors could distinguish individuals with suicide ideation (but who do not attempt suicide) from those who engage in suicide attempt. In line with our findings, prior studies found that impulsivity, a feature of externalizing problems, is a key contributing factor to the progression from suicidal ideation to suicide attempt (Mars et al., 2019a).
Using the same cohort from this study, Brezo et al. (2008) found that females with high comorbid problems from 6 to 12 years old were more likely to attempt suicide than those with low problems (Brezo et al., 2008). We took this line of investigation a step further by showing that females with comorbid problems are also at higher risk for suicide attempts when compared to those with externalizing or internalizing problems alone. However, our findings diverge from the work of Sourander and colleagues (Sourander et al., 2009), who found that males with comorbid childhood problems were at higher risk for attempting suicide compared to females. This divergence may be due to differences in methodology between the two studies, namely the fact that externalizing and internalizing problems were measured at one time point and they focused on hospital admission for suicide attempts. Additionally, our findings are in line with a prior study which showed that females with comorbid externalizing and internalizing problems experienced more negative outcomes, such as substance dependence and adverse school experiences, than those without (Marmorstein & Iacono, 2001).
This large population-based cohort study has several strengths. First, externalizing and internalizing problems were assessed by a different teacher every year for seven years throughout elementary school, providing us with insight from multiple raters. Second, identifying group-based multitrajectories of childhood behavioral problems allowed us to estimate the simultaneous development of externalizing and internalizing problems from early to late childhood. Lastly, lifetime suicidal ideation and attempts were assessed using a validated clinical interview administered to both the participants and their parents in adolescence and again in early adulthood, capturing reports from two informants at two key developmental stages. Despite these strengths, certain limitations remain. First, as in other longitudinal studies, attrition occurred over time, with information on suicidal ideation and suicide attempts available for 71% of the initial sample. Though we may have underestimated the strength of associations of our externalizing and internalizing profiles with suicidal ideation and attempts (as participants with emotional and behavioral problems were more likely to be excluded from the analyses), there were no differences on child and family characteristics (other than sex) between included and excluded participants. Second, our assessments of externalizing and internalizing problems were based on observations performed by teachers. Given that, in a school setting, internalizing problems are more difficult to identify than externalizing problems because they are less disruptive to a classroom setting, teachers may have underrated children's internalizing problems (Cytryn & McKnew, 1996;Green, Clopton, & Pope, 1996).
In sum, the present study suggests that childhood externalizing and internalizing problems are associated with the risk of attempting suicide in adulthood; the highest risk was seen for females with comorbid problems. More importantly, externalizing problems (but not internalizing problems) distinguished individuals who attempted suicide from those who experienced suicidal ideation, without attempting it. Thus, the presence of childhood externalizing problems may be a key contributing factor to the risk of engaging in suicidal acts. Given the exploratory nature of our study, more studies are needed to clarify the specific role of childhood externalizing problems as a potential factor for the transition from suicidal ideation to attempt. Intervention studies aimed at reducing childhood externalizing problems might also investigate whether this translates into a reduction of suicide attempts.