The following is a section of the JBRF response to the proposal of the new classification DMDD.  The response was submitted to the American Psychiatric Association in 2010.  At the time of submission, DMDD was still called Temper Dysregulation Disorder (TDD).

Bipolar Disorder: The Categorical Definition:

The categorical definition of bipolar disorder (BD), which is incorporated in the DSM, asserts two fundamental assumptions:

  • Bipolar disorder is predicated upon the presence of discrete, duration-specific episodes of (hypo)mania.
  • Between episodes of (hypo)mania (and depression) there must be symptom-free intervals.

Despite Kraepelinian conceptualization of the illness which included the possibility of cycling uniquely within various grades of depression, (Katzow, 2003) codification of this narrowly defined phenotype stems in part from NIMH collaborative studies on mood disorders conducted in the 1980’s which used DSM-III criteria and excluded any cases with comorbidity.  For the most part, only those subjects who had non-overlapping forms of psychiatric illness were selected as subjects for these studies.  This effectively eliminated subjects who presented with only depression.

TDD: A Categorical Phenotype:

The classification Temper Dysregulation with Dysphoria (TDD), which has been proposed for inclusion to the DSM5, is the successor to the research phenotype, Severe Mood Dysregulation (SMD).[1]  The SMD classification suggests that a boundary exists between children who exhibit severe chronic irritability and those who meet DSM criteria for bipolar disorder.

The research perspective which guided the investigators towards their proposal of SMD adheres closely to a categorical perspective of mental illnesses generally and the categorical definition of bipolar disorder specifically.  While the investigators may explore how far the concept of mania can stretch and still be considered mania, they do so completely within the DSM presumption of the necessity of its presence and the requirement of some degree of episodicity.  Further, they determine this within a context which unquestioningly accepts the boundaries of all Axis 1 diagnoses; this, despite the fact that none of the DSM diagnostic categories have been validated.  The investigators sum this up nicely when they say,

“… to the extent that the question, “Is SMD a developmental presentation of BD?” is framed categorically, the conclusion from this and previous studies is “no.”

Members of the Work Groups, who propose the inclusion of the SMD/TDD phenotype to DSM5, also adhere to, and perpetuate, this categorical perspective.  In their paper entitled Issues Pertinent to a Developmental Approach to Bipolar Disorder in DSM-5, (APA DSM5, 2010b), they state:

From its earliest descriptions and throughout all iterations of the DSM, BD has been described as an episodic illness. Indeed, in both the DSM-IV and in the proposed criteria for DSM-5, clinicians cannot make a diagnosis of BD until they have explicitly confirmed the presence of a hypomanic or manic episode. However, the question of what constitutes an episode has been the subject of some controversy and confusion, especially in the child psychiatry literature… Consistency in diagnostic practices across the developmental spectrum is important because, while there has been debate about the existence of alternative phenotypes for pediatric BD, it is clear that the “classic” adult phenotype does present in children.

And in the paper Justification for Temper Dysregulation with Dysphoria, (APA DSM5, 2010c), they point out:

Unambiguous agreement about [the fact that the “classic” adult phenotype clearly does present in pre-pubertal children] weighed heavily in the Work Group’s deliberations. The agreement on the existence of this classic phenotype generates an important standard against which all other phenotypes can be compared in terms of longitudinal course, family history, and pathophysiology. Thus, deliberations heavily weighed research comparing alternative BD phenotypes against classic BD, in terms of external validators recognized in the DSM-V Task Force Guidelines.  Among these validators, the Childhood Disorders Workgroup followed the Guidelines in considering longitudinal outcome to be particularly important.  This approach also reflected the salience of longitudinal research for drawing conclusions about development, combined with the lack of well-validated biomarkers for BD in any age group, and the absence of data on differential treatment response or genetics in classic as opposed to alternative pediatric BD phenotypes.

Therefore, more than any other consideration, the Work Group noted that a putative, alternative childhood BD phenotype that differed in its symptom patterns from classic BD would be expected to have longitudinal continuity with the classic DSM-IV BD phenotype.

In the absence of any better evidence, the default decision is to turn to consistency with already agreed upon concepts.

Criticism of the Categorical Conceptualization of Mental Disorders Generally and Bipolar Disorder Specifically:

 Dr. Tom Insel, Director of NIMH, and Dr. Bruce Cuthbert, head of the NIMH Research Domain Criteria initiative (Cuthbert, 2010), succinctly point out however that adherence to a categorical conceptualization of mental disorder is increasingly out of step with current knowledge:

…current psychiatric nosologies were developed approximately 30 years ago and [were] necessarily based upon consensus regarding clinical symptom constellations given the lack of knowledge about the structure and functioning of the brain. The classic formulation of Robins and Guze and subsequent refinements were developed primarily with the aim of increasing reliability in diagnosis, with the assumption that future developments such as laboratory tests and family histories would validate the categories.  However, these assumptions have increasingly proved problematic. Co-morbidity among disorders occurs at much higher rates than would be expected by chance, diagnostic categories are increasingly over-specified, and yet “not otherwise specified” diagnoses are very common.

Perhaps even more troubling for modern science, the categories do not reflect the burgeoning knowledge base about genetics, neural circuits, and behavior relevant to mental disorders. 

Recent research tells us that the neurocircuits which express behavior do not stop at the artificial, categorical boundaries we have imposed on our pathological behavior.   They cut across them.  If development of effective and targeted treatments relies upon understanding the physiology which causes that impairing behavior, then it is absolutely essential for us to let go of preconceptions that diminish the probability of uncovering it.  Imposing a dichotomous and fragmented notion on an often continuous and overlapping reality may provide reliability and manageability, but it will not provide a means to discovery.

In regards to bipolar disorder specifically, the categorical conceptualization of the condition faces significant criticism by many investigative groups who find no evidence for valid cut-off points on the continuous distribution of the symptoms of unipolar depression, bipolar depression, hypomania and even mania. (Angst, 2002 Oct, 2007; Benazzi, 2003 Oct; Cassano et al., 2004; Judd et al., 2002, 2003; Leverich et al., 2001; Phelps et al., 2008)  Cassano et al. (2004) find:

Cumulatively, our empirical findings support a continuous view of the mood spectrum as a unitary phenomenon that is best understood from a longitudinal perspective. Our data suggest that unipolar disorder and bipolar disorder are not two discrete and dichotomous phenomena but that mood fluctuations—up and down—are common to both conditions, as emphasized by Koukopoulos et al. (22). Indeed Swann (23), espousing the parsimonious Kraepelinian view, conceptualized recurrent affective disorders as one illness in which the apparently different bipolar and unipolar course may be determined by increased sensitization after the occurrence of a manic episode.

Using histograms, Franco Benazzi (2003 Oct) found an even distribution of hypomanic symptoms during depression for both subjects with unipolar depression and those with bipolar II disorder.  Angst et al. (2007) say that patients with BD I and II spend about half the time in sub-threshold affective conditions and brief spells of hypomania are far more common than manifestations lasting 4 days or more.  All this would suggest that the significantly greater prevalence of depressive disorder over bipolar disorder may largely be a product of misunderstanding or misdiagnosis.  Angst et al. (2002) contend that the,

Zurich cohort study suggests that with a hard definition of hypomania one fourth of MDE are BD-II and with a soft definitions as many as half of all MDE cases are undetected bipolars.

The list of prominent researchers who underscore the importance of subsyndromal, or non-classic, symptoms as critical to an understanding of the illness, reads like a who’s who in bipolar research.[2] Current thinking has led back to Emil Kraepelin’s unified conceptualization of manic-depressive insanity.  Dr. S Nassir Ghaemi (2008) reminds us:

For Kraepelin, most mood episodes were indeed mixed manic-depressive states involving dimensions of thought and behaviors as well as mood; pure depression and pure mania were less frequent.  This is why he focused on recurrence, rather than polarity (presence of mania, as in SDM-III onwards), as the hallmark of manic depressive illness

and then Ghaemi summarizes:

what is the point to identifying poles of illness if most cases are a mixture?

Simultaneous to the increasing awareness of the importance of the full spectrum of subsyndromal symptoms has been the erosion of the importance of episodic duration.  Threshold durations provided for in DSM III and IV are the product of consensus rather than empirical evidence.  However, since shortly after DSM-IV-TR came out, episodic thresholds have come under scrutiny.  Kramlinger and Post (2006) found:

Affective oscillations spanned a range of cycling frequencies from four episodes per year (rapid cycling) to those occurring within the course of weeks to several days (ultra-rapid cycling), to distinct, abrupt mood shifts of less than 24 hours duration (ultra-ultra rapid or ultradian cycling).

Current research does not indicate a need to move the bar a little to the left or right, but perhaps to eliminate it entirely.  The re-consideration of bipolar disorder from an episodic illness to one of chronicity is supported by the preponderance of time patients spend in combined syndromal and subsyndromal symptoms.  Judd et al. (2002, 2003) found that adults with BD I spent 47% of follow-up time symptomatic[3] and adults with BD II spent 54% of follow-up time symptomatic[4].  In regards to chronicity, they say:

Using the total percentage of weeks symptomatic during the entire course as a measure of chronicity depicted a chronic picture of BD-II. This BD-II cohort, with a total of 54% of follow-up weeks symptomatic, falls between those with BD-I and those with unipolar MDD (47% and 60%, respectively), which suggests a continuum of chronicity among the affective disorders. Previous studies of course chronicity of affective disorder have generally focused on recurrence, number, severity, and characteristics of syndromal episodes, but, as shown in this and a previous study, the exclusive focus on syndromal episodes, although essential, does not delineate the full picture of bipolar course and chronicity. (italics ours)

Similarly, in the 4-year longitudinal analysis of the COBY study, Birmaher et al., (2009) determined that children were symptomatic 60% of the time[5].  They defined chronicity as measured by the percentage of participants who had syndromal and/or subsyndromal mood symptoms ≥75% of the time and found 38% of the participants met that threshold (particularly those with BDI, while only 3% met full syndromal criteria for mood episodes.[6]

The case for polarity, with its emphasis on mania, and episode duration as differentiating criteria for mood disorder is waning.  The dimensional wholeness of the condition is seen with increasing importance.

Applying New Ideas to the Issue of Bipolar Disorder in Youth:

Jumping to the presentation of the illness in youth, much of the controversy has stemmed from the contention that the pediatric presentation is so different than the criteria articulated in DSM BD I and II.  The predominance of irritability, and particularly the scarcity of clear cut (hypo)manic episodes of minimal duration, have caused many to question whether it is the same illness.  Yet, ironically, that “disconnect” may come from the fact that the illness itself may look so different than what is laid out in the criteria for DSM BD I and II.  If one departs from the categorical constraints of the classic presentation, which merely identifies points on a continuum, and adopts the growing consensus for a unified, spectrum illness, then the pediatric presentation falls very understandably within both the characterization and prevalence rates of the illness.  This view provides an important context for Geller et al’s. (2001) finding that on10 year follow-up, 33.3%of youth (n=24) with a prepubertal diagnosis of major depressive disorder converted to bipolar I disorder and 48.6% (n=35) converted to any type of BD compared to normal controls, none of whom converted to BDI and 7.1% (n=2) who converted to any type of BD.

While the 40-fold increase of youth office visits with a diagnosis of BD (Moreno et al., 2007) sounds alarming, it represents 1% prevalence[7].  Indeed, this is exactly what Lewinsohn et al. (1995) determined for lifetime prevalence rate of BD in youth and it is also the approximate lifetime prevalence rates for BD I in adults (Angst, 1998).  When Lewinsohn et al. expanded their study to include subthreshold symptoms, the rate of youth with BD increased by 5.7%.  Again, this is consistent with adult prevalence of BD spectrum disorder which is placed at between 3 and 6.5% (Angst, 1998).  It seems that comparing apples to apples matters.

In support of the continuity between pediatric and adult presentation, it is worth quoting at length a passage from Danner et al. (2009):

Episode duration is one perceived difference in symptom presentation between children and adults.  Researchers have noted that many children experience much shorter periods of manic symptoms lasting from a few hours to a few days, which do not fit into DSM-IV-TR diagnosis requiring periods of at least 4 days for hypomania or 1 week (unless severe enough to warrant hospitalization) for manic or mixed episodes (Geller et al. 2007). Until recently, differences in the symptom duration of child and adult bipolar disorder were thought to provide evidence that they were not the same disorder. Recently outlined spectrum models of adult bipolar disorder, however, are more consistent with the symptom presentations noted in youth (Phelps et al. 2008). Similarly, shorter duration of index mood episodes is consistent with the emerging understanding of the phenomenology of bipolar disorder in adults, who often experience manic periods of less than 1-week duration or hypomanic periods of less than 4 days early in the disorder (Angst et al. 2003; Kramlinger and Post 1996; Suppes et al. 2001). In addition, similarities in impairment of adults with rapid cycling bipolar disorder and children and adolescents with BDSD have lent credibility to the diagnosis in youth (Perlis et al. 2004). Researchers have found more similarities between child and adult bipolar disorder than previously believed (Jerrell and Shugart 2004).

The DSM-IV offers a rapid cycling specifier, defined as at least four mood episodes per year, lasting at least 1 week each, which characterizes approximately 15–54% of adults diagnosed with BDSD (Tillman and Geller 2003). For adults, rapid cycling is correlated with earlier onset of BDSD, more functional impairment, and more time spent in the depressive portion of the cycle (Perlis et al. 2004; Schneck et al. 2004; Suppes et al. 2001). Some researchers have noted that the rate of cycling is related to pharmacological response (Bauer et al. 2008; Goodwin and Jamison 2007; Masi et al. 2006). Schneck et al. (2004) found that participants with rapid cycling, 20% of their sample of adults with BDSD, reported an average of 8 to 9 times as many episodes per year as those without rapid cycling.  Rapid cyclers had a striking range of episodes per year, with some adult individuals showing polarity changes (switching from depressed to manic or euthymic or vice versa) every few weeks and others showing polarity changes multiple times per day (Schneck et al. 2004).

Approximately 10–20% of adults and many youth with BDSD experience multiple mood switches per day, which some researchers have called ‘‘cycles’’ (Tillman and Geller 2003), but could be referred to as polarity changes (Schneck et al. 2004) or mood instability (Youngstrom et al. 2008a).  Adults with mood instability are likely to be diagnosed with a mixed episode; however, mood instability in children and adolescents has led some clinicians to question the validity of the diagnosis in youth. The finding that many adults also experience this instability of manic and depressive symptoms lends credibility to the diagnosis in youth who do not experience week-long episodes of mania (Bauer et al. 2008; Angst et al. 2003). Further credibility comes from a large cohort of adults with bipolar disorder, where approximately 20% reported their first episode had been mixed (Kogan et al. 2004).  This is similar to other recent findings (Perlis et al. 2004), and also is consistent with historical data, where the most common presenting mood episode in adolescents and adults with bipolar disorder was a mixed episode (Kraepelin 1987).   In summary, research on adult BDSD shows more consistency with youth than previously thought, especially in regard to rate of mood shifts. Similarly, adults with mixed episodes or rapid cycling tend to have more impairment than adults with fewer cycles (Phelps et al. 2008).

It would seem that there is little doubt at this point that the real picture is one which is difficult to capture categorically in either children or adults.  Given the most current information about how our brains and genes work, and how mood disorders present themselves, one would be hard pressed to accept the standard of categorical criteria as the foundation for a new diagnosis.

But this is exactly what gave rise to the SMD phenotype and its proposed inclusion to DSM5.  It is the categorical, discreet-entity view of bipolar disorder from which the investigators launch their inquiry and to which they differentiate their findings.  They do this despite the fact that for the last thirty years, no evidence-based research has validated its criteria.  We can’t help but to point out that the use of unvalidated presumptions as both beginning and end points can only mislead new research and create a second generation of artificial boundaries.  We consider the proposal for a new diagnosis that separates the SMD behavioral syndrome from a bipolar spectrum to be merely a lateral move within an unvalidated nomenclature rather than a sign of real progress.  While this may cause frustration to us as an organization which supports research, it causes great distress for us as an organization which would like to advance the likelihood of treatment breakthroughs.

The TDD Phenotype:

Since all studies were conducted with criteria for the SMD phenotype, we will refer to SMD rather than TDD in our discussion.  While we believe that the Work Groups’ last minute, consensus-based elimination of hyperarousal symptoms from the SMD phenotype deserves examination, we have not had the time to explore the effect of that decision on the final proposal.  But we would like to put on record that we find this decision disturbing.  Chronic hyperarousal was one of the two “core” criteria for inclusion in the original conceptualization of SMD (Leibenluft et al., 2003) and its inclusion has been reiterated in every study since.

The phenomenological examination of SMD addresses two main questions:  what is the likely outcome for children with SMD and, to the extent to which SMD is a developmental presentation of BD, do youth with SMD manifest manic or hypomanic episodes meeting DSM-IV criteria.   Longitudinal studies explore:

  • the outcome of children identified with SMD (Brotman et al., 2006),
  • the predictability of chronic irritability (Leibenluft et al., 2006, Stringaris et al., 2009b, 2009 April), and,
  • the conversion of children with SMD to BD by virtue of the onset of (hypo)manic symptoms which fulfill DSM-IV criteria (Stringaris et al., in review).

Conclusions reached are that SMD predicts depression and generalized anxiety disorder rather than BD and that children with SMD do not experience (hypo)manic episodes.

We have many concerns regarding the studies that led to these conclusions.  The methodological issues of some of the studies are addressed elsewhere in this submission.  While we will address below our primary conceptual concerns, we would also like to point out that we find inconsistency across the studies, particularly in regard to the operating definition of irritability[8].  We believe that this undermines the strength of their accumulative conclusions.

Categorical Concerns:

 Not surprisingly, our primary conceptual concern derives from the use of categorical cut-offs and adherence throughout the studies.  While the investigators may indicate that the categorical conclusions indicate a need to separate SMD from BD, we believe that when the categorical guillotine is removed, their findings firmly place the SMD syndrome into a bipolar spectrum.

A salient example of the effect of categorical imposition is seen in the clinic-based NIMH-recruited study (referred to hereafter as the NIMH study) which looks to see if children with SMD convert to BD by the onset of (hypo)manic symptoms and compares that to (hypo)manic episodes experienced by children with BD I (73%) or II during the same time period. (n=84 SMD, n=93 BD).  They find virtually no conversion from SMD to BD and that robust numbers of BD I and II children do experience such episodes.  Having established this contrast, the investigators go on to compare their finding of no cross-over to the 38% conversion rate of short-episode BP-NOS to full-duration hypomanic or manic episode as demonstrated in the COBY study (Birmaher et al., 2009) From this, they draw the conclusion:

…our data, in concert with those from the COBY study, suggest that, whereas the “short-episode” phenotype may be a manifestation of BD and thus may warrant the diagnosis of BP-NOS, such a conclusion is considerably less clearly justified in the case of the non-episodic presentation that is characteristic of SMD.  (Stringaris et al., in review)

The disparity of conversion between children with SMD and those with BP-NOS is found to be differentiating evidence.  But when one recognizes that this is based upon fulfillment of (hypo)manic episodic criteria and that the follow-up period of the COBY study was twice as long as the NIMH study, that disparity becomes less meaningful.

Examples of categorically derived conclusions abound through all the studies and are too numerous to detail here.  Categorical definitions are used not just to draw conclusions, as above, but to determine inclusion, exclusion, diagnosis, co-morbidity and time periods considered sufficient for observation.  Given 1) the discussion above regarding the continuum of depression through mania, 2) the well recognized fact that throughout the bipolar (and depressive) spectrum, the amount of time spent in depressive symptoms far outweighs the amount of time spent in manic symptoms, 3) the likelihood of recurrent depression eventually expanding to include manic symptoms (Angst, 1998), and 4) the prevalence of mixed or depressive episodes as the presentation early in the illness (Danner et al., 2009), the discovery that children with SMD evolve into young adults with depression and generalized anxiety disorder, or that within three years they do not experience DSM-IV defined criteria of (hypo)manic episodes, loses much of its value.

Beyond our contention that the finding of depression as the likely outcome of SMD keeps it within a unified manic/depressive spectrum, Clark and Watson (2006) provide support that the anxiety outcome does so as well.  They find reason to view anxiety and depression on a spectrum with each other.  They report:

Results of genetic studies paralleled the US survey data in that major depressive disorder and generalized anxiety disorder were found to share a single genetic diathesis, …(e.g. Kendler, 1996)…Moreover, a review of the voluminous comorbidity literature by Mineka et al (1998) revealed that, although either type of disorder conveyed an increased risk for later development of the other, anxiety disorders were significantly more likely to appear first, and cases of pure depression were more rare than pure anxiety, raising the possibility that anxiety disorders represented a less severe form of a single spectrum…It now is abundantly clear that these two types of disorders are strongly related and should not be artificially separated into different diagnostic classes…To the extent that the DSM and ICD purport to be empirical documents, the current folk taxonomy must be abandoned and replaced with a data-driven, scientifically supported taxonomy.

Therefore, we would suggest that in the ongoing research that sees unification between disorders previously considered as distinct and separate from each other, the spectrum which goes through anxiety to mania would well encompass children with the SMD syndrome.

The Dimensional Issue of Irritability

The line of research which articulates the type of irritability found in SMD places that irritability squarely within one of the primary behavioral dimensions of the FOH phenotype as delineated from research supported by JBRF.  While SMD investigators may assert the final conclusions in a categorical manner, the underlying reasoning is clearly dimensional and based on domains of behavior.  We think of it as another example of how behavioral domains demand their reconstitution despite categorical efforts to cleave them into separate categories.  In order to fully understand this point, it is necessary to introduce Clark and Watson’s (2006) view of internalizing and externalizing disorders and Stringaris and Goodman’s (2009, 2009 April) articulation of domains of oppositionality in oppositional defiant disorder (ODD).

Clark and Watson (2006) make a case for restructuring the nosology regarding anxiety and mood disorders to reflect internalizing and externalizing disorders.  They say:

An important related question is how the genetic and structural findings for anxiety and depression fit into the broader domain of psychopathology. The answer to this question has emerged over the past decade. During this period, six large-sample independent studies (Lahey et al, 2004; see Clark, 2005 for the five others) have examined the structure of psychopathology by studying diagnostic comorbidity patterns phenotypically and/or genotypically, each using a set of common mental disorders that largely overlapped across studies.  The results have revealed a remarkably consistent structure: a hierarchical model with two broad factors – externalizing and internalizing. Substance dependence, attention-deficit hyperactivity disorder, oppositional defiant disorder, and conduct disorder/antisocial personality disorder define the externalizing factor. The internalizing factor subsumes two highly related subfactors: ‘distress/misery’ – comprising generalized anxiety disorder,  overanxious disorder and depressive disorders– and ‘fear’, which includes simple and social phobias, separation anxiety  disorder and panic disorder…it is noteworthy that this alternative hierarchical scheme consistently captures the comorbidity data better than the DSM model, which separates these syndromes into ‘mood’ and ‘anxiety’ disorders.

It is this concept of ‘distress’ disorders to which Stringaris and Goodman associate the particular dimension of oppositionality which was ultimately adopted as the characterization of irritability in the SMD syndrome.  In a process very similar to the dimensional studies of Mataix-Cols et al. (2005, Feb.), in which the heterogeneous disorder of obsessive-compulsive disorder (OCD) was broken down into behavioral domains which were then able to elucidate, through imaging studies, the different brain areas involved in each of the different behavioral dimensions, Stringaris and Goodman also broke down the heterogeneous disorder of ODD into behavioral dimensions.  They conceived of three dimensions called Irritable, Headstrong and Hurtful.  The Irritable dimension is conceived as:

…the combination of temper outbursts, anger, and a low threshold for being annoyed—as representing negative emotional reactivity and emotion dysregulation. Indeed, it has been shown that children with severe mood dysregulation, a syndrome where negative emotional reactivity is a prominent feature, are more likely to develop depression than a disruptive disorder at follow-up.  In accordance with these findings, we expect that the prediction from the Irritable dimension will be particularly pronounced for depression and generalized anxiety.  These comprise the so-called distress disorders… (Stringaris and Goodman, 2009 April)

Indeed the longitudinal study of this dimension concluded that:

…only the Irritable dimension of oppositionality at baseline was a significant predictor of internalizing disorders at follow-up, even after adjusting for age, sex, and either an  internalizing disorder or the SDQ emotional score at baseline. (Stringaris and Goodman 2009, April)

The other two dimensions are called Headstrong and Hurtful.  Headstrong incorporates behaviors of argumentativeness and Hurtful encompasses callous, premeditated and aggressive acts of offending.  They were found to predict ADHD and conduct order (respectively).  (Stringaris and Goodman 2009, April)  In essence, Stringaris and Goodman have isolated the mood behaviors from a disruptive disorder base and made the case that the type of irritability they represent is associated with the internalizing disorder of the emotional mood spectrum.  Like Mataix-Cols et al., they believe that this dimensional differentiation will lead to important clues of “etiology, pathophysiological mechanisms, prognosis, and treatment choice” (Stringaris and Goodman, 2009 April).  In other words, dimensional analysis of behavioral domains will lead to a better understanding of the impaired behavior.

The original characterization of irritability in SMD was associated with externalizing behavior.  This can be seen in their examination of the Great Smokey Mountain Study in which a screening instrument derived from the “externalizing” scale of the Child Behavior Checklist was used to identify children with SMD.  However, by the second study of the Children in the Community study (Stringaris et al., 2009b) and in the NIMH study, (Stringaris et al., in review) the investigative group clearly refined their characterization of irritability into one associated with internalizing behavior.  In the NIMH study, they clarify:

It is important to note that SMD was created for the purpose of studying children presenting with severe nonepidsodic irritability. SMD overlaps with ODD in that both ascertain temper outbursts, irritability, and anger. Crucially, however, SMD does not include the symptoms of defiance and annoyance that are also included in ODD—this distinction between headstrong and irritable dimensions in ODD has been shown to have important differential predictions.  (Stringaris, in review)

Of note, the aggressive behavior that is associated with the Irritable dimension is attributed to an impulsive, reactive, response to anger and fear rather than an offensive response to anger alone. (Stringaris et al., 2009, 2009b)  This is consistent with the type of aggressive behavior found in the proposed, dimensional FOH phenotype (see below) in which a primary behavioral dimension is defensively retaliatory aggression to perceived threat.  (Papolos et al., 2009)

Articulation of the Irritable dimension of oppositionality by Stringaris and Goodman and the incorporation of that dimension into the SMD syndrome has effectively differentiated SMD from the disruptive components of ODD and specifically re-associated it within a mood disorder continuum.  In our view, this, combined with the removal of the categorical cut-offs make a strong case that children with SMD really are not distinct from children with BD.  To repeat the investigators own words, they are only distinct when viewed categorically.

This is likely to explain why the children with SMD are so severely impaired.  It is well established in the literature that individuals throughout the mood disorder spectrum suffer significant impairment, not just those meeting threshold criteria.  As Birmaher et al. (2009) state:

Bipolar disorder not otherwise specified is characterized by rates of comorbidity, suicidality, functional impairment (except hospitalization), and family history of mood disorders equivalent to those in youths with bipolar I and II.

Part of the justification for placing the TDD/SMD phenotype in the mood disorders section rather than as a specifier to ODD was due to the fact that ODD does not communicate the significant impairment of these youth nor the severe mood disorder from which they suffer. (APA, DSM5b)  We would say the reason that these youth are more commensurate with the degree of impairment and mood dysregulation of BD children than typically found in children with ODD is because that is the proper way to think about them.  Instead, the investigators for the SMD phenotype make a categorically dependent argument that relies upon weak and inconsistent data to prove otherwise and the Work Groups disregard the dimensional nuance of oppositionality which separates these children from the disruptive disorders and simply uses a general, categorical definition of ODD to justify separation from BD instead.

Of final note, the SMD investigators point out that anger, a hallmark symptom of irritability can factor with fear towards the internalizing disorders and separate from fear to associate with the externalizing disorders.  Thus irritability could be at the “interface” between mood and behavioral disorders.  Further, they suggest the possibility that this interface may be stronger in adolescence than in adulthood (Stringaris et al., 2009b).  We assume that this is the thinking that has led the investigators to suggest that SMD, unlike BD, does not come with a life-long prognosis.  While the idea of irritable behavior providing the bridge between mood and behavioral disorders is an interesting one, we do not feel that it weakens our argument that SMD, as defined without offensive, disruptive behaviors of oppositionality, places children with SMD within the same spectrum occupied by children with BD.


Leibenluft et al. (2003) state that:

…ultimately the diagnosis of (hypo)mania, as of other psychiatric illnesses, will rest on an understanding of pathophysiology.

On this point, we could not agree more.  The failure of categorical diagnoses to lead to significant pathophysiological insights is in part what has motivated the move towards dimensional analysis.  While the investigators who propose the SMD classification have identified what they feel to be a pathophysiological differentiation between children with SMD and those with BD, these results pale in comparison to the pathophysiological information that has been elucidated as a result of dimensional exploration of the illness conducted by JBRF-supported researchers.

Pathophysiology Revealed by Categorical Analysis:

Efforts to support the SMD/TDD phenotype with pathophysiological evidence have explored two main avenues:  response to frustration and interpretation of emotion.  In the first instance, investigators claim to have identified an important differentiation between the two populations (Rich et al., 2007).  We have considerable criticism of the methodology used for this study and those concerns are addressed in the methodology section of this submission.

As for the comparative study on emotional labeling, Baroni et al. (2009) find that:

…there does appear to be some shared pathophysiology between SMD and BD, in that both patient groups exhibit deficits in their ability to label face emotions. This deficit is generalized across emotions (Rich et al., in press) and is not seen in youth with ADHD or those with MDD and anxiety disorders (Guyer et al., 2007).

However they minimize this overlap as reasonable and negligible:

It is possible that there are some shared genetic mechanisms between SMD and BD, along with many important genetic differences.  Nonetheless, given that the longitudinal, family history, and pathophysiological data described above identify important differences between the two groups, we do not feel that it is currently justified to diagnose youth with non-episodic irritability and ADHD symptoms as having BD.  (italics ours, Baroni et al., 2006)

As substantiated at great length above, we have little confidence in the longitudinal studies which provide the outcome validators to the phenotype.  While we have not addressed the familial study which purport to show that children with SMD do not come from families with BD (Brotman et al., 2007), we dismiss it due to the same conceptual weakness as the longitudinal studies; it is supported by the complete adherence to a categorical nomenclature.  Therefore, we must subtract these from the weight of evidence that highlights “important differences”.  What we are left with then are the mixed pathophysiological findings.  We are comfortable with the fact that the investigators identify a pathophysiological difference between these populations.  Clearly, there are observable and therefore likely genetic differences between these groups of children.  However, a circumscribed categorical exploration which only focuses on differentiation related to manic duration rather than on the whole syndrome makes any differential take on inflated importance.

Pathophysiology Revealed by Dimensional Analysis:

While the research conducted with the support of JBRF has not specifically separated children with chronic irritability and episodic mania, we believe that the strength of both the research and the results, including a hypothesis of the neurocircuitry and pathophysiology involved in the disorder, as well as the probable identification of a biological marker, strongly support the efficacy of a dimensional perspective.

JBRF-supported investigators did not begin their inquiry on a foundation of codified assumptions; they let the illness tell them what it is.  The dimensional research that led to the Fear-of-Harm (FOH) phenotype (Papolos et al., 2006, 2007, 2009) did not launch from, or measure against, any preconceived boundaries or notions of symptom importance.  It merely looked at the whole symptom profile of a very large number of children (n=2,795) diagnosed with, or at risk for, bipolar disorder.  From this approach a novel picture emerged that would put many of the children who will be cleaved off under a TDD diagnosis, back into a bipolar spectrum.  Approximately 2/3 of the children in this study were found to have the FOH trait.  A full 1/3 of the full group was found to have the trait in its pure form.

It is safe to say that children identified with the FOH phenotype closely resemble a Kraepelinian characterization of bipolar disorder.  95% of the children who fit the FOH phenotype have psychosis and periods of grandiosity and elated mood states.  They have severe levels of depression and mania and high rates of hospitalization.  However, in regard to mood duration there is no clear pattern; some children fluctuate abruptly and rapidly while others fluctuate episodically.

The FOH profile has elucidated a comprehensive neuroanatomical model of the pathophysiology of this phenotype.  While both mania and depression figure prominently in the behavioral profile of the FOH phenotype, the investigators have learned that they are not the most relevant behavioral domains to the pathophysiology of the disorder.  

Multiple regression analyses of large sets of data gathered by the Child Bipolar Questionnaire[9] revealed behavioral dimensions which were then examined in a concordance analysis of affected sib-pairs (Papolos et al., 2007).  This pointed the investigators to the importance of the following behavioral dimensions: Fear of Harm, Aggression, Anxiety, Sensory sensitivity, Sleep/wake cycle disturbances and Attention/executive function deficits.  Mania and depression, while present, demonstrated less heritable relevance than most other dimensions.

Further inquiry and factor analysis focused upon the trait Fear of Harm (FOH) which had demonstrated significantly higher concordance than all other traits.  This study revealed another set of behavioral dimensions which, in turn, led the investigators to the insights that have elucidated the pathophysiology (Papolos et al., 2009).  In particular, symptoms clustered within the factors of Territorial aggression, Harm to self/others, Psychosis/Parasomnias/Obsessions/ Sweet Cravings (PPSO) and Sleep/arousal led the investigators to suspect the orexigenic-neuropeptide circuit.  Ongoing research into this highly complex, ancient pathway, which is intimately involved in both physiological and behavioral survival, has allowed investigators to put together a parsimonious neuroanatomical hypothesis that accounts for the entire complex of symptoms of the FOH phenotype.  From what they have learned, thermoregulatory disturbances, sleep disruption including arousal disorders of sleep (night-terrors, nightmares, enuresis, teeth grinding), contamination fears, carbohydrate craving, and food hoarding are central and inseparable parts of the most severe form of the illness. (The most complete description of this hypothesis and how it accounts for the FOH behavioral phenotype can be found in the separate packet of information submitted directly to the Task Force Co-Chair, Dr. Darrel Regier.)

In our opinion, the categorical analysis that led to the SMD phenotype unintentionally cleaves off a fragment of a unified condition which is defined more accurately by the FOH phenotype.  SMD criteria can, in essence, be considered the symptom-clusters that describe just two of the behavioral dimensions of the FOH phenotype: Territorial Aggression and Fear of Harm.  Symptoms in the other dimensions of the FOH phenotype were not looked at by the SMD investigators and we suggest that if they were, it is likely that many subjects who meet SMD criteria would also meet FOH criteria thereby greatly expanding the degree of genetic and pathophysiological overlap.

Overlapping Syndromes:

In their article, Distinguishing Between the Validity and Utility of Psychiatric Diagnoses, Dr. Robert Kendell and Dr. Assen Jablensky (2003) point out:

The existence of several rival definitions of a syndrome, embracing overlapping populations of patients should raise the suspicion that it is not a valid category because these rival definitions suggest that variations in symptoms is continuous and that no identified zone of rarity indicating the boundaries of the syndrome can be drawn.

In 2005, JBRF-supported investigators conducted a study in which data from 2,795 subjects was examined to determine the rates of Broad, Narrow and Core phenotypes in this cohort   (Broad is the precursor to SMD/TDD and Core is the precursor to FOH).  Almost 92% of children who satisfied Narrow criteria also satisfied Core criteria. (n=1306 of 1420)  Almost 81% of children who met the criteria for the Broad phenotype also met criteria for the Core phenotype (n=187 of 231). Taken together, these findings suggest that both the Narrow and, importantly, the Broad phenotype do not represent completely separate populations from the Core phenotype.

In our view, features which might typically be considered symptoms of comorbid disorders are pathognomonic of juvenile-onset bipolar disorder.  These symptoms have simply not been considered as relevant in other studies, including the one which proposes the SMD classification.  As Dr. Jules Angst (2007) points out:

Too many studies, especially in epidemiology, have used methods tailored and restricted to the current DSM-IV diagnostic concepts and have not collected additional data which would have allowed those concepts to be questioned…

A particularly clear example of this regards the symptom of enuresis; a symptom not associated with bipolar disorder according to a DSM perspective, and not included in any of the inquiries which delineated the SMD/TDD phenotype.  However, in the dimensionally derived FOH phenotype, enuresis was found to be a symptom of one of the primary factors of the phenotype.  The PPSO factor, which was so instrumental in the elucidation of the proposed pathophysiology of the phenotype, includes several arousal disorders of sleep, including enuresis as primary symptoms.  It is noteworthy that in the original Great Smoky Mountain Study, the longitudinal epidemiology study from which SMD subjects were identified and outcome conclusions reached (Brotman et al., 2006), enuresis was identified as the second most prevalent diagnosis yet very few cases of bipolar disorder were identified. (Costello et al., 1996)

Diminishing the Importance of Mania:

By taking a comprehensive, dimensional perspective of the illness rather than sticking to categorical, unvalidated criteria, JBRF investigators brought into their view a constellation of symptoms which became instrumental in arriving at a biological understanding of the illness.   The fact that this model can explain mania and all the other symptoms involved in the profile would seem to put an end to the insistence that bipolar disorder is all about the mania; clearly it is not.  We think it is time that mania, as imposing and observable as it is, is put into a more enlightened context in which it is regarded as a component in a full range of overlapping criteria.

At this point we might suggest that the argument made by Baroni et al., (2006), that minor genetic similarities can be ignored in light of important differences, be turned around to arrive at the exact opposite conclusion, that minor genetic differences (such as those that predispose one to chronic v. episodic mania) may be ignored in light of many important similarities.  Certainly an analysis that examines specific clinical and physiological characteristics of children with TDD in comparison to those with FOH would be illuminating in this regard.  Given the weight of evidence supportive of a dimensional analysis and critical of the proposed TDD phenotype, we can only imagine that an earnest effort towards an empirically validated nomenclature will recognize the importance of such an endeavor.



American Psychiatric Association DSM5 Development (2010a) retrieved April 11, 2010, from:

American Psychiatric Association DSM5 Development (2010b) retrieved April 11, 2010, from:


American Psychiatric Association DSM5 Development (2010c) retrieved April 11, 2010, from:


Angst J: (1998). The emerging epidemiology of hypomania and bipolar II disorder. J of Affective Disorders, 50:143-151

Angst J, Gamma A. Lewinsohn P: (2002, Oct.). The evolving epidemiology of bipolar disorder. World Psychiatry, 1:3

Angst J: (2007). The bipolar spectrum. Brit J Psychiatry, 190;189-191

Baroni A, Lunsford JR, Luckenbaugh DA, Towbin KE, Leibenluft E: (2009). Practitioner Review: The assessment of bipolar disorder in children and adolescents. J of Child Psychology and Psychiatry, 50:3, pp 203-215

Benazzi F: (2003, Oct). Bipolar II Disorder and Major Depressive Disorder: Continuity or discontinuity.  World J Biol Psychiatry, 4(4):166-71.

Birmaher B, Axelson D, Goldstein B, Strober M, Gill MK, Hunt J, et al.: (2009) Four-Year Longitudinal Course of Children and Adolescents With Bipolar Spectrum Disorders: The Course and Outcome of Bipolar Youth (COBY) Study. Am J Psychiatry, 166:795-804

Birmaher B, Axelson D, Strober M, et al. (2006). Clinical course of children and adolescents with bipolar spectrum disorders.  Arch Gen Psychiatry, 63:175-183

Brotman MA, Schmajuk M, Rich BA, Dickstein DP, Guyer AE, Costello EJ, Egger HL, Angold A, Pine DS, Leibenluft E: (2006). Prevalence, Clinical Correlates, and Longitudinal Course of Severe Mood Dysregulation in Children. BIOL PSYCHIATRY, 60:991-997

Brotman MA, Kassem L, Reising MM, Guyer AE, Dickstein DD, Rich BA, Towbin KE, Pine DS, McMahon FJ, Leibenluft E: (2007). Parental Diagnosis in Youth With Narrow Phenotype Bipolar Disorder or Severe Mood Dysregulation.  Am J Psychiatry, 164:1238-1241

Cassano G, Rucci P, Frank E, Fagiolini A, Dell’Osso L, Shear MK, Kupfer D: (2004). The Mood Spectrum in Unipolar and Bipolar Disorder: Arguments for a Unitary Approach. Am J Psychiatry, 161:1264-1269

Clark LA, Watson D: (2006).  Distress and fear disorders: an alternative empirically based taxonomy of the ‘mood’ and ‘anxiety’ disorders.  Brit J of Psychiatry, 189:481-483

Costello E, Angold A, Burns BJ, Stangl DK, Tweed DL, Erkanli A, Worthman CM: (1996, Dec.). The Great Smoky Mountains Study of Youth: Goals, Design, Methods, and the Prevalence of DSM-III-R Disorders. Arch of Gen Psych, 53(12) pp 1129-113

Cuthbert B, Insel T: (2010, Feb). Classification issues in women’s mental health: clinical utility and etiological mechanisms. Arch Womens Ment Health 13(1):57-9

Danner S, Fristad M, Arnold LE, Youngstrom E, Birmaher B, Horwitz S, Demeter C, Findling R, Kowatch R, the LAMS Group; (2009). Early-Onset Bipolar Spectrum Disorders: Diagnostic Issues.  Clin Child Ram Psychol Rev, 12:271-293

Geller B, Zimerman B, Williams M, Bolhofner K, Craney J: (2001). Bipolar Disorder at Prospective Follow-up of Adults Who Had Prepubertal Major Depressive Disorder.  Am J Psychiatry, 158:125-127

Ghaemi SN: (2008). All mixed up: On the absence of diagnostic guidelines for mixed states in the ISBD Diagnostic Guidelines Task Force Report. Bipolar Disorders, 10:129-130

Judd LL, Akiskal HS, Schettler P, Endicott J, Maser J, Solomon D, Leon A, Rice J, Keller MB: (2002). The Long-term Natural Hisotry of the Weekly Symptomatic Status of Bipolar I Disorder.  Arch Gen Psychiatry, 59:530-537

Judd LL, Akiskal HS, Schettler P, Coryell W, Endicott J, Maser J, Solomon D, Leon A, Rice J, Keller MB: (2003).  A Prospective Investigation of the Natural History of the Long-term Weekly Symptomatic Status of Bipolar II Disorder.  Arch Gen Psychiatry, 60:261-269

Katzow J, Hsu J, Ghaemi SN: (2003). The bipolar spectrum: a clinical perspective.  Bipolar Disorders, 5:436-442

Kendell R, Jablensky A: (2003). Distinguishing Between the Validity and Utility of Psychiatric Diagnoses.  Am J Psychiatry, 160:1

Kramlinger KG, Post R: (2006). Ultra-rapid and ultradian cycling in bipolar affective illness. The British Journal of Psychiatry, 168:314-323 (1996)

Leibenluft E, Charney DS, Towbin KE, Bhangoo RK, Pine DS: (2003). Defining Phenotypes of Juvenile Mania. Am J Psychiatry, 160:430-437

Leibenluft E, Cohen P, Gorrindo T, Brook JS Pine DS: (2006). Chronic versus episodic irritability in youth: a community-based, longitudinal study of clinical and diagnostic associations.  J Child Adolesc Psychopharmacol, 16:456-466

Leverich G, Nolen W, Rush AJ, McElroy S, Keck Jr. PE, Denicoff KD, Suppes T, Altshuler LL, Kupka R, Kramlinger KG, Post RM: (2001). The Stanley Foundation Bipolar Treatment Outcome Network I. Longitudinal methodology.  J of Affective Disorders 67: 33-44

Lewinsohn PM, Klein DN, Seeley JR: (1995).  Bipolar Disorders in a Community Sample of Older Adolescents: Prevalence, Phenomenology, Comorbidity, and Course.  J Am Acad Child Adolesc Psychiatry, 34:4 pp 454-463

Mataix-Cols D, Rosario-Campos MC, Leckman JF: (2005 Feb.). A multidimensional model of obsessive-compulsive disorder.  Am J Psychiatry, 162(11):2193-4

Moreno C, Laje, G, Blanco, C, Jiang, H, Schmidt, A.B, & Olfson, M: (2007). National trends in the outpatient diagnosis and treatment of bipolar disorder in youth. Archives of General Psychiatry, 64:1032–1039.

Papolos D, Hennen J, Cockerham M, Thode Jr. HC, Youngstrom E: (2006) The child bipolar questionnaire: A dimensional approach to screening for pediatric bipolar disorder. J Affective Disorders, 95:149-158

Papolos D, Hennen J, Cockerham M, Lachman H. (2007). A strategy for identifying phenotypic subtypes: Concordance of symptom dimensions between sibling pairs who met screening criteria for a genetic linkage study of childhood-onset bipolar disorder using the Child Bipolar Questionnaire.  J of Affective Disorders, 99:27-36

Papolos D, Mattis S, Golshan S, Molay F: (2009). Fear of harm, a possible phenotype of pediatric bipolar disorder: A dimensional approach to diagnosis for genotyping psychiatric syndromes.  J of Affective Disorders, 118(1-3):28-38. Epub 2009 Jul 23

Phelps J, Angst J, Katzow J Sadler J: (2008).  Validity and utility of bipolar spectrum models. Bipolar Disorders, 10:179-193

Rich BA, Schmajuk M, Perez-Edgar KE, Fox NA, Pine DS, Leibenluft E: (2007).  Different Psychophysiological and Behavioral Responses Elicited by Frustration in Pediatric Bipolar Disorder and Severe Mood Dysregulation. Am J Psychiatry, 164:309-317

Stringaris A, Goodman R: (2009).  Three dimensions of oppositionality in youth.  Journal of Child Psychology and Psychiatry 50:3 pp 216–223

Stringaris A, Cohen P, Pine D, Leibenluft E: (2009b).  Adult outcomes of youth irritability: a 20-year prospective community-based study. Am J Psychiatry, 166:1048-1054

Stringaris A, Goodman R: (2009, April).  Longitudinal Outcome of Youth Oppositionality: Irritable, Headstrong, and Hurtful Behaviors Have Distinctive Predictions. J Am Acad Child Adolesc Pshychiatry, 48:4

Stringaris A, Baroni A, Haimm C, Brotman M, Lowe C, Myers F, Rustgi E, Wheeler W, Kayser R, Towbin K, Leibenluft E: (2010, April, in review). Pediatric Bipolar Disorder Versus Severe Mood Dysregulation: Risk for Manic Episodes on Follow-Up.  J American Academy of Child and Adol Psychiatry, (49)4; 397-405

[1] TDD differs from SMD in that symptoms of hyperarousal were dropped from the criteria.

[2] Jules Angst, Hagop Akiskal, David Axelson, Franco Benazzi, Boris Birmaher, Giovanni Cassano, Jean Endicott, Gianni Faedda, Robert Findling, Barbara Geller, S Nassir Ghaemi, Fredrick Goodwin, Kay R Jamison, Lewis Judd, Jacob Katzow, Robert Kowatch, Emanuel Kraepelin, Keith G Kramlinger, David Kupfer, Peter Lewinsohn, Mary Phillips, John Sadler, John Seeley, Demitri Papolos, Robert Post, Patricia Suppes, Eric Youngstrom

[3] Subsyndromal and minor depressive/dysthymic symptoms were much more prevalent than MDE-level symptoms (22.9% vs 8.9% of weeks); subsyndromal manic and hypomanic symptoms were 3 times more common than symptoms at the threshold for mania (7.0% vs 2.3% of weeks). Overall, most of all symptomatic weeks involved subsyndromal, minor depressive, and hypomanic symptoms (74.0%). Only 12.3% of all follow-up weeks were spent with symptoms at the threshold for MDE or mania. (Judd et al., 2002)

[4] Patients experienced approximately 39 times more depressive symptoms (50.3% of all follow-up weeks)

than hypomanic symptoms (1.3% of all follow-up weeks), and depressive symptoms were 22 times more

frequent than cycling/mixed symptoms (2.3% of all follow-up weeks)). Subsyndromal, minor depressive/dysthymic, and hypomanic symptoms (combined) were 3 times more prevalent (40.9% of all

follow-up weeks) than full MDE-level symptoms (13.0% of all follow-up weeks). Patients with BP-II spent only 0.9% of all follow-up weeks with psychotic symptoms during MDEs. (Judd et al., 2003)

[5] Overall, participants spent approximately 40% of the time during the follow-up period asymptomatic and 60% symptomatic (41.8% subsyndromal and 16.6% with syndromal symptomatology). Participants spent significantly more time in syndromal depression or mixed/cycling than in syndromal mania/hypomania. There were no significant differences in time spent in each subsyndromal polarity. (Birmaher et al., 2009)

[6] Most of these chronic symptoms were mixed (46%), followed by depression (33.8%) and mania/hypomania (21%).  (Birmaher et al., 2009)

[7] Increase was from 25 to 1003 visits per 100,000.

[8] Details provided upon request.

[9] The Child Bipolar Questionnaire (CBQ) is a parent report form that was developed to assist in the rapid identification of homogeneous subgroups of children with BD (Papolos et al., 2006). The majority of the CBQ’s 65 items are drawn from DSM-IV symptom criteria for mania and major depression, but symptoms of common comorbid conditions, such as anxiety and behavior disorders, are also represented. Items are rated on a Likert scale: “1—never”, “2—sometimes”, “3—often”, or “4—very often or almost constantly”. In  preliminary inquiries, the CBQ has demonstrated excellent reliability and validity in identifying subjects  that meet a K-SADS diagnosis of bipolar disorder (inclusive of BPDI, BPDII, and BPD-NOS) (Papolos et al., 2006).

  1. Pingback: Skydiving NYC
  2. Pingback: pickleball
  3. Pingback: hotels
  4. Pingback: water slides
  5. Pingback: rocky mount
  6. Pingback: interserver coupon
  7. Pingback: how to
  8. Pingback: click here
  9. Pingback: new website
  10. Pingback: useful reference
  11. Pingback: a fantastic read
  12. Pingback: original site
  13. Pingback: look at this site
  14. Pingback: additional hints
  15. Pingback: More Bonuses
  16. Pingback: Continued
  17. Pingback: Discover More
  18. Pingback: click resources
  19. Pingback: content
  20. Pingback: see post
  21. Pingback: look here
  22. Pingback: read more
  23. Pingback: best info
  24. Pingback: new website
  25. Pingback: news
  26. Pingback: how to
  27. Pingback: useful reference
  28. Pingback: imp source
  29. Pingback: a fantastic read
  30. Pingback: my response
  31. Pingback: their explanation
  32. Pingback: her latest blog
  33. Pingback: More Bonuses
  34. Pingback: check that
  35. Pingback: you could try here
  36. Pingback: Learn More
  37. Pingback: Get the facts
  38. Pingback: official statement
  39. Pingback: important source
  40. Pingback: click here
  41. Pingback: read more
  42. Pingback: click here

Comments are closed.