Turkish reliability and validity study of Barkley’s BAARS-IV self-report current symptoms

Abstract

Objective

Recall bias of patients and their relatives, low reliability of the history, changes in attention deficit hyperactivity disorder (ADHD) symptoms or compensatory behaviors, and various comorbidities may be associated with difficulties in diagnosing ADHD in adults. Barkley Adult Attention Deficit Hyperactivity Disorder Rating Scale-IV (BAARS-IV), a relatively new scale, needs to be studied for validity and reliability in different societies and cultures. In this study, we investigated whether the Turkish version of BAARS-IV: Present Symptoms Section is valid and reliable in the medical students who constituted the sample.

Methods

Adult Attention Deficit Hyperactivity Disorder Self-Report Scale and BAARS-IV: Present Symptoms Section were administered to 402 fifth- and sixth-year medical students. We performed exploratory and confirmatory factor analyzes. Cronbach’s alpha and inter-item correlation were evaluated. Pearson correlation test was used in inter-scale correlation analyses.

Results

For ADHD items, Kaiser-Meyer-Olkin (KMO) (0.882), Measuring of Sampling Adequacy MSA (0.819), chi-square (2512.455), and p (< 0.001) values showed that the data were suitable for factor analysis. A three-factor structure with an eigenvalue above 1 and explaining 53.14% of the total variance was obtained. In the correlation matrix reconstructed with eight Sluggish Cognitive Tempo (SCT) items, the determinant of the correlation matrix was 0.025, the KMO determinant was 0.852, the lowest MSA value was 0.750, and the residual rate was 14%. A two-factor structure was obtained explaining 67.451% of the variance. Cronbach’s alpha values of the inattention and impulsivity subscales were 0.839 and 0.752, respectively. Cronbach’s alpha value of 8 questions in the SCT scale was found to be 0.871.

Conclusion

The results of our study indicate that the BAARS-IV Self-report Present Symptoms scale is valid and reliable among medical students.

Introduction

Attention deficit hyperactivity disorder (ADHD) is a common neurodevelopmental disorder which was initially considered only a childhood disorder, but actually continues into adolescence and adulthood [1]. Recall bias of patients and their relatives, low reliability of the history, changes in ADHD symptoms or compensatory behaviors, and various comorbidities may be associated with difficulties in diagnosing ADHD in adults [2].

Adult ADD/ADHD Diagnosis and Evaluation Inventory based on DSM-IV, Adult Attention Deficit Hyperactivity Disorder Self-Report Scale (ASRS) [3], Brown Attention Deficit Disorder Scale Adult Form [4], and Diagnostic Interview for Adult Attention Deficit Hyperactivity Disorder (DIVA) [5] can be used to assess adult ADHD. Additionally, the Adult Concentration Inventory (ACI) was developed by Becker and colleagues [6], and its validity and reliability were examined in a sample of university students and the general population. Ten out of 16 items have been shown to be effective in distinguishing (DSM-IV), and the BAARS-IV Present Symptoms scale has a 9-item (SCT) from ADHD in adults.

Barkley Adult Attention Deficit Hyperactivity Disorder Rating Scale-IV(BAARS-IV) was developed by Barkley [7] and includes both a self-report scale and a scale of others answered by someone who knows the person closely, such as a parent or partner. In addition to the 18 ADHD symptoms in the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV), the BAARS-IV Present Symptoms scale has a 9-item Slow Cognitive Tempo (SCT) subscale. The validity and reliability of the SCT subscale in Turkish were established [8].

In this study, we aimed to investigate whether the Turkish version of the Barkley Adult Attention Deficit Hyperactivity Disorder Rating Scale-IV (BAARS-IV): Present Symptoms Section is a valid and reliable tool for Medical School Students.

Materials and methods

Participants

This study was approved by the Local Ethical Committee (2020/16) and was performed in accordance with the ethical standards of the 1964 Declaration of Helsinki. We evaluated 432 fifth- and sixth-year medical students for inclusion in the study. All participants signed a consent form. The inclusion criteria were determined as follows: (1) being a medical student, (2) agreeing to participate in the research after having full information about the research, and (3) having a good level of reading and understanding Turkish. Those who reported a previous history of schizophrenia/bipolar disorder and related disorders in the sociodemographic data form (n = 1) and those who refused to participate in the research (n = 29) were not included in the study.

Measures

Sociodemographic data form

A sociodemographic and clinical data form was used, which included information such as participants’ age, gender, marital status, and any psychiatric diagnosis.

Adult Attention Deficit Hyperactivity Disorder Self-report Scale (ASRS)

The validity and reliability study of the Turkish version of ASRS was performed by Dogan et al. [9]. It includes questions evaluating 18 symptoms based on DSM-IV. Each item is scored from 0 to 4, increasing according to the frequency of symptoms. In our study, the Cronbach's alpha value was 0.882.

BAARS-IV-Present Symptoms Self-Report Subscale

This scale consists of 5 parts in total. While the first 27 questions consist of Likert-type questions, the last 3 questions are questions about the effect on functionality and the age of onset of symptoms. The scale included 9 questions about inattention, 5 about hyperactivity, 4 about impulsivity, 9 questions about SCT, and 3 questions about age of onset and functionality. Permission was received from Guilford Press and Barkley to translate the scale into Turkish. The translation from the original English to Turkish was made by two physicians who are fluent in English (COM, DS), and were separately and then together, and the final text was created, taking into account previously valid and reliable ADHD and SCT scales. The back-translation of the scale was made by two physicians who were blind to the original scale and had good English proficiency (BD, YBS), and it was compared with the English original under supervision of Barkley and Sevincok. Barkley RA, the owner of the scale, drew attention to the problems in the translation of questions 16 and 23. When the problem in question 16 was examined, it was determined that the problem occurred due to the error in the back translation. For question 23, the sentence “spacey or in a fog,” translated as “I am confused, my mind is blurred” is left as it is. The word confused is indicated as “astonished” in the back translation. Here is the critique of the owner of the original scale: “The use of the word ‘astonished’ means being surprised when faced with an unexpected event,” but what is meant in the article in question is “not being able to think clearly about something, experiencing confusion”. Since the word “confused” in Turkish means “his thoughts are scattered, confused, he does not know what to do,” no correction has been made, with Barkley’ permission [10].

Statistical analysis

Data were analyzed using SPSS 18.0 (SPSS Inc. Released 2009. PASW Statistics for Windows, Version 18.0. Chicago: SPSS Inc.) and AMOS 26 software packages. Of the 417 participants who agreed to participate in the study, 1 participant was not included in the analysis because she reported that she had a diagnosis of bipolar disorder. Missing data were detected in the BAARS-IV scale of seven (1.7%) participants. Descriptive statistics are given as numbers and percentages. Normally distributed data are presented with mean and standard deviation.

In order to ensure the suitability of the data set for factor analysis, the Mahalanobis distance was calculated. The Mahalanobis distance was divided by the number of items (Mahalanobis D2/independent variables), and the data of seven participants with values above 3 were excluded from the analysis because they were considered outliers [11]. The adequacy of the sample size of the study and the suitability of the data for factor analysis were evaluated with the Kaiser-Meyer-Olkin (KMO) and Barlett Sphericity Test. The suitability of continuous variables in the study for normal distribution was evaluated according to the values of 2 for skewness and 7 for Kurtosis [12]. It was thought that there might be a problem only in Article 13.

Exploratory factor analysis (EFA): Since the factors were thought to be related to each other, the Principal Axis Factoring estimation method and rotation methods that allowed oblique rotation were used [10]. Separate analyses were conducted for the ADHD and SCT questions in the scale.

The maximum likelihood estimation method was used in confirmatory factor analysis (CFA). Eigenvalue number, slope plot and explained cumulative variance ratio in EFA, Relative Chi-square Fit Index in DFA, Goodness of fit Index (GFI), Adjusted Goodness of fit Index (AGFI), Root Mean Square Residuals (RMR), Root Mean Square of Approximate Errors, Standardized Root Mean Square Residual (SRMR), Comparative Fit Index (CFI), and Root Mean Square Error of Approximation (RMSEA) values were examined.

Criterion validity of the ASRS was determined by calculating sensitivity, specificity, positive predictive values, negative predictive values, and likelihood ratios (including 95% confidence intervals) for the ASRS compared with BAARS.

In the internal consistency evaluation, Cronbach’s alpha and inter-item correlation were evaluated. The correlation between the first 18 items of the BAARS-IV-Self-report Scale, included in both the DSM and the ASRS, was examined. To evaluate criterion validity, Pearson’s correlation test was used for parametric data in inter-scale correlation analyses. In the correlation analysis, the Pearson correlation value between 0.0 and 0.39 was interpreted as weak, between 0.4 and 0.69 as moderate, between 0.70 and 0.89 as strong, and between 0.90 and 1.00 as excellent [13].

The scale was re-administered to some participants 12–22 days later. The intraclass correlation (IC) value for test-retest reliability is interpreted as follows: below 0.5 is low, between 0.5 and 0.75 is moderate, between 0.75 and 0.90 is good, and above 0.90 is excellent [13].

Results

Descriptive characteristics of the participants

Of the 402 participants included in the analysis, 48.3% (n = 194) were female and 51.7% (n = 208) were male. The mean age of the participants was 23.74 ± 9.8. 10.0% of the participants (n = 40) reported that they were currently using at least one psychotropic medication (Table 1).

Table 1 Sociodemographic and clinical characteristics of the sample (n = 402)

Construct validity for ADHD items

KMO value was found to be 0.882. Additionally, the item with the lowest value in the Measuring of Sampling Adequacy (MSA) was 0.819. As a result of the Barlett sphericity test, the chi-square value was found to be 2512.455, and the p value was less than 0.001. These values show that the data is suitable for factor analysis.

EFA yielded a three-factor structure with an eigenvalue above 1 and explaining 53.14% of the total variance (Table 1, Fig. 1).

Fig. 1

Scree plot of exploratory factor analysis for ADHD items

Although the item “not listening when spoken to directly,” which is normally an item of the inattention dimension, is loaded at a similar value in the hyperactivity dimension, both factor loadings are lower than the recommended value of 0.4. Model fit was evaluated with CFA in order to confirm the factor structure determined after EFA. In the CFA performed with the data set containing extreme values, it was determined that the fit index values were similar (Fig. 2).

Fig. 2

AMOS image with ADHD items

KMO (0.864), the lowest MSA value (0.772), Barlett sphericity test results (p < 0.001), and the determinant of the correlation matrix (0.016) were found suitable for factor analysis conducted with the nine questions suggested for SCT. Principal axes estimation method and oblimin rotation were used in factor analysis. The pattern matrix and shared variance table of the factor analysis consisting of nine SCT items are presented (Table 2, Fig. 3). Compared to the other items, it was found that the ninth item had the lowest shared variance value and its factor loading after rotation was less than 0.4 (0.399). When CFA was performed by preserving item 9, model fit was found to be lower. In addition, in the analysis performed on items 6 to 9, the corrected item-total correlation was found below 0.5 (0.497) only in the 9th item. Therefore, the ninth item was removed and the analysis was repeated with 8 items. In the analysis performed with eight SCT items, the determinant of the correlation matrix was found to be 0.025, the KMO determinant was 0.852, the lowest MSA value was 0.750, and the residual rate in the reconstructed correlation matrix was found to be 14%. Using the principal axes estimation method and oblimin rotation, a two-factor structure was obtained explaining 67.45% of the variance. EFA findings are shown in Table 3 and Fig. 4.

Table 2 Exploratory factor analysis and factor loadings

Fig. 3

Scree plot of exploratory factor analysis for SCT items

Table 3 Exploratory factor analysis and factor loadings

Fig. 4

AMOS image with SCT items

Reliability and internal consistency findings

The internal consistency of all scales and subscales was evaluated with Cronbach’s alpha number (Table 4). A Cronbach’s alpha value above 0.7 is acceptable, and a value above 0.8 strongly supports internal consistency [14]. Cronbach’s alpha value of the inattention subscale was 0.839. Although there was no item that increased the Cronbach’s alpha when deleted, the corrected item-total correlation of the 3rd item was found below the desired value of 0.5. The Cronbach’s alpha value of the hyperactivity subscale was 0.791, and the corrected item-total correlation of the 10th item is below 0.5, and when deleted, it increases the Cronbach’s Alpha value. The Cronbach’s alpha value of the impulsivity factor is 0.752, and the correlation between items 15 and 17 is below 0.3. Cronbach’s alpha value of 8 questions in the SCT scale was found to be 0.871. Considering the correlation value between the items, the correlation of SCT 1 item with items 7 and 8 remains below 0.3. The Cronbach’s alpha value of the ADHD scale (first 18 questions) is 0.858.

Table 4 Reliability analysis of the scale

Criterion validity

The correlation between BAARS-ADHD items (first 18 items) and simultaneously administered ASRS scores was 0.762 (p < 0.001). An area under the curve (AUC) of 0.891 (95% confidence interval [95% CI]: 0.852, 0.929) was obtained. A cut-off score of 29 for BAARS corresponds to sensitivity = 0.85, specificity = 0.81, positive predictive value (PPV) = 0.56, and negative predictive value (NPV) = 0.94.

Test–retest reliability

IC of the inattention subscale was 0.867 (p < 0.001), and the IC of hyperactivity and impulsivity subscales were 0.573 and 0.843, respectively (p < 0.001). The IC calculated with 8 questions on the SCT scale was 0.888 (p < 0.001). The IC of the SCT absent-mindedness subscale was determined as 0.847 (p < 0.001). The IC of the SCT slowness subscale was found to be 0.896 (p < 0.001).

Discussion

Our sample size is in line with the recommendations of that when conducting EFA, the sample size should be more than five or ten times the number of items examined and the number of participants should be more than 300 [15]. All KMO values examined in our study are above the required value of at least 0.5. In our study, the determinants of the correlation matrices were also determined, factor analysis was performed with those provided the desired value, and principal component analysis was performed with those below the desired value. In this study, impulsivity and hyperactivity items do not correlate with SCT items or show low correlation. For this reason, although the BAARS-IV scale was developed as a scale consisting of ADHD and SCT items, it was thought that it would be more appropriate to evaluate it as two separate scales. When the construct validity of ADHD items is examined, it is suggested that the two-factor model consisting of inattention and hyperactivity/impulsivity is more compatible in children, while inattention, hyperactivity, and impulsivity constitute three different factors in adults [16]. In this study, hyperactivity and impulsivity were separated from each other and formed two separate factors.

In the internal consistency analysis where 26 items were evaluated, the Cronbach’s alpha of the scale increased slightly when the 15^th item was deleted. Although “talking too much” is included in the hyperactivity dimension in DSM-IV-TR, it is also found in the impulsivity dimension in some studies [17, 18]. “Talking too much” is also seen in mood disorders and anxiety disorders at the same rate as adult ADHD [19]. In the past, shyness, not talking much, and introversion were appreciated features in Turkey and other eastern cultures, but the perspective on these features may have changed over time. In our sample, the item “talking a lot in social situations” may have been perceived as a positive feature. For these reasons, when evaluated together with the entire scale, it is understood that deleting the item causes an increase in Cronbach’s alpha value.

In the original validity and reliability study of the scale, four factors—inattention, hyperactivity, impulsivity, and SCT—were created. In our study, a five-factor structure was formed when all factors with an eigenvalue above one were considered. In the original validity and reliability study, the items collected in the SCT factor and the items collected in the inattention factor also took the 8^th item, which was supposed to be inattentive, and created three different factors due to the division of SCT items into two [7].

The slowness factor consists of the 24^th, 25^th, and 26^th (SCT: 6–8) items and does not include items that are symptoms of inattention. In a meta-analysis, it was reported that these three items predicted SCT well and loaded on the SCT factor each time, but not on the inattention factor [20]. Similar to the analysis performed by Takeda et al. [21], it was revealed that the 8 SCT items in the BAARS-IV scale were significantly different from the ADHD inattention dimension. In their study, the authors also indicated that items 1, 5, 6, 7, and 8 of SCT would be distinctive.

In a thesis study examining the validity and reliability of BAARS in Chinese, BAARS-IV scale questions were divided into 5 factors, and question 19 was loaded on the inattention factor; questions 20, 21, and 22 included the factor “Boredness/Boredom”; questions 23, 24, 25, 26, and 27 (SCT: 5–9) constituted the “Slowness factor” [22]. In a study conducted with children [23], hyperactivity-impulsivity and inattention factors were revealed in the factor analysis performed with hyperactivity impulsivity, SCT and inattention items, and SCT items were collected with a higher factor load in the inattention factor. In our data, when the distribution of 2 factors is forced, hyperactivity-impulsivity items constituted one factor, while inattention and SCT items constituted the other factor.

Considering the 8^th item in the BAARS-IV Self-Report Scale – “Easily distracted by external stimuli or irrelevant thoughts”—which is expected to be included in the inattention factor—according to DSM 5, it may include irrelevant thoughts in adults. Being distracted suggests external distractibility, while being distracted by irrelevant thoughts suggests internal distractibility. The so-called mind-wandering which are thoughts independent of external stimuli or tasks, reminds daydreaming or rumination. “Mind-wandering” and “daydreaming” are close concepts. In BAARS-IV, it is understood that daydreaming tendency item (item 19) and the 8^th item, being distracted by irrelevant thoughts, are close to each other, so they combine in the distraction factor. It seems acceptable that these two “task-related” thinking styles, which are used interchangeably in some studies, can not be differentiated by the participants on the self-report scale. In the Chinese study of the scale, item 19, day-dreaming, was loaded on the inattention dimension [22].

Becker et al. [24] examined the factor structure of the BAARS-IV self-report scale in a sample of university students and added some changes to increase model fit as a result of CFA. The analysis program used to evaluate the model fit in the female gender suggested that allowing for correlation between ADHD question 8 and SCT question 1 (item 19) might improve model fit, although the researchers did not make this change because the two items reflect different dimensions. The relationship between these two items is also seen in this study. SCT item 2 (BAARS-IV item 20) means “I have trouble staying alert or awake in boring situations” and is one of the items removed from the SCT scale in the Japanese sample, because it also loads on the inattention dimension [21]. In the present study, it was loaded on the absent-mindedness factor.

In another study conducted with children, the two items expected to represent SCT in the data form filled out by teachers—“low entrepreneurship” and “gets bored easily, needs stimulation”—were loaded with a higher factor load on the inattention factor [25]. In our study, it was shown that the 4^th item (BAARS-IV scale, 22nd item, “I get bored easily”) in the “inattention” factor was not distinctive for SCT and was also loaded on the inattention factor, and this item was not sufficiently distinctive in the Japanese SCT scale study [21]. It can be thought that this item may be related to both SCT and inattention.

SCT Item 5 (BAARS-IV Item 23) “I am confused or confused” is a similar statement to the item “My mind feels like it is in a fog” in the ACI validation study [6] on university students and was not found to be sufficiently distinctive. In a study conducted with adolescents in South Korea, the items “I am slow in doing things,” “I feel confused/feeling confused,” “I think slowly,” and “I have difficulty putting my thoughts into words” in the self-report scale did not meet sufficient divergent and convergent validity for SCT [26]. Researchers have stated that individuals may have difficulty evaluating themselves with these symptoms due to slowness and mental confusion, or may not be aware of themselves despite being aware of others.

In a study using 5 items related to SCT in the Child Behavior Check List (CBCL), the item “low activity” was not used in the study, because it showed a low correlation with other items [27]. Possibly, this item can be associated with both the SCT and the inattention dimension [28]. In our study, the item “I am less active or have less energy than others” (item 25, item) is included in the slowness factor. In the study conducted by Willcutt and colleagues with children [28], the items “seems not to hear,” “easily confused,” and “gets distracted,” which were expected to be suitable for SCT, were not used in the study because they were also added to the inattention dimension. Additionally, the major depressive disorder item (psychomotor retardation/agitation) was loaded on the SCT factor. Similar to our study, it was found that symptoms representing low energy and slowness were more successful in differentiating it from ADHD than cognitive symptoms such as confusion.

When hyperactivity-impulsivity symptoms among SCT and ADHD items were evaluated, it was seen that SCT and hyperactivity/impulsivity symptoms were negatively related to each other [29]. Similarly, in our study, the correlations between the slowness factor and the impulsivity and hyperactivity factors appear to be quite low and insignificant. Our findings also demonstrated that the slowness factor was more clearly separated from ADHD items than the absent-mindedness factor. The items in the distraction factor and the ADHD items, especially items 8^th and 9^th items, may be items common to both disorders.

In the Turkish validity and reliability study of the SCT items of the same scale [8], similar problems were also experienced in the 9^th item of SCT. SCT Item 9 (I don’t seem to understand information as quickly or accurately as others) was removed from the analysis because it caused problems in both validity and reliability analyses. This may be because both groups were more successful than the general population. Although it was stated that they should evaluate themselves according to the last 6 months, the participants may have evaluated themselves according to the general population and therefore do not see themselves as “understanding information slowly.” Although it seems consistent that this item was excluded both in our study and in Gul and Gul’s study [8], it is difficult to reflect this to the general population. It would be more accurate to re-evaluate this item in a study involving a general sample before deciding that this item is not suitable for cultural reasons.

When the correlation of the scales and factors was examined, a non-significant relationship between the slowness factor, which included the items “I am sleepy and tired compared to others,” “I am less active or have less energy compared to others,” “I move slowly,” BAARS-IV and the hyperactivity and impulsivity factors. The correlation between the ASRS hyperactivity/impulsivity factor is quite low, although the p value is significant. The “slowness” factor was more related to the inattention and absent-mindedness factors than the depression scale. Although factor analysis has shown that SCT is different from depression/anxiety disorders, internalization disorders such as depression/anxiety disorders are common in SCT. This study has some important limitations. Since our sample consists of medical school students, it may differ from the distribution in the general population and may not be representative the ADHD sample in the population. In addition, cultural differences may have played a role in the formation of different factor structures. All applied scales are self-report scales. Although the use of a self-report scale is a low-reliability method, the fact that our population consists of medical students indicates that the intellectual levels of the participants are high and that they can provide more accurate information in the self-report scales. Additionally, no diagnostic interview was conducted. Bipolar disorder was detected in only one participant on the self-report scale. This rate was considerably lower than the general population.

Conclusion

BAARS-IV Self-report form differs from the original factor structure in Turkish. In our study, the scale items formed a five-factor model. BAARS-IV self-report ADHD items (items 1–18) are divided into three factors: impulsivity, hyperactivity, and inattention. The results of our study indicate that the BAARS-IV Self-report Present Symptoms scale is valid and reliable. Item 15 of the ADHD scale “I talk a lot in social situations” caused some problems in internal consistency. We think that it may be necessary to add some explanations to this question in further studies using the scale. Separate analyses were conducted for ADHD and SCT items. SCT items are divided into two subscales. The 9^th item of the SCT was excluded from the analysis due to its low factor load, and the items from the 1^st to the 5^th item of the SCT constituted the absent-mindedness factor from, and the 6^th, 7^th, and 8^th items constituted the slowness factor. The data show that the 8-item SCT scale is valid and reliable.