http://www.dx.doi.org/10.14718/ACP.2019.22.1.7
MAGALY NÓBLEGA *, 1
GABRIELA CONDE 2
JUAN NUÑEZ DEL PRADO 3
PATRICIA BÁRRIG 4
FRANCESCO MARINELLI 5
NATALI ALCÁNTARA 6
RODRIGO CÁRCAMO 7
* Docente principal del Departamento de Psicología, Av. Universitaria 1801, San Miguel, Lima, Perú. Teléfono: 626 2000, anexo 4534. mnoblega@pucp.pe
1 Pontificia Universidad Católica del Perú ORCID: https://orcid.org/0000-0001-6572-813X
2 Pontificia Universidad Católica del Perú ORCID: https://orcid.org/0000-0002-6103-1025
3 Pontificia Universidad Católica del Perú ORCID: https://orcid.org/://0000-0001-5502-514X
4 Pontificia Universidad Católica del Perú ORCID: https://orcid.org/0000-0003-4685-5760
5 Pontificia Universidad Católica del Perú ORCID: https://orcid.org/0000-0002-0470-0130
6 Pontificia Universidad Católica del Perú ORCID: https://orcid.org/0000-0003-4721-7023
7 Universidad de Magallanes, Chile ORCID: https://orcid.org/0000-0002-9227-7132
Recibido, marzo 28/2018;
Concepto de evaluación, mayo 4/2018;
Aceptado, mayo 22/2018
How to quote this article: Nóblega, M., Conde, G., Nuñez del Prado, J., Bárrig, P., Marinelli, F., Alcántara, N. & Cárcamo, R. A. (2019). Evidencias de validez de constructo y de criterio de la Escala Massie-Campbell de Apego Durante Estrés (ADS). Acta Colombiana de Psicología, 22(1), 141-151. doi: http://www.dx.doi.org/10.14718/ACP.2019.22.1.7
Abstract
The Massie-Campbell Attachment During Stress Scale (ADS, Massie & Campbell, 1983) is an observation guide of the interactions between mothers (or caretakers) and children from 6 to 18 months. This study sought the construct validity (convergent and discriminant) of ADS, as well as its criterion validity (sensitivity, specificity, and predictive values). To that end, child attachment was measured in a group of 32 children aged 8 to 10 months from Lima, Peru using ADS and Attachment Q-set 3.0 (AQS, Waters, 1995). Maternal sensitivity was assessed as well. It was found that ADS attachment classifications were related to AQS attachment security continuous scores (r = .41, p = .02) and to its dichotomous classifications [secure vs. insecure; x2 (1, N = 32) = 4.69, p = 0.03, d = 0.83]. Additionally, no significant differences between the ADS's maternal sensitivity of mothers with children classified as secure and mothers with children classified as insecure were found. Since ADS's sensitivity did not reach satisfactory levels (33.3 %, 95 % CI = [15.48, 56.90]), a cut-off point of 5 secure behaviors was used, which improved its sensitivity (47.6 %, 95 % CI = [26.4; 69.7]) while maintaining satisfactory levels of specificity (90.9 %, 95 % CI = [57.1; 99.5]). The instrument shows adequate convergent validity but lacks evidence of discriminant validity. Further exploration of ADS's sensitivity, specificity, and predictive values is recommended in order to obtain a cut-off point with greater validity, given that the instrument is considered a screening test.
Keywords: attachment, Massie-Campbell Scale, ADS, validity.
Resumen
La escala Massie-Campbell de Apego Durante Estrés (ADS; 1983), es una guía de observación de la interacción entre madres (o cuidadores) y sus hijos de 6 a 18 meses. Este estudio busca obtener evidencias de validez de constructo (convergente y divergente) y de criterio (sensibilidad, especificidad y valores predictivos) de dicho instrumento. Para ello, se evaluó el apego en 32 niños y niñas de ocho a diez meses de Lima, Perú, utilizando el ADS y el Attachment Q-set 3.0 (AQS), de Waters (1995), así como la sensibilidad de sus madres. Como resultado, se encontró que las clasificaciones del apego seguro e inseguro dadas a partir del ADS se encuentran relacionadas con los puntajes continuos de seguridad del apego del AQS (r = .41, p = .02) y con las clasificaciones dicotómicas (seguro vs. inseguro) obtenidas a partir del mismo [x2 (1, N = 32) = 4.69, p = .03, d = .83]. Adicionalmente, no se hallaron diferencias significativas en la sensibilidad materna de las madres de los niños clasificados como seguros y la de los inseguros. Dado que la sensibilidad de la escala ADS no alcanzó niveles satisfactorios (33.3 %, IC 95 % = [15.48; 56.90]), de manera exploratoria se evaluó un punto de corte de cinco conductas seguras, el cual mejora su sensibilidad (47.6 %, IC 95 % = [26.4; 69.7]) y mantiene sus niveles satisfactorios de especificidad (90.9 %, IC 95 % = [57.1; 99.5]). En conclusión, el instrumento muestra adecuadas evidencias de validez de constructo convergente, pero no divergente; y, al ser considerado un instrumento de tamizaje, su sensibilidad, especificidad y valores predictivos necesitan seguir siendo estudiados con el fin de conseguir un punto de corte con mayor validez.
Palabras clave: apego, escala Massie-Campbell, ADS, validez.
Resumo
A Escala Massie-Campbell de Apego durante o Stress (ADS; 1983) é um guia de observação da interação entre mães (ou cuidadores) e seus filhos de 6 a 18 meses. Este estudo procura obter evidências de validade de constructo (convergente e divergente) e de critério (sensibilidade, especificidade e valores preditivos) desse instrumento. Para isso, foi avaliado o apego em 32 crianças de oito a dez meses, de Lima, Peru, utilizando o ADS e o Attachment Q-set 3.0 (AQS), de Waters (1995), bem como a sensibilidade de suas mães. Como resultados, constatou-se que as classificações do apego seguro e inseguro dadas a partir do ADS se encontram relacionadas com as pontuações contínuas de segurança do apego do AQS (r = .41, p = .02) e com as classificações dicotômicas (seguro versus inseguro) obtidas a partir deste [x2 (1, N = 32) = 4.69, p = .03, d = .83]. Além disso, não foram encontradas diferenças significativas na sensibilidade materna das mães das crianças classificadas como seguras e a das inseguras. Tendo em vista que a sensibilidade da escala ADS não atingiu níveis satisfatórios (33.3 %, IC 95 % = [15.48; 56.90]), de maneira exploratória, foi avaliado um ponto de corte de cinco comportamentos seguros, o que melhora sua sensibilidade (47.6 %, IC 95 % = [26.4; 69.7]) e mantém seus níveis satisfatórios de especificidade (90.9 %, IC 95 % = [57.1; 99.5]). Em conclusão, o instrumento mostra adequadas evidências de validade de constructo convergente, mas não divergente; e, ao ser considerado um instrumento de rastreamento, sua sensibilidade, especificidade e valores preditivos precisam continuar sendo estudados a fim de conseguir um ponto de corte com maior validade.
Palavras-chave: apego, escala Massie-Campbell, ADS, validade.
Introduction
Bowlby's theory of attachment (1969) defines attachment as an affectional bond between a child and his or her primary caregiver that is sustained over time and space (Ainsworth, Blehar, Waters, & Wall, 1978). According to this theory, human beings have an innate need to develop attachment to one or more individuals providing care and protection. For this reason, a number of instinctive attachment behaviors intended to promote caregiver proximity —crying, gaze orientation, vocalization, and grasping, among others— are inborn in humans. Based on a child's mental and physical development as well as his or her repeated experiences of interaction with the caregiver, these behaviors are organized into specialized strategies in order to achieve physical and/or affectional proximity with the attachment figure. In parallel, infants develop exploratory behaviors aimed at learning and managing their physical and social environment; these behaviors will allow them to gain progressively more independence from their attachment figures and focus their attention on contextual stimuli.
The term secure base behavior describes the balance between proximity-seeking and exploration behaviors; it is defined as the child's ability to use the primary caregiver as a secure base for the exploration of the surrounding environment to which he/she can return when facing a threat or in moments of stress (Bowlby, 1988). The achievement of this balance signals the existence of a secure attachment bond, whereas an imbalance towards either of the two ends of the spectrum reveals insecure attachment (Cassidy, 2016). Specifically, an imbalance towards exploration signifies an avoidant insecure attachment, and an imbalance towards proximity seeking indicates an ambivalent insecure attachment.
The most important instrument for assessing secure base behavior is the Strange Situation Procedure (SSP, Ainsworth et al., 1978). The SSP uses observations in a moderately stress-inducing laboratory situation to classify the child's behavior into different types of attachment. Despite it being considered the "gold standard" of attachment assessments, some authors have identified limitations in the SSP connected to its application procedures, its rating system, and its ecological validity (Clarke-Stewart, Goosens, & Allhusen, 2001; Waters & Deane, 1985). For instance, it has been argued that the age range for SSP application is overly short, that the recommended period between applications is overly long, and that its administration and scoring costs are excessive. Also, the SSP's scoring system has been criticized for providing only limited information about the child's attachment system in its taxonomic results, and for constraining the options for analysis. Finally, in connection to the SSP's ecological validity, critics argue that in children from certain cultures, who may be more habituated to the routine of being with a stranger, having the mother's absence as a source of stress can interfere with the regular display of attachment behaviors.
The Attachment Q-set 3.0 (AQS, Waters, 1995) was developed in response to some of these criticisms. It is considered the second most important instrument for assessing attachment (Van IJzendoorn, Vereijken, Bakermans- Kranenburg, & Riksen-Walraven, 2004). The purpose of the AQS is to evaluate the organization of secure base behavior using the Q-sort methodology. This procedure requires observing child-caregiver interactions for a period of two to six hours. The AQS provides a measure of the child's level of attachment security by correlating the observed behavior to a security criteria previously defined by experts.
The AQS's indicators have been deemed adequate as an instrument for assessing attachment in infants and preschoolers in different contexts. For example, Cadman, Diamond and Fearon (2017) reported adequate evidence of the test's validity and reliability in a meta-analysis that included 268 independent samples. Regarding the evidence of validity of the instrument, on the one hand, they found a moderate effect size of r = .20 between the AQS scores and the SSP classifications (r =.39 for observations over 180 minutes long), and also a significant association of r = .17 with disorganized attachment, compared to the other combined SSP categories. On the other hand, they reported that observed AQS scores are related to caregiver sensitivity with an effect size of r = .32, and to the child's social-emotional competence with an effect size of r = .24. In addition, they found a significant association of r = .27 in the expected direction with the child's temperament. In terms of the evidence of its reliability, the test-retest analysis revealed significant associations over time for the observed scores (r = 41), and an agreement between self-reported scores and observed scores (r = .43), based on 12 studies.
In Peru, the AQS has been used in different socioeconomic strata and in family and non-family environments, showing adequate psychometric characteristics (Conde, Nóblega, Fourment, & Nuñez del Prado, 2015; Nóblega, Fourment, Núñez del Prado, Conde, & Bahamonde, 2015; Posada et al., 2013; Posada et al., 2016). These studies reported a variation of .43 to .60 between the AQS's security scores and maternal sensitivity scores measured with the Maternal Behavior for Preschoolers Q-set. Meanwhile, the inter-rater reliability indexes reported by studies carried out in a Peruvian context fall between .76 and .96 (Buitrón, 2011; Conde et al., 2015; Dávila, 2013; Nóblega, 2012; Posada et al., 2016; Posada et al., 2013; Ugarte, 2014). Finally, the use of this instrument to assess the precursors of secure base behavior in children under one year old was explored, with similar findings (Nóblega et al., 2016).
Like SSP, there are drawbacks to the AQS, such as a long observation period and a complex rating system that, depending on the coder's level of expertise, can take between 30 and 60 minutes. This renders the use of the AQS problematic for large-scale research studies. In this context, the ADS has emerged as an alternative instrument, and in recent years it has gained relevance in Latin America. It makes it possible to identify potentially problematic patters of interaction (of the avoidant or ambivalent type). The ADS's psychometric properties have been reported as adequate for assessing the quality of the interaction of child-caregiver dyads in stress situations (Massie & Campbell, 1983, 1992; Tryphonopoulos, Letourneau, & Ditommaso, 2014).
Given the great importance of the caregiver's behavior for the display of the child's attachment behaviors, this instrument's scoring takes into account both the child's and the caregiver's behavior in two separate sub-scales (Massie & Campbell, 1983).
Among the advantages of the ADS over other instruments are its application to dyads with younger children, from birth to 18 months of age, its short observation period, between five and fifteen minutes, and the possibility of using it for observations both in natural environments and in structured or laboratory situations (Massie & Campbell, 1992).
Although the ADS has been used in research (Figueroa, Binda, & Santelices, 2012; Kotliarenco, Gómez, & Muñoz, 2009; Lecannelier et al., 2009; Leigh, Vergara, & Santelices, 2012), evidence about its psychometric characteristics is limited (UIT, n. d.)
The research conducted by Cárcamo, Van IJzendoorn, Vemeer, and Van der Veer (2014) is the most recent psychometric study accounting for various types of evidence of the validity of the ADS. In terms of convergent validity, it found a significant association between the secure and non-secure categories of the ADS total general scale and the SSP [x2 (1, N = 69) = 13.20, p <.001, d = 0.96], and for the child behavior sub-scale X2 (1, N = 69) = 4.94, p = .026, d = 0.50]. However, a comparison of the three types of attachment found that although the classifications of both scales were associated, secure attachment is over-represented and avoidant attachment under-represented in the ADS. With regards to evidence of discriminant validity, the analy sis found that the group classified as insecure using ADS included mothers with significantly lower sensitivity scores than those in the group classified as secure (Mmothers of insecure children = 4.70; SDmothers of insecure children = 1.35; Mmothers of secure children = 5.66; SDmothers of secure children = 1.14, t = -2.85, gl = 67, p = .006, d = 0.76). Finally, to test content validity, a group of four attachment experts was asked to classify the ADS statements according to the three types of attachment featured in the scale; in this procedure, the four experts correctly classified 40 of the 70 statements.
Based on these results, the authors concluded that the performance of the ADS is "satisfactory in distinguishing between secure and insecure children" (Cárcamo et al., 2013, p. 772), taking into account the mother and child scores globally and also taking into account only the child's scores. However, at a more granular level, they report that the ADS has difficulty discriminating between different types of attachment and recommend that the results be interpreted cautiously. Also, given the limitations of their study, the authors emphasize the importance of evaluating the instrument in a natural context, not a laboratory, and with populations of different social and economic characteristics.
In this way, and given that public policy in several Latin American countries, including Peru, prioritize attachment as an important result of early childhood development, it is important to have a short, easy to apply assessment instrument with evidence of validity for its use on a large scale. This is why our psychometric, instrumental study sought to analyze the evidence of construct validity (convergent and discriminant) and criteria validity (analysis of sensitivity, specificity, and predictive values) of the ADS scale in a group of mothers and their children, 8 to 10 months old in Lima, Peru.1
To generate evidence of convergent construct validity, ADS and AQS scores were compared; this instrument was used since its performance with children under one year of age has been tested in the Peruvian context (Nóblega et al., 2016). Additionally, given that there are studies that identify maternal sensitivity as a different construct associated with the development of secure attachment (Jin, Jacobvitz, Hazen, & Jung, 2012; Nóblega, 2012; Nóblega et al., 2016; Posada, 2013; Posada et al., 2016; Van IJzendoorn, Bakermans-Kranenburg, & Sagi-Schwartz, 2006), for the evidence of discriminant construct validity, the relationship between ADS scores and sensitivity scores of the mothers in the MBQS 2.1 was assessed (Pederson & Moran, 1995; Pederson, Moran, & Bento, 1999). Finally the sensitivity, specificity, and predictive values of the ADS were evaluated using the AQS's dichotomous scores as criteria.
Method
Participants
The participants in the study were 32 mothers and their children, all from Lima, Peru. The mothers' average age was 29.6 years (SD = 6.28, Min = 19, Max = 44). In terms of educational attainment, 43% had completed technical or college-level higher education, 53.1% were high school graduates, and 3.1% only had primary education. In terms of their domestic situation, 84.4% were married or had a stable partner, and 81% were mainly homemakers. The participating children were 20 boys (62.50%) and 12 girls (37.50%), and their ages were between 8 and 10 months 29 days (M = 8.91; SD = 0.96). In all cases, the main caregiver was the mother. Maternal reports verified that none of the children presented an evident pathology of development or health, autism, Down and related syndromes; or prematurity.
The study was approved by the financing university's Ethics Committee. The selection of participating dyads was based on convenience: participants were selected from among those who took their children to a government-run health center. The study was explained to each mother and they were invited to participate; only one of the mothers was referred by one of the researchers' contacts, a resident of the area where the study was conducted. Mothers who agreed to participate voluntarily signed an informed consent form describing the procedure to be followed, including benefits and possible risks; the voluntary nature of their participation was guaranteed, with the option of withdrawing at any time, as was the anonymity and confidentiality of the results.
At the end of their experience, participating mothers received an economic reward equivalent to U.S. $50.00 and a DVD recording of the observation. They were offered a psychological counselling session with two members of the research team, focused on any parenting issues they may have previously raised as concerns or on any aspects of the interaction that emerged during the observation. The research team also had a directory of professionals for the referral of possible at-risk cases, if necessary.
Instruments
Massie-Campbell Attachment During Stress Scale (ADS). The ADS is an observation guide designed for the evaluation of child-caregiver interactions in situations of moderate stress, such as a medical visit or episodes of separation (Massie & Campbell, 1983). This instrument assesses the child's and the mother's or caregiver's behaviors independently. The intensity and frequency of six behaviors are evaluated for each: gaze, vocalization, support, touch, affection, and proximity seeking. Each behavior is scored on a five-level continuum, ranging from its absence to its strong expression. A score for the dyad's interaction is assigned. Additionally, given that the original handbook does not specify a procedure for a single score, Cárcamo et al. (2014) adapted one where the presence of scores of 1 or 2 in more than half of the items is an indicator of insecure avoidant attachment; the presence of scores of 3 and 4 in more than half of the items is an indicator of secure attachment; and scores of 5 in most of the items is an indicator of ambivalent attachment.
For this study, the evaluation of mother-child interactions using the ADS was based on the observation of video recordings of structured procedures carried out as part of an earlier study. The procedure consisted of five different times. It began with 5 minutes of mother and child free play with toys placed in the space. Then there was a time when the mother left the room and the child was alone with the toys for two minutes approximately. In the third time, the mother returned to the room and interacted with the child for two more minutes, until a new person, not previously seen by the dyad, entered and removed all the toys except for one they had not played with. Finally, the dyad continued to interact for two more minutes. The procedure lasted approximately 11 minutes.
For the scoring of behaviors observed using the ADS, this study used the system proposed by Cárcamo et al. (2014) First, each of the behaviors was scored on a scale of 1, 2, 3, 4, 5 or not observed. Then, each behavior was classified as secure or insecure according to the following procedure: behaviors with scores of 3 or 4 were classified as secure and those with scores of 1, 2 or 5 were classified as insecure. Finally, dyads were classified as secure when 50% or more of their behaviors were secure; otherwise, they were classified as insecure.
The videos were rated by a group of five coders, experts in the subject of attachment, who received specialized training and rated 16 training videos. Using a prevalence-adjusted and bias-adjusted Kappa coefficient (PABAK), their inter-observer reliability was moderate, .56.
To ensure the reliability of the scores in this study, two additional independent observers rated 30% of the videos. The PABAK inter-rater reliability was moderate (.45) for the global classification of the dyads, and adequate or fair (M= .36; Min = .09, Max = .64) for the classification by behavior.
Attachment Q-Set (AQS) version 3.0. Waters' AQS (1995) is an observation instrument that evaluates the organization of secure base behavior with the purpose of assessing attachment security. This study used Nóblega's linguistic adaptation (Nóblega 2012) of the Spanish version (Posada et al., 1999). The AQS uses the Q Sort methodology and consists of 90 items that describe different behaviors of the child. As detailed in the introduction, this instrument has adequate psychometric characteristics for the evaluation of children 12 to 69 months old (Cadman et al., 2017; Nóblega, 2012; Nóblega et al., 2016; Posada et al., 2013; Posada et al., 2016; Solomon & George, 2016; Vaughn et al., 2007). Also, it was used in the Peruvian context in a previous study with children under 12 months (Nóblega et al., 2016).
For the research from which this study is derived, free mother-child interaction was recorded during approximately an hour and two independent evaluators rated the resulting video. To improve the quality of the scores obtained for maternal behavior, the rating of the observations occurred in two stages, following the procedures of previous studies (Bárrig, 2004; Nóblega, 2012; Posada et al., 2004; Posada et al., 1995; Posada et al., 2002; Vaughn et al., 2007). Average inter-rater reliability for the first stage was good, .83 (SD = 0.07, Min = .65 and Max = .92). In the second stage, items in which there was a high discrepancy between the ratings given by the observers (difference of more than 3 points) were identified for each case. The team discussed these items on the basis of behavioral referents observed during the visit, arriving at a shared description of the child's behavior. The final scores consisted of an average of the scores given by the observers to each item.
According to the procedure proposed for the instrument, the assurance of the child's attachment was calculated from the correlation between the scores given by the observer to each of the 90 items and the scores that make up a criterion agreed upon by experts regarding the scores of a hypothetical ideally secure child (Cadman et al., 2017). This theoretical score is called a security criterion and has been validated in different contexts, including the Peruvian one (Cassibba et al., 2000; Nóblega, 2012; Posada et al., 1995
There is no cut-off point that determines an attachment classification based on the scores obtained with the AQS (Van IJzendoorn et al., 2004). However, a study by Van Bakel & Riksen-Walraven (2004) with 129 Dutch dyads found that the mean attachment security in children classified as secure was significantly higher than that in children classified as insecure on the basis of their SSP assessment (Msecure children = .32, SDsecure children = .25, Minsecure children = .12, SDinsecure chlidren = .24; t = 4.34, p < .01). Similarly, Van IJzendoorn et al. (2004) reported that the mean of the security scores in the typical population differed significantly from the mean in the clinical population (Mtypical population = .32, SDtypical population = .16;Mclinical population = .21,SDclinical population = .08; t(297.20) = 16.06, p < .01). On their part, Geenen and Conveleyn (2014) differentiate secure and insecure attachment at a score of .32 in the AQS. Based on this evidence, those dyads with AQS scores above .32 can be considered secure.
Maternal Behavior Q-Sort (MBQS) version 2.1. The MBQS (Pederson & Moran, 1995) is an observation instrument that assesses maternal sensitivity using the Q-Sort methodology to describe maternal behavior for 90 items. It has been validated in a variety of contexts (Posada, 2013; Posada et al., 2016). The present study used the version adapted to the Latin American context by Posada et al. (2002).
Observations of maternal sensitivity were made simultaneously with observations of child behavior using the AQS. In order to improve the quality of the scores obtained for maternal behavior, the two-stage procedure described for the AQS was also carried out in this case. Two coders worked independently from each other and from their AQS counterparts. In the first stage, the scores of both observers were correlated, resulting in a good average inter-observer reliability, .82 (SD = 0.09). In 31 cases, the minimum obtained was .61 and the maximum, .92. In one case, a moderate reliability of .59 was obtained, and for this reason, a third evaluator was asked to code the recording of the interaction; after this procedure, inter-observer reliability for this video was .75.
Following the pre-established procedure, a global sensitivity score was obtained for each mother based on the correlation between their score for each item and the sensitivity criteria established by expert consensus. (Pederson & Moran, 1995).
Procedure
This trans-sectional quantitative study derives from a larger project [ATI (Uruguay) & the Pontificia Universidad Javeriana of Colombia] and was carried out in a variety of contexts, exploring the relationship between attachment precursors and maternal sensitivity in children under one year.
In that larger research project, two meetings were held with the participants. The first meeting took place in the mother's house, where participants signed the informed consent form and their socio-demographic data were collected. Next, the dyad's interaction was observed and recorded for one hour, and then the mother was asked to play with her child in a fixed space for five minutes. Finally, the mother completed a narrative technique to assess attachment representations. The second meeting was held in a Gesell chamber. The mothers followed a structured procedure of separations and encounters, and these were recorded. After the procedure was completed, information was collected on their rearing practices and the use of lullabies in their child's upbringing. Finally, the psychological counselling session described above was conducted.
This study used attachment-security scores (AQS) and maternal-sensitivity scores (MBQS) resulting from the observation of mother-child interactions in a home setting, and video recordings of the same dyads in a laboratory context, rated using the ADS.
Data Analysis
The IBM SPSS Statistics 22 data package was used to perform the statistical analyses in this study. Following recommendations by Cárcamo et al. (2014), it was decided to use only the infant scale for all the analyses.
First, the data set were analyzed for normality of distribution and homogeneity of variance. The skewness indicators and the kurtosis, as well as a Shapiro-Wilk test and the normality graphs, led to the conclusion that the AQS's continuous security scores (SW (32) = .949; Kurtosis = -.094; Skewness=-.675) and maternal sensitivity scores (SW (32) = .905; Kurtosis = .125; Skewness = -.880) were normal.
At a specific level and based on the same indicators, it was concluded that the distribution of security scores in groups of children classified as secure and insecure was normal (SW(25)sec, secure children group = .938; Kurtosis sec, secure children group = 1.018; Skewness sec, secure children group = -.975) vs. insecure (SW(7)sec, insecure children group = .913; Kurtosissec, insecure children group = -1.189; Skewnesssec, insecure children group = -.269), as was the distribution of scores of maternal sensitivity for mothers of secure children, established on the basis of the ADS scores (SW(25)sen, secure children group = .873; Kurtosissen, secure children group =.122; Skewnesssen, secure children group = -1.010) and insecure (SW(7)sen, insecure children group = .939; Kurtosissen, insecure children group = 1.997; Skewnesssen, insecure children group = -.007)
For evidence of the convergent validity of the ADS, the relationship between the ADS classification of secure and insecure attachment and AQS security scores was analyzed, using a point-biserial correlation. Similarly, a Student /-test mean comparison analysis of the AQS and ADS scores for children classified as a secure and insecure was carried out.
Next, Cramer's Chi-Square and V tests were performed to examine the concordance between the secure and insecure classifications in ADS and AQS. For a classification of secure vs. insecure attachment using the AQS scores, the cut-off point was .32, which earlier studies have reported as the security mean in different non-clinical populations (Geenen & Conveleyn, 2014; Van Bakel & Riksen-Walraven, 2004; Van IJzendoorn et al., 2004).
To demonstrate the scale's discriminant validity, the relationship between the ADS and MBQS maternal sensitivity scores were analyzed, using the Student's /-test to contrast the means and compare the secure and insecure groups' maternal sensitivity scores.
Based on the results of the discrepancy analysis, the sensitivity, specificity, positive predictive value, and negative predictive value of the ADS scale were examined, using the AQS classifications as reference criteria.
Finally, to identify a way to classify children with secure attachment that optimizes the sensitivity and specificity of the ADS, a Receiver Operating Characteristics (ROC) analysis was performed. For this to be achieved, a continuous score for the ADS was calculated by adding the scores of the 7 items converted to secure (1 point) or to insecure (0 points). Youden's index to define an optimal cut-off point was used.
Results
The following section presents, first, the descriptive statistics for the sample's scores in each of the instruments. Then, the results of the analysis performed to assess the evidence of convergent validity of the scale are presented, using scores and classifications provided by the AQS. Next, the results of the analysis of the scale's discriminant validity performed using the maternal sensitivity scores (MBQS) are shown. Finally, in order to demonstrate criterion validity, the results of the scale's sensitivity and specificity analyses and predictive values, as well as the ROC Curve analyses are brought forward, performed to find a new, more appropriate cut-off point.
Descriptive statistics
Assessed using the AQS, the mean for attachment security was .195 (SD =.235; Min = -.358; Max = .586). Based on the ADS's dichotomous classification, 78.1% of participating children were classified into the secure-attachment group and 21.9% into the insecure-attachment group. A media of .543 (SD =.242; Min = -.021; Max = .834) was found for maternal sensitivity using the MBQS.
Evidence of Convergent Validity: AQS and ADS Classifications
This study found a positive association between the ADS's dichotomous classification (secure attachment - insecure attachment) and the AQS scores (rpb = .408, p = .021). This means that the greater the attachment security assessed with the AQS, the more likely the child will be classified as secure based on his or her ADS scores.
Based on their ADS score, children were classified in two groups, secure and insecure attachment, and a mean comparison was performed to identify any differences. The results showed that children classified as insecure had significantly lower security scores in the AQS (Msecure children = .25, SDsecure children = .22, Minsecure children = .02, SDinsecure children = .22, t(30) = -2.446, p = .02, d = 0.89; 95% CI [ -.419; -.037]).
Similarly, a significant association between the dichotomous classifications of secure and insecure attachment provided by the ADS and the AQS [X2 (1, N = 32) = 4.69, p = .03, V = .383] was found. As shown in Table 1, in general, 56.3% of participants (18 children) have the same classification in both instruments. In addition, all participants classified as insecure in the AQS have the same classification in the ADS, and all participants classified as insecure by the ADS had scores compatible with such categorization in the AQS. However, it should be noted that 56% of the participants classified as secure by the ADS (14 children) were classified as insecure by the AQS.
Table 1. Cross Table ADS Classification and AQS Classification secure - insecure (N = 32)
Evidence of Discriminant Validity: Comparison of Maternal Sensitivity in Mothers of Secure and Insecure Children
The maternal sensitivity scores of mothers of children in the secure attachment group as determined by the ADS were not significantly higher than those of mothers of children in the insecure attachment group (Mmothers of secure children = .55, SDmothers of secure children = .26, Mmothers of insecure children = .52, SD mothers of insecure children = .19, t(30) = -.34, p = .74, d = -0.123; 95% CI [-.249;.179]).
Sensitivity, Specificity, and Predictive Values of ADS
The sensitivity of the ADS scale was 33.3% with a CI 95% CI = [15.48; 56.90]; this is to say, when using the secure/insecure classification based on the ADS, the probability of correctly identifying the cases that AQS classifies as insecure attachment is between 15% and 57%. At the same time, the ADS has a specificity of 99%, CI 95% = [67.85; 99.16], which means a 68% to 99% probability of correctly identifying cases with secure attachment.
In terms of the ADS scale's positive predictive value, the probability that a child classified in the insecure attachment group by the AQS will obtain the same classification in the ADS scale is 99%, IC 95% = [56.09; 98.68]. With regards to the negative predictive value, however, the probability that a child in the secure attachment group will be similarly classified by the AQS is only 44% [95% CI = 25.02; 64.7].
Based on these results, ROC curve and Jouden's Index analyses were performed in order to find a cut-off point for the ADS that would optimize both its sensitivity and specificity levels. The AQS scores were used as a criterion, and a continuous score for the ADS was used, adding the items' scores.
Figure 1 shows the ROC curve; the indicator of the ADS's discriminant capacity is the area under the curve corresponding to .28 (IC 95% = [.105; .453]; z = .09, p = .04), which indicates that the proposed criterion is not better than chance.
Figure 1. ROC curve of the ADS continuous scores, using the AQS scores as a criterion (cut-off point of .32, following earlier studies.)
The Youden's index analysis defines the optimal cut-off for a child to be classified in the secure attachment group based on the ADS as 4.5 secure behaviors (see Table 2.)
Table 2, Sensitivity, Specificity, and Youden's Index for the various cut-off point obtained from ROC Curve analysis
Since behaviors are discrete indivisible units, this datum was rounded and a new cut-off point was established: 5 or more secure behaviors. This new classification methodology resulted in a sensitivity of 47.6% (CI 95% = [26.4; 69.7]) and a specificity of 90.9 % (CI 95% = [57.1; 99.5]).
As can be seen in Table 3, using 5 or more secure behaviors as a cut-off point for the secure attachment classification in the ADS scale results in 65.6% of children classified as secure. The correlation with attachment security as measured by AQS is enhanced slightly (rbp = .43, p = .01) and the association between the two classifications continues to be significant [X2 (1, N = 32) = 4.75, p = .03, V = .38]. In terms of the consistency between the secure and insecure attachment classifications in the two instruments, agreement results in 62.5% of cases (20).
Table 3. Cross Table ADS classification with a new cut-off point and secure - insecure AQS classification with a. 32 cut-off point (n = 32)
Discussion
The objective of this study was to generate evidence of convergent and discriminant validity for the ADS, as well as evidence of criterion validity. The study used data generated in earlier research with a group of 32 mothers and their 8 to 10-month old children.
First, adequate evidence of convergent validity for the ADS in its dichotomous classification mode (secure-insecure) was found, expressed in its positive and moderate relation with the security scores in the AQS, and in the higher score for children classified as secure in the AQS compared to those classified as insecure in the ADS. It is also important to note that the magnitude of the relationship found in this study is similar to the relationship between AQS scores and SSP classifications reported by Van IJzendoorn et al. (2004). These results support the use of the ADS in attachment research with children under 18 months, given that the AQS is the second most used instrument in attachment research in a variety of contexts (Van IJzendoorn et al., 2004), including Latin America.
No adequate evidence of discriminant validity with MBQS scores (Pederson & Moran, 1995) was found, as no significant differences resulted in sensitivity for mothers of children classified as secure and insecure by the ADS. These results are similar to those in Cárcamo et al. (2014), which used Ainsworth's Maternal Sensitivity Scale (1974), indicating that children's attachment as measured by ADS is not related to maternal behavior. One possible explanation could reside in the different contexts in which each instrument was used: in this research, attachment was assessed using ADS in a stress situation lasting approximately 15 minutes, while maternal sensitivity was assessed in an everyday interaction situation lasting approximately one hour. This possible explanation must be explored further in future research, controlling for methodological variables that may have influenced the results.
Concordance between ADS and AQS classifications was also analyzed, assuming for the latter a categorical classification of secure-insecure attachment. The results of the present study showed that these two tests differ significantly in their attachment security classifications of the dyads. Specifically, a tendency to the overrepresentation of secure attachment in the ADS shows was observed. Our analyses of sensitivity, specificity, positive predictive values, and negative predictive values reinforced this conclusion, showing that using ADS to identify cases of insecure attachment may pose a challenge. These results coincide with an earlier study using the SSP as a criterion (Cárcamo et al., 2014). It must be pointed out, however, that this earlier study used a trichotomic classification for ADS results (secure, insecure ambivalent, and insecure avoidant).
Finally, the results of the ROC curve analysis allowed proposing a new way to classify a child with secure attachment based on his or her ADS scores. The analysis of the present work suggests a secure attachment classification when the child displays five of the seven behaviors considered secure. This method of classification increased the scale's sensitivity from 33% to 47.6%, slightly increased its relationship with the AQS scores, and reduced the frequency of the classification of secure attachment, placing it at a value more in line with what has been reported in the international literature. However, the specificity of the scale decreased from 99% to 90.9%. Taken together, these results appear to show that, even while adjusting the methodology for classifying a child's attachment as secure, the evidence is insufficient to support the use of the ADS as a screening test for insecure-type attachment.
It is important to emphasize that aspects such as the size of the study sample as well as the criterion's psychometric properties are especially relevant to criterion validity results (Evers et al., 2013). Thus, although the reference criterion in this study is amply supported by the literature, the study's small sample size limits the accuracy of its results. Therefore, we recommended that these procedures be reproduced with a larger and more representative sample.
Similarly, the fact that the results of both tests are related while the classifications obtained from both instruments differ, may be the result of differences in the way in which the ADS and the AQS conceptualize attachment. This suggests a need for future studies to review and adjust the content of the ADS items in order to make them more compatible with other instruments such as the AQS. Along those lines, future research could identify which behaviors proposed in the ADS contribute most to the overrepresentation of secure attachment, and modify them.
Finally, the literature on the psychometric evaluation of psychological instruments maintains that while all evidence of validity is important for the assessment of tests, evidence of criterion validity is especially significant when the intention is to use measurement to make predictions (Evers et al., 2013). In that sense, studies like this one become more relevant, given the recent growth in the use of ADS as a screening test in population assessments.
Despite the limitations described above, this study is part of a broader effort to generate psychometric evidence for ADS, and it is the first to assess its sensitivity, specificity, and predictive values. Further study of these characteristics of the test is recommended, given the importance of the ADS for population assessments and for research on a topic as complex and of such growing interest as attachment.
Notas:
1 For the classification of the evidence of validity we referred to the European Federation of Psychologists Association - EFPA's Review Model for the Description and Evaluation of Psychological and Educational Tests (Evers, Hagemeister, Hostmaelingen, Lindley, Muñiz, & Sjöberg, 2013).
References
Ainsworth, M. D. S., Bell, S. M., & Stayton, D. J. (1974). Infant-mother attachment and social development: Socialization as a product of reciprocal responsiveness to signals. In M. P. M. Richards (Ed.), The integration of a child into a social world (pp. 99-135). London: Cambridge University Press.
Ainsworth, M., Blehar, M., Waters, E., & Wall, S. (1978). Patterns of attachment. A psychological study of the Strange Situation. New Jersey: Lawrence Erlbaum Associates.
Bárrig, P. (2004). Relation between children's attachment representations and secure base behavior (Unpublished master's thesis). Purdue University, Indiana, United States of America.
Bowlby, J. (1969/1982). Attachment and loss: Attachment. New York: Basic Books.
Bowlby, J. (1988). A secure base. New York: Basic Books.
Buitrón, V. (2011). Apego en hijos de madres adolescentes víctimas de abuso sexual en Lima Metropolitana (Unpublished license thesis). Pontificia Universidad Católica del Peru, Lima, Peru. Retrieved from http://tesis.pucp.edu.pe/repositorio/handle/123456789/625
Cárcamo, R., Van IJzendoorn, M., Vermeer, H., & Van der Veer, R. (2014). The validity of the Massie-Campbell Attachment During Stress Scale (ADS). Journal of Child and Family Studies, 23(5), 767-775. doi:10.1007/s10826-013-9728-z
Cassibba, R., Van IJzendoorn, M. H., & D'Odorico, L. (2000). Attachment and play in child centers: Reliability and validity of the attachment Q sort for mother and professional caregiver in Italy. International Journal Behavioral Development, 24(2), 241-255. doi :10.1080/016502500383377.
Cassidy (2016). The nature of the child's ties. In J. Cassidy & P.R. Shaver (Eds.), Handbook of attachment: Theory, research, and clinical applications. 3th ed. (pp. 3-24). New York: Guilford Press.
Cadman, T., Diamond, P. R., & Fearon, P. (2017). Reassessing the validity of the attachment Q-sort: An updated meta-analysis. Infant and Child Development. Advance online publication. doi: 10.1002/icd.2034
Clarke-Stewart, K., Goossens, F., & Allhusen, V. (2001). Measuring infant-mother attachment: Is the Strange Situation Enough? Social Development, 10(2), 143-169. doi:10.1111/1467-9507.00156
Conde, G., Nóblega, M., Fourment, K., & Nuñez del Prado, J. (2015, August). Secure Base Behavior in preschoolers: AQS comparison between home and park observations in Lima, Peru. Poster presented at the International Attachment Conference - IAC, New York, NY.
Dávila, D. (2013). Apego y sensitividad materna en madres y niñospreescolares del distrito de Los Olivos (Unpublished license thesis). Pontificia Universidad Católica del Peru, Lima, Peru. Retrieved from http://tesis.pucp.edu.pe/repositorio/handle/123456789/5010
Evers, A., Fernández, J. M., Hagemeister, M., Hagemeister, C., Hstmaelingen, A., Lindley, P., Sjöberg, A., & Bartram, D. (2013). Assessing the quality of tests: Revision of the EFPA review model. Psicothema, 25(3), 283-291. doi: 10.7334/psicothema2013.97
Evers, A., Hagemeister, C., Hostmaelingen, A., Lindley, P., Muñiz, J., & Sjöberg, A. (2013). EFPA Review model for the description and evaluation of psychological and educational tests. Test review form and notes for reviewers. Version 4.2, 6. http://www.efpa.eu/professional-development/assessment
Figueroa, F., Vergara, V., & Santelices, M.P. (2012). Enhancing early attachment: Design and pilot study of an intervention for primary health care dyads. Journal of Child Health Care, 16(17), 82-91. doi:10.1177/1367493512447533
Geenen, G., Corveleyn, J., & Koroleff, P. T. (2014). Vínculos protectores: Apego en padres e hijos en vulnerabilidad. Lima: Fondo Editorial de la Pontificia Universidad Católica del Peru.
Kotliarenco, M., Gomez, E., Muñoz, M., & Armijo, I. (2009). Evaluación pre-post del desarrollo psicomotor y el estilo de apego en usuarios de los Centros de Desarrollo Infantil Temprano. SUMMA Psicológica UST, 6(2), 89-104.
Jin, M.K., Jacobvitz, D., Hazen, N., & Jung, S. (2012). Maternal sensitivity and infant attachment security in Korea: Cross-cultural validation of the Strange Situation. Attachment & Human Development, 14(1), 33-44. doi:10.1080/14616734.2012.636656.
Lecannelier, F., Undurraga, V., Olivares, A. M., Rodriguez, J., Nuñez, J. C., Hoffmann, M., Goldstein, N., & Larraín, C. (2009). Estudio de eficacia sobre dos intervenciones basadas en el fomento del apego temprano en díadas madre-bebé provenientes de la región Metropolitana en Santiago de Chile. Revista Argentina de Clínica Psicológica, 18(2), 143-156.
Leigh, F., Vergara, V., & Santelices, M. (2012). Enhancing early attachment: Design and pilot study of an intervention for primary health care dyads. Journal of Child Health Care, 17(1), 82-91. doi:10.1177/1367493512447533
Massie, H., & Campbell, K. (1983). The Massie-Campbell Scale of Mother-Infant Attachment Indicators during Stress. In J. Call, E. Galenson & R. Tyson (Eds.), Frontiers of infant psychiatry (pp. 394-412). New York: Basic Books.
Massie, H., & Campbell, K. (1992). The Massie-Campbell Scale of mother-infant attachment indicators during stress. For use during the pediatric examination and other child-care situations. Berkeley: Instruction Guide.
Nóblega, M. (2012). Conducta de base segura y sensitividad en niños y madres del distrito de los Olivos (Doctoral thesis). Pontificia Universidad Catolica del Peru, Lima, Peru. Retrieved from http://tesis.pucp.edu.pe/repositorio//handle/123456789/4491
Nóblega, M., Bárrig, P., Conde, G., Nuñez del Prado, J., Carbonell, O. A., Altmann de Litvan, M... Bauer, M. (2016). Cuidado materno y seguridad del apego antes del primer año de vida. Universitas Psychologica, 15(1), 245-260. doi:10.11144/Javeriana.upsy15-1.cmsa
Nóblega, M., Fourment, K., Núñez del Prado, J., Conde, G., & Bahamonde, D. (2015, August). Maternal sensitivity and child attachment security at 3-5 years old: Differences between low and middle socioeconomic status at Lima, Peru. Poster presented at the International Attachment Conference - IAC, New York, NY
Pederson, D. R., & Moran, G. (1995). Appendix B. Maternal Behavior Q-set. Monographs of the Society for Research in Child Development, 60(2-3 Serial No. 244), 247-254. doi:10.2307/1166182
Pederson, D. R., Moran, G., & Bento, S. (1999). Maternal Behaviour Q-sort. Extraído de http://ir.lib.uwo.ca/psycholo-gypub/1
Posada, G., Carbonell, O., Alzate, G., & Plata, S. (2004). Through Colombian lenses: Ethnographic and conventional analyses of maternal care and their associations with secure base behavior. Developmental Psychology, 40(4), 508-518. doi:10.1037/0012-1649.40.4.508
Posada, G., Gao, Y., Wu, F., Posada, R., Tascon, M., Schöelmerich, A. Synnevaag, B. (1995). The secure-base phenomenon across cultures: Children's behavior, mothers' preferences, and experts' concepts. Monographs of the Society for Research in Child Development, 60(2/3), 27-48. doi: 10.2307/1166169
Posada, G., Jacobs, A., Carbonell, O., Alzate, G., Bustamante, M., & Arenas, A. (1999). Maternal care and attachment security in ordinary and emergency contexts. Developmental Psychology, 35(6), 1379-1388. doi:10.1037//0012-1649.35.6.1379
Posada, G., Jacobs, A., Richmond, M., Carbonell, O., Alzate, G., Bustamante, M., & Quiceno, J. (2002). Maternal care-giving and infant security in two cultures. Developmental Psychology, 38(1), 67-78. doi:10.1037//0012-1649.38.1.67
Posada, G., Kaloustian, G., Richmond, M., & Moreno, A. (2007). Maternal secure base support and preschoolers' secure base behavior in natural environments. Attachment & Human Development, 9(4), 393-411. doi:10.1080/14616730701712316
Posada, G., Lu, T., Trumbell, J., Kaloustian, G., Trudel, M., Plata, S. J... Lay, K. L. (2013). Is the secure base phenomenon evident here, there, and anywhere? A cross-cultural study of child behavior and experts' definitions. Child Development, 84(6), 1896-1905. doi:10.1111/cdev. 12084
Posada, G., Trumbell, J., Noblega, M., Plata, S., Peña, P., Carbonell, O. A., & Lu, T. (2016). Maternal sensitivity and child secure base use in early childhood: Studies in different cultural contexts. Child Development, 87(1), 297-311.
Solomon J., & George C. (2016). The measurement of attachment security and related constructs in infancy and early childhood. In J. Cassidy & P.R. Shaver (Eds.), Handbook of attachment: Theory, research, and clinical applications. 3th ed. (pp. 366-398). New York: Guilford Press.
Tryphonopoulos, P. D., Letourneau, N., & Ditommaso, E. (2014). Attachment and caregiver-infant interaction: A review of observational-assessment tools. Infant Mental Health Journal, 35(6), 642-656. doi:10.1002/imhj.21461
Ugarte, A. (2014). Conducta de base segura con el padre y representaciones de apego en niños preescolares (Unpublished license thesis). Pontificia Universidad Catolica del Peru, Lima, Peru. Retrieved from http://tesis.pucp.edu.pe/repositorio/handle/123456789/5599
Van Bakel, H., & Riksen-Walraven, J. (2004). AQS security scores: What do they represent? A study in construct validation. Infant Mental Health Journal, 25(3), 175-193. doi:10.1002/imhj.20001
Van IJzendoorn, M., Vereijken, C., Bakermans-Kranenburg, M., & Marianne Riksen-Walraven, J. (2004). Assessing attachment security with the Attachment Q Sort: Meta-analytic evidence for the validity of the observer AQS. Child Development, 75(4), 1188-1213. doi:10.1111/j.1467-8624.2004.00733.x
Van IJzendoorn, M., Bakermans-Kranenburg, M., & Sagi-Schwartz, A. (2006). Attachment across diverse sociocultural contexts: The limits of universality. In K. Rubin & O. Boon (Eds.), Parenting beliefs, behaviors and parent-child relations (pp. 107-142). New York: Psychology Press.
Vaughn, B., Coppola, G., Verissimo, M., Monteiro, L., Santos, A., Posada, G... Korth, B. (2007). The quality of maternal secure-base scripts predicts children's secure-base behavior at home in three sociocultural groups. International Journal of Behavioral Development, 31(1), 65-76. doi:10.1177/0165025407073574
Waters, E. (1995). Appendix A: The Attachment Q-set (version 3.0). Monographs of the Society for Research in ChildDevelopment, 60(2-3), 234-246. doi:10.1111/j.1540-5834.1995.tb00214.x