Artículo
|
|
Artículo Multidimensional Self-Regulation Battery for Reading Comprehension: Preliminary Evidence for Criterion ValidityBatería Multidimensional de Autorregulación para la Comprensión Lectora: evidencia preliminar de validez de criterio |
Adriana Satico Ferraz 1
Centro de Ensino Unificado de Brasília (CEUB), Brazil; Centro Universitário Belas Artes de São Paulo (FEBASP), Brazil; Universidade São Francisco (USF), Campinas, Brazil
0000-0002-9856-0094
adrianasatico.as@gmail.com
Acácia Aparecida Angeli dos Santos
Universidade de Brasília (UnB), Campinas, Brazil
0000-0002-8599-7465
Ana Paula Porto Noronha
Universidade São Francisco (USF), Campinas, Brazil
0000-0001-6821-0299
Leandro S. Almeida
Universidade do Minho (UMinho), Braga, Portugal
0000-0002-0651-7014
1 Contact information: Adriana Satico Ferraz, Centro Universitário de Brasília, Quadra SEPN 707/907 Asa Norte, 70790075 - Brasília, DF - Brasil, Phone: +55 61 39661474;
Authors' note:
(a) Paper derived from the doctoral thesis: Self-regulation for reading comprehension in elementary education II: proposal for assessment and intervention, defended in 2022 at Universidade São Francisco, Campinas - SP (Brazil).
(b) The database is available. You can access it via the link https://osf.io/p2exn/?view_only=442e340699b544d5b8b4349dc31f7242
(c) The authors declare that there are no competing interest related to the research topic. We thank
for the financial support: Process n° 2018/19897-4, Foundation for the
Promotion of Research of the State of São Paulo (FAPESP).
Recibido: marzo 14/2022
Concepto de evaluación: abril 4/2024
Aceptado: mayo 24/2024
Cómo citar [APA]: Satico Ferraz, A., Angeli dos Santos, A. A., Porto Noronha, A. P., & Almeida, L. S. (2024). Multidimensional Self-Regulation Battery for reading comprehension: Preliminary evidence for criterion validity. Acta Colombiana de Psicología, 27(2), 169-188. https://doi.org/10.14718/ACP.2024.27.2.10
Abstract
The purpose of this study was to investigate the validity based on relationships with other variables of the Self-Regulation for Reading Comprehension Multidimensional Battery (BAMA-RC). This study was developed quantitatively, with a cross-sectional and correlational design. Participants were 182 middle school students from Brazil. Nine BAMA-RC scales and a Cloze test were used to assess reading comprehension. The data were analyzed using statistics that evaluate relationships between variables (correlation, linear regression, chi-square test, Kruskal-Wallis and Mann-Whitney). The results identified preliminary evidence of criterion validity, confirming the hypotheses of relationships between the dimensions of self-regulation for learning and reading comprehension. These findings encourage new studies to extend the psychometric properties of the BAMA-RC.
Keywords: self-regulated learning, reading comprehension, motivation, metacognition, psychoeducational assessment.
Resumen
Este estudio tuvo como objetivo investigar la validez de la Batería Multidimensional de Autorregulación para la Comprensión Lectora (BAMA-RC) con base en la validez de criterio. Fue desarrollado de manera cuantitativa, con un diseño transversal y correlacional. Los participantes fueron 182 estudiantes de educación media de Brasil. Utilizamos nueve escalas BAMA-RC y una prueba de Cloze para evaluar la comprensión lectora. Los datos fueron analizados mediante estadísticas que evalúan las relaciones entre variables (correlación, regresión lineal, prueba de chi-cuadrado, Kruskal-Wallis y Mann-Whitney). Los resultados identificaron evidencia preliminar de validez de criterio, corroborando las hipótesis de relaciones entre las dimensiones de autorregulación para el aprendizaje y comprensión lectora. Estos hallazgos alientan nuevos estudios para ampliar las propiedades psicométricas del BAMA-RC.
Keywords: aprendizaje autorregulado, comprensión lectora, motivación, metacognición, evaluación psicoeducativa.
Self-regulated learning (SRL) manifests itself in the activation and maintenance of cognitive, metacognitive, motivational, affective, and behavioral aspects directed toward the planning and execution of school activities, as well as in the students' self-assessment of their general performance and specific contexts, as in the case of reading comprehension, the focus of this study (Berkeley & Larsen, 2018; Kusumarasdyati, 2023; Skibbea et al., 2019; Zimmerman & Risemberg, 1997). Self-regulated learning is associated with the development of literacy skills (phonological awareness and vocabulary) in the early school years (Skibbe et al., 2019). In addition, the review study conducted by Berkeley and Larsen (2018) showed that SRL interventions aimed at improving reading comprehension performance of students from different levels of primary education and with some learning difficulties showed a large effect size for its effectiveness when considering post-test and follow-up assessments.
Considering the motivational contribution of SRL, how students perceive and behave in everyday school life is partly explained by achievement goals, where the learning goal is driven by interests in intellectual growth; the performance approach goal is centered on achieving success and, consequently, on gaining people's approval; and the performance-avoidance goal is characterized by fear of exposure to impending failure (Urdan & Kaplan, 2020). Learning goal orientation predicts engagement in reading and reading comprehension (Richey et al., 2017). It represents less rigid beliefs about intelligence (also perceived as ability and aptitude) than the performance-avoidance goal. In a sample of middle school students, Santos et al. (2018) found that the learning goal was positively correlated with reading comprehension, while the performance-avoidance goal presented a negative directional relationship.
Therefore, self-efficacy, which generally refers to students' perception of their competence to carry out school activities (White & DiBenedetto, 2015), emerges as a predictor of reading comprehension performance (Lee & Jonson-Reid, 2016; Louick et al., 2016; Tarchi, 2016). Specifically, self-efficacy beliefs provide motivational support for students with reading difficulties (Lee & Jonson-Reid, 2016) and are related to components associated with reading comprehension, such as working memory, vocabulary, and phonological awareness (Carroll & Fox, 2017).
In turn, intrapersonal causal attributions in SRL allude to the students' reflection on the outcomes and situations of success and failure, expressed in the citation of causes to justify school performance (Graham, 2020; Vettori et al., 2018). These causes generally focus on the attribution of intelligence, effort, task difficulty, and luck, classified on the basis of three psychological dimensions that influence their prospective motivation: locus, stability, and controllability (Graham, 2020). In the context of reading comprehension, Berkeley and Larsen (2018) tested the effectiveness of an intervention aimed at guiding students to attribute their performance in this cognitive-linguistic skill to effort, in order to shift the focus from causes that are not controllable, as in the case of task difficulty, luck, and even intelligence.
Self-regulated learning also requires the intentional and appropriate use of reading strategies. In a review study, Ferraz and Santos (2021) found that intervention programs aimed at improving students' reading comprehension through SRL were based on activities that focused on the use of strategies to facilitate comprehension of textual materials. Furthermore, a review of self-regulation interventions by Berkeley and Larsen (2018), mentioned the encouragement of reading comprehension through strategies focused on formulating questions based on the title of a work and searching for the main idea of the text. Rodrigues et al. (2023) observed that strategies can vary according to the level of proficiency in reading comprehension. An excellent ability to understand texts makes it possible to use strategies such as selecting main ideas, reviewing the text, self-assessment after reading, forming mental images, and re-reading the most complex parts of the text.
Another component present in the SRL domain is time management, which is expressed in students' planning and organizing to meet deadlines (White & DiBenedetto, 2015; Zimmerman & Risemberg, 1997). A sign of problems in the self-regulation of time management is procrastination, characterized by the systematic behavior of completing tasks at the last moment (Ziegler & Opdenakker, 2018). Therefore, it is suggested that proper time management contributes to reading activities, both didactic and recreational. In addition, Garces-Bacsal and Yeo (2017) found that time management may be related to interest in reading, as students who considered themselves regular readers tended to find it easier to incorporate reading activities in their free time, compared to students who did not read frequently.
In addition to the appropriate use of reading strategies, self-regulation also involves self-monitoring the effectiveness of this effort during the action (White & DiBenedetto, 2015; Zimmerman & Risemberg, 1997). Self-monitoring includes cognitive (memory, attention, conscience etc.), and metacognitive (ability to reflect on one's thoughts) aspects that allow students to ask themselves how much of the reading they are comprehending (Joseph et al., 2016). This self-assessment consists of cognitive and metacognitive elements that allow the adaptation of strategies according to the specificities of the activity, configuring an aspect that enables the SRL process (Berkeley & Larsen, 2018).
However, adaptive and maladaptive self-reactions are also associated with self-assessment, with a focus on outcomes, whether positive or negative, (Schunk & Usher, 2013). Adaptive self-reactions are often observed in students who are classified as self-regulated and are expressed through behaviors that contribute to overcoming difficulties and maintaining good performance. In contrast, maladaptive self-reactions are characteristic of students with self-regulation problems and are expressed, for example, in demotivation, procrastination, and dropping out (Kitansas & Cleary, 2016; Schunk & Usher, 2013). Therefore, it is hypothesized that good performance in reading comprehension is related to adaptive self-reactions in self-regulated students. In contrast, students with self-regulation difficulties and low reading comprehension performance may exhibit maladaptive self-reactions to activities that require comprehension of text.
Self-regulation also includes students' autonomy and their ability to identify the physical spaces and instructional resources that best meet their reading needs and to modify and/or construct environments consistent with their requirements (Schunk & Usher, 2013; White & DiBenedetto, 2015; Zimmerman & Risemberg, 1997). Another aspect of self-regulation relates to the identification of people who serve as references in the performance of specific roles, in this case, good readers. In addition to the identification of these models, self-regulation is also validated in the way students seek help, which is selectively carried out in order to overcome some difficulties or acquire new knowledge (White & DiBenedetto, 2015; Zimmerman & Risemberg, 1997).
Self-regulation of the physical and social environments is influenced by social and economic variables. For example, visiting reading spaces in and out of school, is related to the encouragement and habit that parental figures provide to their children, with this type of stimulus also related to the socioeconomic issues of access to these spaces (Chen et al., 2018; Garces-Bacsal & Yeo, 2017; Strand & Schwippert, 2019). In turn, the role of the school in the self-regulation of the environmental and social dimensions for reading consists of the support that educational institutions provide to students to have contact with didactic and recreational textual materials (Klang et al., 2022). In this sense, Stack et al. (2015) found that the stimulus provided by schools to bring students closer to reading increases their confidence and self-efficacy as good readers, favoring the development of reading comprehension.
These meanings describe the existing associations within the SRL regarding the development and performance of students in reading comprehension. Therefore, the general aim of this study was to investigate the evidence of criterion validity for the nine scales of the Self-Regulation for Reading Comprehension Multidimensional Battery - BAMA-RC (Bateria Multidimensional da Autorregulação para a Compreensão de Leitura - BAMA-Leitura; Ferraz, 2022), as recommended by the American Educational Research Association (AERA) et al. (2014).
The BAMA-RC assesses the self-regulation of Brazilian middle school students in performing activities that require reading comprehension, taking into account achievement goals (Achievement Goals Scale for Reading Comprehension - EMR-CL), self-efficacy (Self-Efficacy Scale for Reading Comprehension - EA-CL), intrapersonal causal attributions (Causal Attribution Scale for Reading Comprehension - EAC-CL), and reading strategies (Reading Comprehension Strategies Scale - EE-CL), time management (Reading Time Organization Scale - EOT-L), self-monitoring (Self-monitoring Scale for Reading - Eau-L), adaptive and maladaptive self-reactions (Self-Reactions Scale for Reading Comprehension - Ear-CL), self-selection of the physical environment (Reading Environment Scale - Eam-L), and selective help-seeking (Social Environment Scale for Reading - EAS-L).
BAMA-RC has a study of its initial psychometric properties (e.g., content validity, evidence of validity based on internal structure, and reliability estimates) with a sample of Brazilian middle school students compatible with its target population reported in Ferraz (2022). However, it is necessary to extend its psychometric properties to contribute to its application in research and professional practice in psychology and education (AERA et al., 2014). The Cloze test, which assesses reading comprehension, was selected for its technical quality (psychometric properties) and because it is an instrument that is easy to use and correct and is low cost, characteristics that are in line with the concept of social technology (Ferraz et al., 2021).
This study aimed to analyze the evidence for the criterion validity of the BAMA-RC according to its relationship with the Cloze test. As hypotheses for the objective, it was expected to identify (a) a statistically significant correlations and predictive effects in a positive direction of the factors that make up the BAMA-RC scales and performance in the Cloze test. In this case, the exception to these hypotheses was made for the factors Meta Performance-Avoidance (EMR-CL), Procrastination (EOT-L), Maladaptive Self-Reactions (Ear-CL), Maladaptive Help-Seeking (EAS-L), and for causal attributions whose causes were classified as external, unstable and/or uncontrollable (EAC-CL); (b) an interaction effect between the high and low achievement groups in reading comprehension with each of the factors of the BAMA-RC scales (Berkeley & Larsen, 2018; Kusumarasdyati, 2023; Rodrigues et al., 2023; Skibbe et al., 2019; Zimmerman & Risemberg, 1997).
Method
Type of study
This study was developed quantitatively, with a cross-sectional and correlational design (Field, 2009).
Participants
The sample consisted of 182 middle school students from two public schools located in the state of São Paulo (n=109; 59.9%) and in Rio Grande do Sul (n=73; 40.1%), with a mean age of 13.66 years (SD = 1.27). Of these students, 91 were female (50.3%), and 39 had a history of repeating grades (21.8%), with the number of repeats ranging from one to three.
Most students were unable to report the education of their mother (n = 64; 36%) or father figure (n = 78; 42.9%). However, among those who knew this information, there was a higher proportion of mothers and fathers with high school education (n=30, 16.9%; n = 25, 14.2%, respectively) and a lower proportion of mothers with an incomplete higher education (n = 2; 1.1%) and fathers who had never attended school (n = 4; 2.3%).
In addition, the proportion of students who reported being "sometimes" or "always" encouraged to read by their parents/guardians was 44.3% (n = 78 for each alternative). Most of the students "always" had access to reading materials (n = 83; 47.2%), were "sometimes" interested in reading (n = 121; 69.1%), and "always" usually read in their free time (n = 98; 59.8%).
Instruments
Self-Regulation for Reading Comprehension Multidimensional Battery (BAMA-RC, Ferraz, 2022)
The construction of the BAMA-RC was based on the six conceptual dimensions of SRL proposed by Zimmerman and Risemberg (1997): motivation, method, time management, self-perceived behavior, physical environment, and social environment. This battery is designed to assess self-regulation for reading comprehension with middle school students as the target population.
The BAMA-RC has 116 items distributed in nine scales, referring to intrapersonal achievement goals, evaluated by the Achievement Goals Scale for Reading Comprehension (Escala Metas de Realização para a Compreensão de Leitura - EMR-CL), with 20 items divided into the factors Learning Goal, Performance-Approach Goal and Performance-Avoidance Goal; self-efficacy - Self-Efficacy Scale for Reading Comprehension (Escala Autoeficácia para Compreender a Leitura - EA-CL), with 21 items, unifactorial; reading strategies - Reading Comprehension Strategies Scale (Escala Estratégias para Compreender a Leitura - EE-CL), with 17 items, unifactorial; planning and managing deadlines - Reading Time Organization Scale (Escala Organização do Tempo para Leitura - EOT-L), 10 items, factors Time Organization and Procrastination; self-monitoring - Self-monitoring Scale for Reading (Escala Automonitoramento para Ler - EAu-L), nine items, unifactorial; self-reactions related to difficulties in reading comprehension - Self-Reactions Scale for Reading Comprehension (Automações para a Compreensão de Leitura - Ear-CL), nine items, Positive Self-Reactions factors and Negative Self-Reactions factors; self-selection of the physical environment and instructional resources - Reading Environment Scale (Escala Ambiente para a Leitura - Eam-L), 12 items, unifactorial; and selective help- seeking - Social Environment Scale for Reading (Escala Ambiente Social para Ler - EAS-L), 10 items, Adaptive Help-Seeking and Maladaptive Help-Seeking factors.
Responses are given on a 4-point Likert-type scale, with labels that vary according to the construct being assessed. As mentioned in the introduction of this article, the psychometric study of these BAMA-RC scales was carried out by Ferraz (2022) with a sample of 522 middle school students. As a result, the scales were found to have evidence of content validity (analysis of judges and target population), evidence of validity based on internal structure (plausibility of models generated by factor analysis techniques1), and reliability estimates (appropriate indices of internal consistency2).
Finally, the Causal Attribution Scale for Reading Comprehension (Escala Atribuições Causais para a Compreensão de Leitura - EAC-CL) has two situations, A and B, corresponding to good and poor reading comprehension. Each situation has four items related to the selection of the attributed cause - these are intelligence, effort, text difficulty, and luck, and related to the psychological dimensions underlying the cause selected in the first item, namely locus(internal or external), stability (stable or unstable) and controllability (controllable or not controllable). In the first investigation of the psychometric properties of the EAC-CL with a sample of middle school students, Ferraz et al. (2023) identified evidence of content validity (analysis of judges and target audience), evidence of validity based on internal structure (associations between attributed causes and the psychological dimensions locus, stability, and controllability through interaction effect verified by x2 test with p < .05), and evidence of criterion validity through the interaction effect with achievement goals and self-efficacy - interaction effects evaluated by the Kruskal-Wallis (H) and Mann-Whitney (U) test, whose p < .05.
Cloze Test "Things from Nature" (Santos, 2005)
This test has 32 items, where the sum of correct answers indicates the level of reading comprehension skills of middle school students. The 32 gaps in the text were determined through a fixed ratio, which implies that words are replaced by lines corresponding to their size, based on a standard interval of five words. The Cloze test was corrected, by giving 1 point for each correct answer and 0 points for an error, with the presentation of words identical to those that were suppressed and the absence of writing errors as the correction reference. The text "Things from Nature" has psychometric properties for its administration with students from 6th to 9th grade (evidence of validity based on the relationship with other variables, discriminant type, through the relationships between Cloze scores and verbal reasoning - r = .54; p < .001 (Lima, 2015) and presents evidence of validity based on the response process, from the identification of the types of errors made by students - x2 test value = 383.70; p < .001 (Cunha et al., 2020).
Data collection procedure
This paper report is part of a larger project approved by the Research Ethics Committee of the teaching institution to which it is attached (Authorization No. 3.263.350). It is a simple randomized sample. Eligibility criteria were based only on access of students without a diagnosis of neurodevelopmental and other disabilities or psychopathological disorders. Data collection took place in November 2021, referring to the period when students returned to face-to-face classes, formerly carried out remotely due to social distancing measures to contain the advance of the COVID-19 pandemic.
In order to carry out the research in the schools, permission was obtained from the school boards, the two educational institutions, the parents/guardians of the students (who signed the terms of the consent form), and the research participants (who signed the consent form). The instruments were administered in pencil and paper format in the following order: BAMA-RC, Cloze test, and identification questionnaire. Data collection took place in person and collectively during class time. Students took an average of 40 minutes to complete the scales. A total of 190 students participated in the study. However, eight protocols were excluded because the Cloze test was not completed.
Data analysis procedure
Analyses were performed using the SPSS, version 22.0, and ]ASP, version 0.16.2 (Goss-Sampson, 2020). First, the Shapiro-Wilk test was used to determine whether the data showed deviations from normality at p < .001 .Then, the results of the quartile analysis were used to create the two performance groups in the Cloze test: low-performance group, scores between 0 and 18 (G_Low) and high-performance group, scores between 19 and 29 (G_High). Situations A and B of the EAC-CL (BAMA-RC) with four response options regarding the attribution of the cause to intelligence, effort, text difficulty and luck were transformed into dummy variables. This categorization was carried out as follows: 1 for other causes and 2 for the focused cause. The calculation of factor scores was used in data analysis for the other BAMA-RC scales and the Cloze test.
Descriptive statistics (mean and standard deviation) were calculated for the BAMA-RC scales, based on a scale of 0 to 4 of the answer keys (except for the EAC-CL). For the two situations of the EAC-L, cross-reference tabulation was applied to verify the frequency and percentage between the psychological causes and dimensions attributed by the students. Descriptive statistics were also applied to determine the Cloze test score.
To examine the relationships between the BAMA-RC scales: EMR-CL, EA-CL, EE-CL, EOT-L, EAu-L, Ear-CL, Eam-L, and EAS-L and the Cloze test, Spearman's rank correlation (p) was used. The point-biserial correlation test was applied for the relationships between the EAC-CL and the reading comprehension instrument. The interpretation of the magnitude of correlations was based on values of .29 or less, qualified as weak; between .30 and .49 as moderate; and .50 or greater as strong (Goss-Sampson, 2020).
The Mann-Whitney test (U) was used to evaluate possible differences in performance on the EMR-CL, EA-CL, EE-CL, EOT-L, EAu-L, Ear-CL, Eam-L, and EAS-L according to the Cloze test performance groups (G_Low and G_High). Cohen (1992) framework was used to interpret the effect size of these comparisons, with r < .49 classified as small; r between .50 and .79 as moderate; and r > .80 as large. The comparison of the EAC-CL with G_Low and G_High in the Cloze test was performed using the Chi-square test (x¿). To identify the groups in which there were differences, with p < .05, we considered the adjusted residuals (AR) value equal to or above 2. The effect size of comparisons was verified using Phi (0) (Field, 2009).
To investigate the predictive potential of the BAMA-RC for the Cloze test, a linear regression analysis was performed using the entry method. In this analysis, the bootstrap method (1000 samples) was used to generate the standard error and confidence interval values (95% CI). The autocorrelation between the residuals of the tested models was verified using the Durbin-Watson test, where values between 1 and 3 are considered adequate (Field, 2009). For the EAC-CL, it was decided to test a prediction model of performance on the Cloze test (dependent variables) for each item of the scale, taking into account Situations A and B (independent variables).
The internal consistencies of the BAMA-RC tests that have an interval scale (EMR-CL, EA-CL, EE-CL, EOT-L, EAu-L, Ear-CL, Eam-L, and EAS-L) and the Cloze test were analyzed using McDonald's omega (total ω).
Results
First, the descriptive statistics of the BAMA-RC scales and the Cloze test were verified. The results are shown in Table 1 and Table 2. Table 1 presents the internal consistency for the BAMA-RC scales (ω total values).
Table 1. Descriptive Statistics and Internal Consistency of the BAMA-RC Scales and the Cloze Test
Table 2. Descriptive Statistics: Causal Attribution Scale for Reading Comprehension - Situations A and B
In the process of investigating the relationships between the BAMA-RC scales and reading comprehension, Table 3 shows that among the statistically significant correlations, there was a positive association of moderate magnitude between the Self-Efficacy (EA-CL) and the Cloze test. The other correlations had weak magnitudes and positive directions, involving the Cloze and the Self-Selection of the Physical Environment (Eam-L),
Reading Strategies (EE-CL), Self-Monitoring (EAu-L), Time Organization (EOT-L), Learning Goal (EMR-CL), and Positive Self-Reactions (Ear-CL); and negative directions between the Cloze and the Performance-Avoidance Goal (EMR-CL) and Maladaptive Help-Seeking (EAS-L).
In the Causal Attributions of the BAMA-RC, Table 3 shows that there was only one statistically significant negative direction and weak magnitude between the attribution of luck (Situation A, successful in reading comprehension) and the Cloze test. The same was true for Situation B of the Causal Attributions (unsuccessful reading comprehension). For this situation, too, correlations of low magnitude were observed between the Cloze test and the attribution of effort (positive direction) and the psychological stability dimension (negative direction).
Table 3. Correlations between the BAMA-RC Scales and the Cloze Test
Note. Values in bold indicate statistically significant comparisons, ***p < .001; **p <.01; *p < .05.
In order to further investigate the associations between the BAMA-RC scales and reading comprehension, Table 4 shows the result of the comparisons of the EMR-CL, EA-CL, EE-CL, EOT-L, EAu-L, Ear -CL, Eam-L, and EAS-L for low and high performance in the Cloze test (G_Low and G_High). Looking at the presentation of statistical significance, in the EMR-CL, there was a higher mean rank value for G High than in G Low in the Learning Goal, with the opposite verified for the Performance Avoidance Goal (both with r = .17). The G_High students also obtained higher mean ranks compared to the G_Low in the Self-Efficacy (EA-CL; r = .22), Reading Strategies (EE-CL; r = .15), Time Organization (EOT-L; r=.16), Self-Monitoring (EAu-L; r = .15) and in the Self-Selection of the Physical Environment (Eam-L; r = .19). The r values indicate that these comparisons had small effect sizes.
Table 4. Comparison of Low and High-Performance Groups in Cloze Test and EMR-CL, EA-CL EE-CL EOT-L EAu-L Ear-CL Eam-L and EAS-I
Note. Values in bold indicate statistically significant comparisons, ***p < .001; **p <.01; *p < .05.
For the EAC-CL of BAMA-RC, Situation A, in which the results of the X2 test are shown in Table 5, statistical significance was found in the comparison involving the low and high-performance groups in the Cloze test and the psychological dimension of causal attributions locus, X2 = 5.215, df=1, p < .05, ø = .17. The other comparisons of Situation Awerenotstatisticallysignificant,regardingthe four causal attributions, X2=5.321, df = 3, p=.15, ø = .17, the psychological dimension stability, X2= .035, df=1, p = .85, ø = .01; and controllability, X2= .112, df=1, p = .74, ø = .02.
For Situation B of the EAC-CL, statistically significant differences were found for G_Low and G_High in the Cloze with the causal attributions of effort and luck, X2 = 11.051, df = 3, p = .05, ø = 0.25; and the psychological dimensions of stability, X2= 7.665, df = 3, p = .01, ø = .20; and controllability, X2= 4.325, df=1, p < .05, ø = .15. There was no statistically significant difference in G_Low and G_High of the Cloze for the locus, X2 = 0.790, df = 1, p = .37, f = .07.
Table 5. Comparison of Low and High-Performance Groups in Cloze and EAC-CL
Legend. OF = Observed Frequency; RF = Relative Frequency; AR = Adjusted Residues
Note. Values in bold indicate statistically significant comparisons, ***p < .001; **p <.01; *p < .05.
Finally, we examined the predictive potential of the BAMA-RC scales for students' reading comprehension performance on the Cloze test. Table 6 shows that the EMR-CL explained 11% of the variance of the Cloze test, with a 1-point increase in the Learning Goal reflected as an increase of .31-point increase in reading comprehension performance. In contrast, a 1-point increase in the Performance-Approach Goal and Performance-Avoidance Goal was reflected as a decrease of .17 and .20 points, respectively, in this cognitive-linguistic ability. Also, regarding motivation as assessed by the BAMA-RC, the Cloze had 11% of its variance explained by the Self-Efficacy (EA-CL). A one-point increase in self-efficacy increased reading comprehension performance by .34 points.
Reading Strategies (EE-CL) explained 7% of the variance in the Cloze, with each point of reading strategies use increasing performance on the test by .27 points. The Cloze test had 3% of its variance explained by the EOT-L - the addition of 1 point in Time Organization predicted an increase of .18 points in reading comprehension. Self-Monitoring (EAu-L) accounted for 6% of the variance explained in the Cloze test. It was found that an increase of 1 point in self-monitoring reflected an increase of .07 points in reading comprehension performance. The Cloze test had 10% of its variance explained by the Eam-L. A 1 point increase in self-selection of the physical environment and instructional materials for reading was reflected in a .07-point increase in student performance on the test. In turn, the EAS-L explained 2% of the variance of the Cloze test. A 1-point increase in Maladaptive Help-Seeking predicted a .15-point decrease in reading comprehension.
Table 6. EMR-CL, E-CL, E-CL, ET-L, EAu-L, Ear-CL, Eam-L and EAS-L Predicting Cloze Test Performance
Legend. SE = Standard Error; D-W = Durbin-Watson test
Note1. Standard Error and Confidence Interval (95%) values were based on 1000 bootstrap samples.
Note2. Values in bold indicate statistically significant p, with ***p < .001; **p <.01; *p < .05.
Table 7 shows the results of the EAC-CL prediction analysis for Cloze test performance. In Situation A (success in reading comprehension), only the causal attribution of luck explained 4% of the variance in the Cloze test. This attribution predicted a .21-point decrease in performance on the reading comprehension test. Regarding Situation B (unsuccessful reading comprehension), the causal attributions of effort and luck each explained 5% of the variance on the Cloze test. The attributions of effort increased reading comprehension. by .24 points, and the attribution of luck implied a decrease of .23 points in this cognitive-linguistic skill. Also, considering Situation B of the EAC-CL, the Psychological Stability dimension explained 6% of the variance in the Cloze test. In this case, the belief that being unsuccessful in reading comprehension is stable predicted a decrease of .26 points in performance in this test.
Table 7. EAC-CL Predicting Cloze Test Performance
Legend. Dim Psico = Psychological dimension; SE = Stardard Error; D-W = Durbin-Watson test
Note1. Durbin-Watson test, Standard Error, and Confidence Interval (95%) values were based on 1000 bootstrap samples.
Note2. Values in bold indicate statistically significant ¡3, with ***p < .001; **p <.01; *p < .05.
Discussion
The results of this study provide indications that the scores of the BAMA-RC scales present preliminary evidence of criterion validity with reading comprehension (AERA et al., 2014; Kusumarasdyati, 2023; Rodrigues et al., 2023). These findings support the initial hypotheses regarding the relationships between reading comprehension and the constructs assessed by BAMA-Reading, that comprise the dimensions of SLR (Berkeley & Larsen, 2018; Kusumarasdyati, 2023; Rodrigues et al., 2023; Skibbea et al., 2019; Zimmerman & Risemberg, 1997).
The Achievement Goals Scale for Reading Comprehension (EMR-CL), which assesses achievement goals in the context of reading comprehension, was able to capture the existing associations between the learning goal and the performance-avoidance goal with the performance of middle school students in the Cloze test "Things from Nature," as also verified by Santos et al. (2018). The predominance of goal-oriented learning was found in the group of students with better performance in reading comprehension, with this achievement goal also being a predictor of this cognitive-linguistic ability. From this perspective, the characteristics of the motivational profile of the learning goal, characterized by curiosity, proactivity, creativity, and interest in complex and challenging activities, are in line with aspects commonly associated with reading comprehension proficiency, such as motivational quality (Richey et al., 2017; Santos et al., 2018; Urdan & Kaplan, 2020) and the efficient mobilization of basic cognitive processes (e.g., memory, attention), which favor the elaboration of inferences and the construction of deeper levels of textual interpretation (Berkeley & Larsen, 2018; Kintsch & Rawson, 2013).
In terms of prediction, the EMR-CL Performance-Approach Goal and Performance-Avoidance Goal had a negative effect on reading comprehension. The latter goal also prevailed in the group with the lower performance in the Cloze test. Both performance goals, take into account students' emphasis on performance, which limits learning to a superficial level (Urdan & Kaplan, 2020). Therefore, it is possible that these motivational orientations do not contribute to the development of higher levels of proficiency in reading comprehension. On the one hand, this assumption is based on the characteristics of students guided by the performance-avoidance goal, who tend to avoid challenging tasks due to the belief that this type of activity may increase the chances of failure. On the other hand, the performance-approach goal orientation may contribute to students not perceiving reading comprehension tasks and even reading habits as a way to demonstrate their success, especially in environments where these practices are not cultivated and/or valued (Urdan & Kaplan, 2020; Santos et al., 2018).
Subsequently, through Self-Efficacy (EA-CL), the existence of a relationship between self-efficacy and the correct answers obtained in the Cloze test was verified at the correlational and predictive levels, as well as the presentation of a higher mean in the group with better performance in reading comprehension. This result is similar to those of the studies by Lee and Jonson-Reid (2016), Louick et al. (2016), and Tarchi (2016), which portray the contribution of self-efficacy beliefs to reading comprehension proficiency.
Also, taking into account the contribution of motivation assessed by the BAMA-RC, the results obtained with the Causal Attributions (EAC-CL), both in Situation A and in Situation B (being successful and unsuccessful in reading comprehension), indicated the negative contribution of the attribution of the cause of luck to performance in the Cloze test. This result indicates that underlying the attribution of luck is the perception of a lack of merit in successful situations and a lack of accountability for failure in the context of failure (Graham, 2020), and that these beliefs do not contribute to good performance in reading comprehension (Berkeley & Larsen, 2018). The opposite result was found for effort, which led to incorrect responses in the Cloze test. This type of causal attribution is widely used in feedback-based interventions that seek to deconstruct students' maladaptive performance beliefs that focus on causes beyond their control, such as intelligence and task difficulty (Berkeley & Larsen, 2018; Ferraz & Santos, 2021).
Regarding the psychological dimensions inherent to intrapersonal causal attributions, it was found that the indication of the stability of the cause was detrimental to reading comprehension, with this dimension being one of the determining aspects of students' involvement in performing similar activities in the short and medium-term (Graham, 2020). Furthermore, the locus psychological dimension for Situation A of the EAC-CL (success in reading comprehension) had a comparable effect in the Cloze test performance groups. This finding is partially corroborated by the study of Vettori et al. (2018), in which internal locus attributions for success and failure were associated with reading ability.
The BAMA-RC scales that focus on the evaluation of the procedures adopted in reading comprehension, namely, the Reading Strategies (EE-CL), Self-Monitoring (EAu-L), the Time Organization (EOT-L), were associated with good performance on the Cloze test. These findings confirm what has been demonstrated in studies, indicating a broad repertoire of strategies applied in a self-regulated manner, plus the ability to manage time and self-monitor the effectiveness and pertinence of these procedures as favorable to the comprehension of the content read (Berkeley & Larsen, 2018; Ferraz & Santos, 2021; Joseph et al., 2016; Rodrigues et al., 2023).
In the adequate scope of SRL, the Positive Self-Reactions (Ear-CL) were related to the performance in the Cloze test. This result is an indication that through difficulties in reading comprehension (situation proposed in the Ear-CL), those students who experience positive effects related to the feeling of self-satisfaction seek ways to surmount them, thus increasing the chances of overcoming adversity (Kitansas & Cleary, 2016; Schunk & Usher, 2013). However, there were no statistically significant associations between the Negative Self-Reactions (Ear-CL) and the Cloze scores, which limits considerations of the negative effects of self-dissatisfaction when students face reading comprehension problems. Therefore, future studies using the BAMA-RC should investigate the role of self-reactions arising from reading comprehension difficulties, assessed by the Ear-L, and should consider their antecedents expressed by intrapersonal causal attributions, as well as the role of motivation and behaviors involving the metacognitive aspects of self-regulation, such as self-monitoring (White & DiBenedetto, 2015).
In the analysis of the Self-Selection of the Physical Environment (Eam-L), it was found that the self-selection and structuring of the physical environment and instructional resources are related to reading comprehension, in line with the indications of White and DiBenedetto (2015) and Zimmerman and Risemberg (1997), with the students' perception of the aspects that contribute more or less to reading comprehension, taking into account the places frequented and the means of accessing textual materials. In light of this, it is recommended that the interpretation of the Eam-L for the psychological assessment process be based on an additional survey that considers the possibilities of the student's contact with reading, taking into account the home, community, and school environments (Klang et al., 2022; Stack et al., 2015; Strand & Schwippert, 2019).
In the EAS-L, the research results with the Maladaptive Help-Seeking showed the potential impairment of the lack of self-regulation in seeking selective help on reading comprehension performance. Since the items in this factor focus on the teacher as a figure who can clarify doubts about aspects not understood in the reading, it is speculated about the quality of relationships established in the classroom and how students evaluate the ability of teachers to help them (Graham, 2020; Schunk & Usher, 2013; Stack et al., 2015; Strand & Schwippert, 2019). Another point to be discussed is the lack of association between Adaptive Help-Seeking (EAS-L) and Cloze test scores, since sharing the information provided by social interactions is one of the aspects that contributes to the development of reading skills and comprehension, as well as learning motivation (Ministério da Educação, 2017; Stack et al., 2015; Strand & Schwippert, 2019).
As the sample investigated in this study was restricted to two schools located in the Southeast and South regions of Brazil, which at the time of data collection had a reduced number of students per class due to the COVID-19 pandemic. It is suggested that future studies with the BAMA-RC investigate whether the results obtained here are maintained with the expansion of the sample and the inclusion of a larger number of educational institutions, covering the other Brazilian regions. This proposal focuses mainly on the results that did not reach statistical significance in the present study, in the process of investigating the relationships between reading comprehension and procrastination, maladaptive self-reactions, selective help-seeking, and causal attributions in situations of success and failure in this cognitive-linguistic skill.
Although it is expected that middle school students have an established reading ability, it is necessary to consider possible losses in reading comprehension with the implementation of Emergency Remote Teaching/ Hybrid Teaching during 2020 and 2021, especially for students who reported not practicing recreational reading (40.2% of the sample) and those who did not have access to reading materials at home (9.7% of the students). Another point to consider is the w sum values that were below .70 for the Meta Performance-Avoidance (EMR-CL), Negative Self-Reactions (Ear-CL), and Maladaptive Help-Seeking (EAS-L) factors (see Table 1), as this may have compromised the accuracy estimates of these scales. Therefore, results achieved with scales that include these factors should be interpreted cautiously. Investing in new BAMA-RC applications in samples of middle school students is also needed to assess their reliability estimates and potential impact on validity evidence.
Another limitation of this study is the use of a single instrument to assess reading comprehension. It is recognized that the Cloze test "Things from Nature" is suitable for assessing students' grammatical and textual skills, both from the point of view of its psychometric qualities (Cunha et al., 2020; Lima, 2015) and its application in the previously cited study by Santos et al. (2018), which addressed reading comprehension and motivation from the perspective of achievement goals. However, it is considered that this test limits the assessment of the reading comprehension to a restricted set of words, sometimes randomly assigned, resulting from the elaboration by a fixed ratio procedure. In addition, the literal correction method applied in this study eliminates the bias of the examiner's subjectivity. However, it does not accept the presentation of synonymous words or words with spelling errors. In the latter case, the Cloze test score also depends on writing ability. Therefore, these aspects may disregard part of the sample of assiduous readers (59.8% of students reported reading in their leisure time), who are more likely to respond to the Cloze test with synonymous words because they have a broader vocabulary.
As an alternative for assessing reading comprehension it is recommended to use the Cloze technique, which is based on other procedures, such as rational deletion, as it allows a specific composition of words and also the use of the system of displaying dotted lines instead of a single dash, to provide clues about the number of letters of the words omitted from the text. It is also appropriate to use tests that assess reading comprehension through other stimuli and a broader assessment of components related to this cognitive-linguistic skill, such as word recognition, writing, metatextual awareness, and phonological awareness.
Finally, despite not being within the scope of this study, it is necessary to take into account the peculiarities that characterize the sample studied here, regarding the reading habits of the students and their relationship with the level of education and parental incentives for reading. The consideration of these variables should guide the construction of explanatory models closer to the reality to be investigated, concerning the contribution of the dimensions of self-regulation evaluated by the BAMA-RC scales to the reading comprehension ability of middle school students.
Notas
1 EMR-CL: x2/df = 2.20; RMSEA = .05; CFI = .94; TLI = .93; EA-CL = x2/df = 2.15; RMSEA = .05; CFI and TLI = .96; EE-CL: x2/df = 3.65; RMSEA = .07; CFI = .91; TLI = .90; EOT-L: x2/df = 2.61; RMSEA = .06; CFI = .97; TLI = .95; EAu-L: x2/df = 5.46; RMSEA = .09; CFI = .95; TLI = .93; Ear-CL: x2/df = 2.65; RMSEA = .06; CFI = .98; TLI = .97; Eam-L: x2/df = 9.56; RMSEA = .13; CFI = .82; TLI = .77; EAS-L: x2/df = 5.91; RMSEA = .10; CFI = .94; TLI = .89 -values in bold indicate the indices that conferred plausibility to the tested models.
2 Composite Reliability values (CR). EMR-CL: Factor Learning Goal, CR = .83; Factor Performance-Approach Goal, CR = .81; Factor Performance-Avoidance Goal, CR = .71; EA-CL (unifactorial), CR = .91; EE-CL (unifactorial), CR = .88; EOT-L, Factors Procrastination and Time Organization, CR = .75; EAu-L (unifactorial), CR = .86; Ear-CL, Factor Positive Self-Reaction, CR = .82, Factor Negative Self-Reaction, CR = .71; Eam-L (unifactorial), CR = .87; EAS-L, Factor Maladaptative Help-Seeking, CR = .71, Factor Adaptative Help-Seeking, CR = .83.
References
AAmerican Educational Research Association, American Psychological Association, & National Council on Measurement in Education (2014). Standards for educational and psychological testing. American Educational Research Association.
Berkeley, S., & Larsen, A. (2018). Fostering Self-Regulation of Students with Learning Disabilities: Insights from 30 Years of Reading Comprehension Intervention Research. Learning Disabilities Research & Practice, 33(2), 75-86. https://doi.org/10.1111/ldrp.12165
Carroll, J. M., & Fox, A. C. (2017). Reading self-efficacy predicts word reading but not comprehension in both girls and boys. Frontiers in Psychology, 7, Article 2056. https://doi.org/10.3389/fpsyg.2016.02056
Chen, Q., Kong, Y., Gao, W., & Mo, L. (2018). Effects of Socioeconomic Status, Parent-Child Relationship, and Learning Motivation on Reading Ability. Frontiers in Psychology, 9, Article 1297. https://doi.org/10.3389/fpsyg.2018.01297
Cohen, J. (1992). Quantitative methods in psychology: A power primer. Psychological Bulletin, 112(1), 155-159. https://psycnet.apa.org/doi/10.1037/0033-2909.112.1.155
Cunha, N. D. B., Lima, T. H. D., Santos, A. A. A. D., & Oliveira, K. L. D. (2020). Teste de Cloze: Evidência de validade por processo de resposta. Psicologia Escolare Educacional, 24, Article e191537. http://dx.doi.org/10.1590/2175-35392020191537
Ferraz, A. S. (2022). Autorregulação para a compreensão de leitura no ensino fundamental II: proposta de avaliação e intervenção. [Doctoral Thesis in Psychology]. Universidade São Francisco. https://www.usf.edu.br/publica-coes/teses.vm
Ferraz, A. S., Cunha, N. de B, & Santos, A. A. A. (2021). Interferência da habilidade de escrita no resultado do teste de Cloze por razão fixa Impact of writing skills on the fixed-ratio Cloze test scores. Revista Iberoamericana de Psicología, 14(3), 47-55. https://doi.org/10.33881/2027-1786.rip.14305
Ferraz, A. S., & Santos. A. A. A. (2021). Self-regulation for learning intervention in reading comprehension: an integrative review. Estudos de Psicologia (Campinas), 38, Article e190179. https://doi.org/10.1590/1982-0275202138e190179
Ferraz, A. S., Santos. A. A. A., Noronha, A. P. P, & Almeida, L. S. (2023). Propiedades psicométricas de la Escala de Atribuciones Causales para la Comprensión Lectora. Avances en Psicología Latinoamericana, 41(1), 1-22. https://doi.org/10.12804/revistas.urosario.edu.co/apl/a.11267
Field, A. (2009). Descobrindo a estatística usando o SPSS. Artmed.
Garces-Bacsal, R. M., & Yeo, S. D. (2017). Why and What They Read When They Don't Have to: Factors Influencing the Recreational Reading Habits of Gifted Students in Singapore. Journal for the Education of the Gifted, 40(3), 247-265. https://doi.org/10.1177/0162353217717035
Goss-Sampson, M. A. (2020). Statistical Analysis in JASP 0.16.2: A Guide for Students. https://jasp-stats.org/jasp-materials/
Graham, S. (2020). An attributional theory of motivation. Contemporary Educational Psychology, 61, Article 101861. https://doi.org/10.1016/j.cedpsych.2020.101861
Klang, N., Asman, J., Mattsson, M., Nilholm, C., & Folkeryd, J. W. (2022). Intervention combining cooperative learning and instruction in reading comprehension strategies in heterogeneous classrooms. Nordic Journal of Literacy Research, 8(1), 44-64. http://dx.doi.org/10.23865/njlr.v8.2740
Joseph, L. M., Alber-Morgan, S., Cullen, J., & Rouse, C. (2016). The Effects of Self-Questioning on Reading Comprehension: A Literature Review. Reading & Writing Quarterly, 32(2), 152-173. https://doi.org/10.1080/10573569.2014.891449
Kintsch, W., & Rawson, K. A. (2013). Compreensão. In M. J. Snowling, & C. Hulme (Orgs.), A ciência da leitura (pp. 227-244). Editora Penso.
Kitansas, A., & Cleary, T. J. (2016). The developmental of self-regulated learning during secondary school years: A social cognitive instructional perspective. In K. R. Wentzel, & D. B. Miele (Eds.), Handbook of motivation at school (pp. 169-187). Routledge Taylor & Francis Group.
Kusumarasdyati, K. (2023). Reading comprehension in EFL: an overview. Proceedings of the international Joint Conference on Arts and Humanities 2022 (IJCAH 2022), 782-791. https://doi.org/10.2991/978-2-38476-008-4_84
Lee, Y. S., & Jonson-Reid, M. (2016). The Role of Self-Efficacy in Reading Achievement of Young Children in Urban Schools. Child and Adolescent Social Work Journal, 33(1), 79-89. https://doi.org/10.1007/s10560-015-0404-6
Lima, T. H. (2015). Compreensão de leitura em alunos do ensino fundamental ii: o teste de Cloze como alternativa de avaliação. [Doctoral Thesis in Psychology]. Universidade São Francisco. https://connect.itf.edu.br/galeria/getImage/427/666024783450523.pdf
Louick, R., Leider, C. M., Daley, S. G., Proctor, C. P, & Gardner, G. L. (2016). Motivation for reading among struggling middle school readers: A mixed methods study. Learning and Individual Differences, 49, 260-269. https://doi.org/10.1016/jJindif.2016.06.027
Ministério da Educação, Brasil (2017). Base Nacional Comum Curricular. http://basenacionalcomum.mec.gov.br/ images/BNCC_EI_EF_110518_versaofinal_site.pdf
Richey, J. E., Bernacki, M. L., Belenky, D. M., & Nokes-Malach, T. J. (2017). Comparing Class- and Task-Level Measures of Achievement Goals. The Journal of Experimental Education, 86(4), 560-578. https://doi.org/10.1080/00220973.2017.1386155
Rodrigues, B., Cadime, I., & Ribeiro, I. (2023). Cognitive and metacognitive strategy use in poor comprehenders: an exploratory study. Australian Journal of Learning Difficulties, 28(2), 139-153. https://doi.org/10.1080/19404158.2023.2287611
Santos, A. A. A. (2005). Cloze Test "Things from Nature". Unpublished Technical Report. Universidade São Francisco.
Santos, A. A. A., Moraes, M. S. D., & Lima, T. H. (2018). Compreensão de leitura e motivação para aprendizagem de alunos do ensino fundamental. Psicologia Escolare Educacional, 22(1), 93-101. https://doi.org/10.1590/2175-35392018012208
Schunk, D. H., & Usher, E. L. (2013). Barry J. Zimmerman's theory of self-regultated learning. Em H., Bembenutty, A., Kitsantas, & T. J., Cleary (Eds.). Applications of self-regulated learning across diverse disciplines: A tribute to Barry J. Zimmerman (pp. 1-28). Information Age Publishing.
Skibbe, L. E., Montroy, J. J., Bowles, R. P., & Morrison, F. J. (2019). Self-regulation and the development of literacy and language achievement from preschool through second grade. Early Childhood Research Quarterly, 46, 240-251. https://doi.org/10.1016Zj.ecresq.2018.02.005
Stack, M. H., Moorefield-Lang, H., & Barksdale, M. A. (2015). Able: An Instrument for Assessing Elementary Students' Perceptions of Access to Books, Beliefs, and Literacy Environment. Reading Psychology, 36(6), 499-518. https://doi.org/10.1080/02702711.2014.893275
Strand, O., & Schwippert, K. (2019). The impact of home language and home resources on reading achievement in ten-year-olds in Norway, PIRLS 2016. Nordic Journal of Literacy Research, 5(1), 1-17. http://dx.doi.org/10.23865/njlr.v5.1260
Tarchi, C. (2016). Comprehending Expository Texts: The Role of Cognitive and Motivational Factors. Reading Psychology, 38(2), 154-181. https://doi.org/10.1080/02702711.2016.1245229
Urdan, T., & Kaplan, A. (2020). The origins, evolution, and future directions of achievement goal theory. Contemporary Educational Psychology, 61, Article 101862. https://doi.org/10.1016/j.cedpsych.2020.101862
Vettori, G., Vezzani, C., Bigozzi, L., & Pinto, G. (2018). The mediating role of conceptions of learning in the relationship between metacognitive skills/strategies and academic outcomes among middle-school students. Frontiers in Psychology, 9, Article 1985. https://doi.org/10.3389/fpsyg.2018.01985
White, M. C., & DiBenedetto, M. K. (2015). Self-regulation and the Common Core: Application to Ela Standards. Routledge.
Ziegler, N., & Opdenakker, M. C. (2018). The development of academic procrastination in first-year secondary education students: The link with metacognitive self-regulation, self-efficacy, and effort regulation. Learning and Individual Differences, 64, 71-82. https://doi.org/10.1016/jJindif.2018.04.009
Zimmerman, B. J., & Risemberg, R. (1997). Self-Regulatory Dimensions of Academic Learning and Motivation. In G. D. Phye (Eds), Handbook of Academic Learning: Construction of knowledge (pp. 105-125). Academic Press. https://doi.org/10.1016/b978-012554255-5/50005-3