Psychometric properties of a Brazilian adaptation of the teacher observation of classroom adaptation- revised scale for children's behavior.

Abstract

The aim of this study was to describe the process of psychometric analysis of the Teacher Observation of Classroom Adaptation- Revised Scale (TOCA-R) for its use in Brazilian schools and to evaluate its validity and reliability. To evalu- ate the “Elos Program”, which is the Brazilian culturally adapted version of the North American Program “Good Behavior Game”, the TOCA-R was used. The researchers adapted the instrument in 2014, consisting of 33 items in a three-point ordinal response scale. A longitudinal quasi-experimental design with a single group was used. Participants were children aged 6 to 10 years evaluated by their teachers, before (n = 1448) and after (n = 673) the implementation of the Elos Program in 2014. The study involved initially four schools, 68 classes and their respective teachers. The analytical procedures were exploratory factorial analysis, confirmatory factorial analysis, longitudinal invariance analysis and reliability analysis by precision coefficients. The results of the exploratory factorial analysis showed an acceptable adjustment of five factors with 25 items, with a total explained variance of 60% and mean residual error of 0.02. The confirmatory factorial analysis expressed a satisfactory fit of the model (χ2 = 961, df = 265, RMSEA = .078, 95% IC [.07, .08], and CFI = 0.9). A configu- rational, metric and scalar invariance of latent structure was identified, which, together with the amplitude of variation of the precision coefficients between the instrument dimensions (α = .78, .92; ω = .76, .92), demonstrate evidence of validity and reliability for using the TOCA-R in evaluating the Elos Program in Brazilian schools.

Keywords:

Psychometric analysis Child mental health Preventive programs Cross-cultural validation Teacher observation of classroom adaptation-revised

References

American Institutes for Research [AIR]. (2010). TOCA-R and other scales as used in the Good Behavior Game Professional Development Models Trial. Washington, D.C.: AIR.

Bayer, J., Hiscock, H., Scalzo, K., Mathers, M., McDonald, M., Morris, A., & Wake, M. (2009). Systematic review of preventive interventions for children’s mental health: what would work in Australian contexts? Australian New Zealand Journal of Psychiatry, 43(8), 695-710. doi: https://doi.org/10.1080/00048670903001893

Borsa, J., Damásio, B., & Bandeira, D. (2012). Adaptação e validação de instrumentos psicológicos entre culturas: algumas considerações. Paidéia, 22(53), 423-432. doi: http://dx.doi.org/10.1590/1982-43272253201314

Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233-255. doi: http://dx.doi.org/10.1207/S15328007SEM0902_5

Conduct Problems Prevention Research Group. (1992). A developmental and clinical model for the prevention of conduct disorders: The Fast Track Program. Development and Psychopathology, 4,(4) 509-527. doi: http://dx.doi.org/10.1017/S0954579400004855

Coster, W. J., & Mancini, M. C. (2015). Recomendações para a tradução e adaptação transcultural de instrumentos para a pesquisa e a prática em Terapia Ocupacional. Revista de Terapia Ocupacional da Universidade de São Paulo, 26(1), 50-57. doi: http://dx.doi.org/10.11606/issn.2238-6149.v26i1p50-57

Durlak, J. A., & Dupre, E. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41 (3-4), 165-407. doi: http://doi.org/10.1007/s10464-008-9165-0

Embry, D. D. (2002). The Good Behavior Game: A best prac- tice candidate as a universal behavioral vaccine. Clinical Child and Family Psychology Review, 5(4), 273-297. doi: https://doi.org/10.1023/A:1020977107086

Flay, B. R., Biglan, A., Boruch, R., Castro, F. G., Gottfredson, D.,… Ji, P. (2005). Standards of evidence: criteria for efficacy, effectiveness and dissemination. Prevention Science, 6(3), 151-175. doi: http://doi.org/10.1007/s11121-005-5553-y

Ford, C. B., Keegan, N., Poduska, J. M., Kellam, S. G., & Litman. J. (2013). Good Behavior Game Implementation Manual. Washington, D. C.: American Institute for Research.

George, M., Squicciarini, A. M., Zapata, R., Guzmán, M. P., Hartley, M., & Silva, C. (2004). Detección Precoz de Factores de Riesgo de Salud Mental en Escolares. Revista de Psicología de la Universidad de Chile, 13(2), 9-20. doi: https://doi.org/10.5354/0719-0581.2012.17651

Guzmán, J., Kessler, R. C., Squicciarini, A. M., George, M., Baer L., … Murphy, M. J. (2015). Evidence for the Effectiveness of a National School-Based Mental Health Program in Chile. Child & Adolescent Psychiatry, 54(10), 799-807. doi: http://dx.doi.org/10.1016/j.jaac.2015.07.005

Kaplan, D. (2000). Structural equation modeling: foundations and extensions. Newbury Park, C. A.: Sage.

Kellam, S. G., Mackenzie, A. C., Brown, C. H., Poduska, J. M., Wang, W., Petras, H., & Wilcox, H. C. (2011). The Good Behavior Game and the future of prevention and treatment. Addiction Science & Clínical Practice, 6(1), 73-84. Recuperado de https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3188824/#

Kellam, S. G., Wang, W., Mackenzie, A. C., Brown, C. H., Ompad, D. C., Or, F., & Windham, A. (2014). The impact of the good behavior game, a universal classroom-based pre- ventive intervention in first and second grades, on high-risk sexual behaviors and drug abuse and dependence disorders into young adulthood. Prevention Science, 15(1), 6-18. doi: https://doi.org/10.1007/s11121-012-0296-z

Koth, C. W., Bradshaw, C. P., & Leaf, P. J. (2009). Teacher Observation of Classroom Adaptation-Checklist: Development and Factor Structure. Measurement and Evaluation in Counseling and Development, 42(1), 15-30. doi: https://doi.org/10.1177/0748175609333560

Kourkounasiou, M., & Skordilis, E. (2014).Validity and reliabi- lity evidence of the TOCA-C in a sample of Greek students. Psychological Reports: Measures & Statistics, 115(3), 766- 783. doi: https://doi.org/10.2466/08.11.PR0.115c31z5

Murphy, J. M., Abel, M. R., Hoover, S., Jellinek, M., & Fazel, M. (2017). Scope, scale, and dose of the world's largest school-based mental health programs. Harvard Review of Psychiatry, 25(5), 218-228. doi: 10.1097/HRP.0000000000000149

Murphy, J. M., Guzmán, J., McCarthy, A. E., Squicciarini, A. M., George, M., … Jellinek, M. S. (2015). Mental health predicts better academic outcomes: a longitudinal study of elementary school students in Chile. Child psychiatry and human development, 46(2), 245-256. doi: http://doi.org/10.1007/s10578-014-0464-4

Nolan, J. D., Houlihan, D., Wanzek, M., & Jenson, W. R. (2014). The Good Behavior Game: A classroom- behavior intervention effective across cultures. School Psychology International, 35(2), 191.205. doi: https://doi.org/10.1177/0143034312471473

Pasquali, L. (2010). Instrumentação psicológica: fundamentos e práticas. Porto Alegre: Artmed.

Pedroso, R. T., Abreu S., & Kinoshita, R. T. (2015). Aprendizagens da intersetorialidade entre saúde e educa- ção na prevenção do uso de álcool e outras drogas. Textura, 17(33), 9-24. Recuperado de http://www.periodicos.ulbra.br/index.php/txra/article/view/1339

Pérez, V., Fernández, A. M., Rodríguez, J., & de la Barra, F. (2004). Evaluación del efecto de una intervención conduc- tual en la agresividad escolar. Terapia Psicológica, 23(1), 91-98. Recuperado de https://www.redalyc.org/articulo.oa?id=78523110

Pérez, V., Rodríguez, J., de la Barra, F., & Fernández, A. M. (2005). Efectividad de una estrategia conductual para el manejo de la agresividad en escolares de enseñanza básica. Psykhe, 14(2), 55-62. doi: https://dx.doi.org/10.4067/S0718-22282005000200005

Pianta, R. C., & Steinberg, M. S. (1992). Teacher-child relation- ships and the process of adjusting to school. En R. Pianta C. (ed.), Beyond the parent: The role of other adults in children’s lives. New directions in child development (vol. 57, pp. 61-80). San Francisco: Jossey Bass.

Revelle, W. (2018). Psych: Procedures for Personality and Psychological Research. Northwestern University, Evanston. Recuperado de https://cran.r-project.org/web/packages=psych

Rosseel, Y. (2012). Lavaan: An R package for structural equa- tion modeling. Journal of Statistical Software, 48(2), 1-36.

Satorra, A., & Bentler, P. M. (2001). A scaled difference chi-square test statistic for moment structure analysis. Psychometrika, 66(4), 507-514. doi: https://doi.org/10.1007/BF02296192

Schneider D. R., Pereira A. P., Cruz J. I., Strelow M., Chan G.; Kurki A., & Sanchez Z. M. (2016). Evaluation of the Implementation of a Preventive Program for Children in Brazilian Schools. Psicologia: Ciência e Profissão, 36(3), 508-519. doi: https://doi.org/10.1590/1982-3703000592016

Tingstrom, D. H., Sterling-Turner, H. E., & Wilczynski, S. M. (2006). The Good Behavior Game: 1969-2002. Behavior Modification, 30(2), 225-253. doi: https://doi.org/10.1177/0145445503261165

Werthamer-Larsson, L., Kellam, S., & Wheeler, L. (1991). Effect of First-Grade Classroom Environment on Shy Behavior, Aggressive Behavior, and Concentration Problems. American Journal of Community Psychology, 19(4), 585-602. doi: https://doi.org/10.1007/BF00937993

##submission.citations.for##