Content analysis and critical thinking. An assessment study

An important passage in the 2010 OECD (Organisation for Economic Cooperation and Development) report Investing in Human and Social Capital underlines the need for formal education to enter the workplace: In the nations where work is organized to support high levels of employee discretion in solving complex problems, the evidence shows that firms tend to be more active in terms of innovations developed through their own in house creative efforts (OECD, 2010, theme 1, p. 10). The key point is in fact that new skills are needed, because traditional skills learnt at school or at university are disappearing and are not deemed useful in facing the needs for innovation and growth that society today demands. The concept behind our project, Contributions for the Definition of a Critical Technology, is therefore that of verifying the effectiveness of a model constructed to increase critical thinking skills, which are essential in environments such as those described by the OECD, in the above-mentioned document. The present contribution aims to describe the results of the study carried out at DIPED - Dipartimento di Progettazione Educativa e Didattica (Department for Educational Design) - Roma Tre University, where the levels of critical thinking skills of students were assessed through an ad hoc content analysis protocol. The different sections explain why content analysis is considered a valid and reliable technique in the assessment of critical thinking skills and how the procedure was used in the above-mentioned project. The research is set within this context and, though being implemented in higher education, aims to project its results into different settings, in order to improve other areas, such as lifelong learning, and enhance development in various fields of knowledge. The project principally aims to assess the hypothesis that, in providing further cultural insights, according to well-defined models on which to undertake guided discussions coordinated by an experienced tutor, critical thinking skills of students increase. This is made possible through the development of an ad hoc online module, Critical Thinking Skills and Reading of the Classics, available to students in Education (Faculty of Education Sciences). In order to assess critical thinking skills, the students’ written productions were treated with a lexicometric analysis using the Taltac software, and with content analysis, through an adaptation of the Newman, Webb and Cochrane (1997) model. The main categories of the analysis include relevance, importance, introduction of new ideas, information and solutions, reference to personal experience and opinions, clarification of doubts, new knowledge, elaboration of new solutions, critical evaluation, practical use of new solutions, width of understanding. The ability to think critically and therefore to make functional use of what is learnt is what the OECD report itself mentioned as vital if wanting to enhance the development of new skills and in particular skills that are effective for growth and innovation in complex organisations.

Keywords: Content analysis, critical thinking, assessment, innovation, critical technology.

• Creative Approaches to Technology-Enhanced Learning for the Workplace and Higher Education Antonella Poce, pp.94 (ISBN:978-3-031-73426-7)
• A systematic review on critical thinking intervention studies in higher education across professional fields Blanca Puig, Paloma Blanco-Anaya, Inés M. Bargiela, Beatriz Crujeiras-Pérez, in Studies in Higher Education /2019 pp.860
  DOI: 10.1080/03075079.2019.1586333
• Project and Design Literacy as Cornerstones of Smart Education Poce Antonella, De Medio Carlo, Amenduni Francesca, pp.143 (ISBN:978-981-13-9651-9)
• Contributi per la definizione di una tecnologia critica interdisciplinare Laura Corcione, Annalisa Iovine, Antonella Poce, in CADMO 1/2013 pp.41
  DOI: 10.3280/CAD2013-001004
• Pen or keyboard. An empirical study on the effects of technology on writing skills Benedetto Vertecchi, Antonella Poce, Francesco Agrusti, Maria Rosaria Re, in CADMO 2/2017 pp.33
  DOI: 10.3280/CAD2016-002004

Antonella Poce, Laura Corcione, Annalisa Iovine, Content analysis and critical thinking. An assessment study in "CADMO" 1/2012, pp 47-63, DOI: 10.3280/CAD2012-001006