Critical data literacy in praxis: An open education approach for academic development

This paper reports the pedagogical approach and outcomes of a series of academic development programmes organised between 2016 and 2022 in different countries, which are grounded in the ethos of open educational practices, critical thinking, citizenship and pedagogy as well as ideas around social justice, data justice and data ethics using Open Data as open educational resources, to enable critical reflections and practical exercises with academics from different regions. Our recommendations and conclusions provide practical advice promoting a dialogue between different stakeholders to facilitate the development of curricula, workshops and resources using an open model for academic development.


INTRODUCTION
Higher Education (HE) has a social responsibility for forming critical and active citizens (Di Nauta et al., 2018;Pee, & Vululleh, 2020); therefore, it must enable spaces for academic development that foster critical citizenship connecting society, industry, innovation and research by developing transversal skills, which are defined as critical, innovative, interpersonal and intrapersonal skills and global citizenship (UNESCO, 2015).HE should therefore bridge educational processes with issues such as human rights; economy; migration; environment and sustainable development, and for that, the use of Open Data can be an effective tool to facilitate the interaction between teaching, research and society.
In research, we often say that data is rather benignly 'collected', suggesting that, like wild flowers or berries, it occurs in nature and belongs to no-one.The language of the business of data also implies that data is already present, but it tends to be discussed as a raw material which, through technological innovation, can more aggressively be mined or extracted (Mezzadra & Neilson, 2017).Without for a moment wishing to conflate the purposes of researchers and tech companies, we note that both ways of speaking about data buy into, and rather conveniently reinforce, the 'commonsense' idea that data comes from, and represents or reflects reality -that it transparently reports on the nature of the real, rather than being something made by human or machine.
To enhance critical thinking through citizenship and research skills, we promote the adoption of open education and science principles, including the use of Open Data (OD) as open educational resources (OER).Through the analysis of OD, using different methodologies in reallife scenarios, we can bridge learning and social problems, promoting research-based learning activities in multidisciplinary teams using the same raw material used in scientific, journalistic and public policy contexts, to promote a collaborative digital-enabled culture to empower people in sharing data, information, and knowledge (Ramachandran et al., 2021).
Van Es and Schäfer (2017) argue "students need to be educated to become critical data practitioners who are both capable of working with data and of critically questioning the big myths that frame the datafied society" (p.12).Therefore, we aim to support academics in developing critical and inquisitive relationships with data (Atenas et al., 2020;Holmes et al., 2022) to question it uses to address the data-literacy gap, to prevent widening the inequalities and power dynamics embedded in data-practices (Richterich, 2018).
Our approach in building critical data literacies aims at facilitating the Continuing Professional Development (CPD) for academics, supporting them in developing evidence-based learning and teaching programs and activities driven by OD, fostering critically and collaboratively research skills by studying their socio-political environment, cultural relationships and interactions between groups to uncover power relations and inequalities.
We report here the experiences and outcomes of a series of academic CPD courses delivered between 2016 and 2022, which are grounded in the ethos of open educational practices (Cronin & MacLaren, 2018), critical thinking, citizenship and critical theory as well as in social justice, data justice and data ethics (Markham, 2019(Markham, , 2020;;Floridi, 2021), to enable reflective and practical exercises with academics from different regions.Our recommendations and conclusions aim at providing practical advice to promote dialogue between different stakeholders to facilitate the development of teaching and learning activities and OER.

Our approach to critical data literacy
Data literacy is defined by Prado and Marzal (2013), as the set of skills that "enables individuals to access, interpret, critically assess, manage, handle and ethically use data" (p.126) whereas, critical data literacy is defined by Sander (2020) as "the ability to critically engage with datafication by reflecting on the societal implications of data processing and implementing this understanding in practice" (p.2).To critically interact with data, we followed Kellner's (2003) ideas of "education for democratising and reconstructing education to meet the challenges of a global and technological society" (p.1), as critical theory can enable us to identify how socioeconomic structures are being produced and reproduced in a datafied society, allowing us to and rethink our pedagogies using a critical, feminist and data justice approach (Braidotti, 2016;Dencik et al., 2016;Dencik & Sanchez-Monedero, 2022;D'Ignazio & Klein, 2020;Heeks & Swain, 2018;hooks, 1994;Taylor, 2017).
In bringing this range of frameworks and ideas into conversation with each other and with learners on our programmes, we fostered a curriculum for academic CPD that acknowledges the circumstances of oppression in the current datafication of society, to enable academics to build critical competences on their learners.As OD in teaching contexts, offers multiple opportunities to develop transversal, civic, literacy and numeracy skills, stimulating the development of critical thinking, developing research and citizenship skills necessary for life in a democratic yet datafied society (Atenas, et al., 2023;Buttiglione & Reggi, 2015;Couldry, 2020).This aligns with the call for action on the European Union's framework for digital competence (2022) which states that everyone should acquire understanding of new and emerging technologies including Artificial Intelligence (AI) and be aware of data ethics, including data protection and privacy, to prevent bias and discrimination, supporting the development of critical citizenry.

METHODOLOGY
We present an open and sustainable model to support academic CPD, to enable through a critical approach a series of data literacies that can facilitate the understanding of the risks, challenges, barriers and opportunities in the use of data for educators and students.Our approach for curriculum design and co-creation, showcases how we have enabled active citizenry and research skills in academic development, presenting the outcomes of a series of courses through the data collected between 2016 and 2022 in the form of evidence-based curricula, resources, and pre and post course surveys.
We provide an overview of the courses where our model was applied and their outcomes through quantitative and qualitative data analysis, showcasing a series of results in regards with curriculum design and data literacy, from formulating a problem, collecting, cleaning and analysing data until narrating results using data storytelling techniques.This has allowed us to identify gaps in knowledge and key resources and spaces educators need to gain a thorough understanding of data, datafication and data literacies from different angles, including OD, AI and learning analytics to present recommendations to shape curricula both for academic development and learner-centred activities.

Curriculum design
Our design rationale was framed in the current discussions around datafication of society (Schäfer & van Es, 2017) and data justice (Dencik et al., 2016;Taylor, 2017), as we argue that HE must lead in fostering critical socio-technical pedagogic approaches to build capacity in social engagement with issues of datafication, data-driven systems, and data-informed claims.Critical data literacies are needed to counteract what Ball (2015) describes as 'the tyranny of numbers'.We considered that to strengthen individual and collective agency, academic CPD should have a focus on social justice and participation, as data can easily marginalise data illiterate groups, rendering their points of view and lived experiences into data entry points and objects of study.
OD per se cannot universally promote justice, thus we agree with Gurstein (2011) that to prevent negatively impact on groups considered vulnerable, it is necessary to emphasise in data literacy training fostering the skills needed to critically debate the meaning and significance of data, to find responsible ways in the use of OD within the democratic debate (Atenas et al., 2015;Johnson, 2014;Taylor, 2017).

Curriculum co-creation
Our pedagogic approach is organised around involving students and educators in designing research using quantitative and qualitative techniques, to present their results in an open and ethical manner (Gilardi & Lozza, 2009).Co-creation is at the heart of our model and transversal to all activities and content we designed and co-curated with different stakeholders from academia and civil society to model good practices in collaborative pedagogies, while our participants were guided to co-create their own activities.
Our curriculum evolved due to the complexities and intricacies of the rapid developments in the data landscape as in our first courses we focused in promoting the adoption of practices from Open Government such as policy co-creation and participatory data stewardship using of OD towards enabling collaborative research, while in last courses the driver was questioning the impact of big data, machine-learning, and artificial intelligence (AI), due to the massive scale of data-driven automated decision-making can be life-altering by propagating and perpetuating gender and other social bias.
The outcomes of the courses, such as research-based learning activities, action plans and research proposals were designed to be shared, contributing to building a culture of open education, to support fellow educators in adapting, contextualising and reusing pedagogic reflective and practical activities with their learners.

Curriculum framework
We aimed at upskilling educators and learners across HE on data and datafication linking technical and socially-driven data skills across the data cycle to address issues and understand the ethical dimensions of data (Louie et al., 2022), to critically apply ideas of social and data justice into research-based learning, building on the idea of agency, so participants can acknowledge the problems and conundrums of datafication of society.The courses were designed to develop innovation skills in educators using open and emerging practices creating spaces for dialogue and research between students, educators, civil society and experts in OD to enhance education (Zamorski, 2002).
We first outlined pedagogical, reflective and collaborative practices that course participants could replicate or adopt in their teaching to scaffold understanding of data and datafication promoting the idea of civic monitoring and participation by humanising data using a humancentred-approach (see Table 1).To implement our curriculum, we were supported by researchers and practitioners from the academia and civil society, which allowed us to work with educators using real-life scenarios to develop knowledge through critical reflections contextualised in different countries (Littlejohn, et al., 2012).Through these courses we aimed at challenging the current economic and political status quo fostering inquiry in the power structures of the datafied society and in the education sector (D'Ignazio & Klein, 2020).The courses were hosted in different countries and had a variety of approaches and participants public as can be seen in Table 2. Our approach was dialogical to support the participants to develop their understanding and reasoning by examining dilemmas to propose solutions by designing collaborative researchbased learning activities to enable critical thinking through the use of data with students.

2016 -Open Data as OER: project, explore and narrate. Uruguay
This course was co-delivered with the NREA, and led by academics and data experts from Chile, Uruguay, UK and Italy and followed the model of the A Scuola di Open Coesione (Ciociola & Reggi, 2015), a pedagogic approach to develop in a short period of time, a series of skills for educators they could adopt to work with OD in multidisciplinary contexts to find solutions to real problems.
This course aimed at creating interdisciplinary instances of the use of OD as OER, using strategies to develop critical thinking, preparing educators to support students in analysing data to effectively communicate their research results, enabling a citizenship-led curricula for social participation.The outcomes were a series of group projects that addressed different social issues, providing a series of reports to the local authorities on its impact on family economy and society.Also, the groups produced a series of OD-led OER activities for anyone to access and adapt in their teaching.

2017 -Open Data and Open Government Policies and academic practice -Costa Rica
This international course was co-designed in collaboration with the University of Costa Rica and Abriendo Datos Costa Rica for academics, librarians and researchers in the use of OD as OER to promote curricular innovation.The academic team for this course comprised scholars, researchers, lawyers and members of the civil society who for a week, guided participants in understanding basic concepts of OD and Open Government to create learning activities with their students.
The training materials were adapted OERs from A Scuola di OpenCoesione and School of Data, and the dynamic of the course was dialogical and collaborative, so participants had to organise themselves into groups to solve a challenge using OD and report their results to the public and the competent authorities at the ConDatos and Abrelatam -Costa Rica conferences.We aimed to raise awareness of potential uses of OD and open government data in research and teaching, using strategies to support the publication of teaching materials and research outcomes in open formats, reviewing different strategies for the use of public information, Wikipedia, Wikidata and OD, promoting the adoption of citizen science approaches in collaboration with the civil society to develop democratic and citizenship skills.

2018 -Open Data and Open Governance: A tool for participation -Costa Rica
In partnership with the presidency of Costa Rica, and considering that Costa Rica does not have a law on Access to Information but does have a decree in OD and is a member of the Open Government Partnership, this course was focused on training Costa Rican public officials and academics from public universities to implement the objectives of Costa Rica's National Open Data Strategy which focuses on: Transparency and Access to Information; Fight against corruption and Citizen participation.
In our work with the participants from academia, we aimed at developing a critical mass that could effectively reuse the data published by the government and also, to become active in the discussion of their national action plan for Open Government Partnership.

2019 -Open Data and Open Governance: A tool for participation -Argentina -Uruguay
The course aimed to promote the use of Open Government Data and civic participation in collaboration with Government Agencies in Uruguay and the Government of Mendoza in Argentina, so participants could understand the impact of open government and OD, from their role as public officials or academics and as citizens.The training plan was designed to scaffold skills to lead OD projects in their organisations to improve the services, teaching and research in the public sector and academia.
The course presented a series of challenges and practical examples to understand the role participants must have in their organisations, providing participants with a series of theoretical and practical knowledge on OD to build and consolidate communities of practice that promote and champion the reuse of open government data, promoting interaction between government, civil society, private sector and academia.
The main outcome of the course was a series of proposals for the development of institutional and research plans, curricula and enquiry-based learning activities using OD to inform policymaking addressing social issues.

2019 -Open Data in academic practice -Uruguay
This course explored the educational potential of OD as a driver for interdisciplinary dialogue in the design of pedagogical practices.Participants were offered opportunities to design OERs and activities in two simple phases: A conceptual and dialogic introduction and a practical cocreation exercise.
The conceptual introduction presented the principles, political context and existing practices in open education, data and science, citizen science, responsible research and pedagogic innovation to collectively explore the connections between data literacy and active citizenship in HE, presenting a series of interdisciplinary challenges and co-creation exercises grounded on the sustainable development goals.The outcomes of this course were a series of multidisciplinary project-based learning ideas shared as OER to foster activities in which students could help their local communities to solve real life problems.

2019 -Understanding data: praxis and politics -Uruguay
This course was co-designed based on the idea that the society requires critical data literacies to address datafication of society and data-driven politics and academic practices (Atenas, 2021).The participants had 14 different nationalities and the course was facilitated by a large group of academics, researchers and members of the civil society enabling dialogues to provide educators with tools to operate ethically, technically and politically in a datafied society, adopting a critical perspective to question the ethical implications of data in people's daily life activities.
Our curriculum was grounded on the idea of ethics as a method, so we discussed issues such as data ethics; data privacy; data justice; personal and collective agency and legal and regulatory frameworks on OD and AI, towards promoting critical approaches to AI, including issues such as opacity and bias in algorithms, and regulations for automated decision-making and predictive analytics and power imbalances in data.
Our aim was to enable participants to contest the uses of data with the wherewithal to challenge the advancement of data inequalities, promoting inclusive growth, sustainable development and human-centred values.
The main outcomes of the course were a series of OER available for the HE community published in English and Spanish, and a series of reflections and research-based activities openly available in areas related with smart cities; data feminism; medical ethics; data and childhood; AI and education and data for development and socio-economic innovation.

2019 -Artificial Intelligence: present and future for development -Uruguay
This course was designed for academics and professionals in positions of decision-making in the public, academic, private and civil society sectors, who participate in different aspects of the cycle of public policies related to AI for development.The design of this course sought to Issue 84 -September 2023 Special issue: Open Educational Practices in Higher Education train key stakeholders in enabling the principles of AI for development, through dialogues and OERs created and adapted ad-hoc for the course.The pedagogical approach considered the study of cases of good and bad practices, the review of the international and regional panorama on public policy issues and recommendations for the development of effective and sustainable projects on AI in the public and HE sector.
The dialogues focused on a series of cross-cutting questions, such as the present and landscape of AI policies globally covering themes such as human rights, health, poverty, ethics, privacy and data governance the risks of AI; bias, exclusion and mitigation strategies; the, and explored what is required for effective co-creation of policies in the local context.The main outcomes of the course were a series of projects outlined by the public sector and civil society and research projects in academic fields, that can enhance the reflection on the state of AI in the region, considering the different edges that present the challenges in AI for development.

Review of competences development in participants
The rubric described in Table 3, was used across all the programmes between 2016 to 2022 to measure the level of the participants both at the beginning and end of each the course as a self-assessment questionnaire, without providing the skills levels in the questions, so they could self-evaluate their proficiency and understanding of different elements of data literacy.The progression data on the courses from between 2016 to 2022 reflects the average progression on a sample of 180 participants out of the 455 people that completed our training programmes.In general terms, we can see a consistent advancement from basic skills on the competences listed on Table 3 to intermediate or proficient self-assessment of competences.However, the progression to advanced skills was quite limited, perhaps because participants did not feel confident with the new knowledge they acquired, and also, because some of the advanced skills required more practical exercises.Therefore, we provided to the participants with OERs so they could continue advancing their skills upon completion of the course (see Figure 1).Also in the courses Understanding data: praxis and politics and Artificial Intelligence: present and future for development we included questions to assess understanding on issues related with AI and machine-learning in relation with its uses, legal frameworks, policy, and governance dimensions, as we expected that participants from these courses could actively participate in policy and governance arenas in their institutions, as most of them were leading research and teaching in these dimensions (see Figure 3).
Figure 3 AI policy and governance skills 3.6.Feedback from the courses Our courses were complex and enriching and grounded in the ethos of open education, as these proposed a series of personal and group reflections about our lives as academics and citizens.We enabled spaces for discussion, dialogue, creativity and co-creation in international and multidisciplinary contexts.Therefore, these courses were challenging for the participants and for the pedagogic teams, but in general terms the participants embraced them as a dare with a collaborative spirit, giving us the space to further the discussions and engage with their observations and projects.
Across the years, the participants' feedback can be summarised in the value of reviewing different perspectives when presenting a phenomenon or problem to students to develop critical thinking, providing different perspectives to solve a challenge.The participants consistently commended our innovative training model which offered a variety of formulas to answer questions through OD to facilitate diverse activities.A participant mentioned that "The format of the course was very enjoyable, and the tutors are very professional and expert, they know what they are talking about, it was a pleasant learning experience" (Uruguay-2016) and that "I really enjoyed it a lot, the subject of access to information seems fascinating to me, the course was too good with super bright, attentive and patient instructors.Such a complex subject became very easy to understand and apply" (Chile-2017).
The feedback reflects the success of our curriculum and how participants appreciated the work of the facilitators and the content, the safe spaces for dialogs and also, having access to experts in the field to guide them on their learning.One participant commented that "What I liked the most were the speakers and the amount of knowledge I had access to" (Costa Rica-2017), while another participant mentioned that "What I liked the most was being able to listen to experts from our region as normally 99% of the materials and training come from the global north" (Costa Rica-2018).
We put a lot of focus on developing communities of practice across the courses, and, as a participant mentioned, "The first utility of data in higher education is the possibility of enlarging the OD community with the incorporation of students so that they add value to the different data sets" (Uruguay-2016).Participants also mentioned the "usefulness of learning to critically analyse the information presented by the media as it changes the way of looking at information and research" (Uruguay-2021).Also, the participants discussed the opportunities given by the courses to co-create research with students because everyone has the same access to the data that will allow them to do cooperative investigations using multidisciplinary approaches to add value to research.
Finally, one of the most frequent comments on the courses' feedback were the references to the dialogical approaches and spaces for collaboration.For example, one participant noted that "The course was very enriching for me on several levels, such as quality of the content, the interdisciplinary dialogue, and the participation of people from various countries" (Uruguay-2021).While someone mentioned that they valued "The exchange of ideas and the contributions of the colleagues, the feedback from the tutors, the clarity of the readings and material, the participation in the group work, all this was enriching" (Uruguay-2022).
The feedback allowed us to improve our practices and document our methodologies to share them with other practitioners and academics that are willing to enable critical data literacies and promote the use of OD in the HE context to upskill fellow educators, researchers and most of all students to develop lifelong skills to participate in the datafied society.

RECOMMENDATIONS AND CONCLUSIONS
Given the diverse background and fields of expertise of scholars taking the different courses and their uneven knowledge on research methods, we proposed a series of structured synchronous and asynchronous activities dealing with various topics associated with data and its relationship with open education, government and science, transparency, AI, innovation, privacy, ethics and inclusion, as well as access to public information and data journalism.
Thus, we outlined a series of suggestions to work with their students grounded on social justice values that can be easily adapted and contextualised in different HE institutions as shown in Table 4.We suggest that thematic activities are led by a diverse group of experts, aiming to present diverse and heterogeneous views of the topics towards problematise issues discussed beyond the cases showcased in these sorts of activities.
Activities for academic CPD should scaffold learning by integrating in the curriculum global and local perspectives and expertise to provide participants with a wider panorama of the critical issues of a datafied society.Alike our pedagogic approach, we promote the inclusion of mentorship from academic advisors and guest experts to enhance dialogue and collective reflections.
To develop critical perspectives which examine and challenge power dynamics through data, our approach was built in layers which examine particular perspectives including data ethics, data feminism, agency and social justice and data justice.We contend that this approach supports educators in enhancing their teaching practice, providing them with the strategies necessary to design learning activities that help their students develop a critical understanding of the ethical dimension of data-dynamics and identify problematic data practices.In turn, we hope that students will be empowered and inspired to examine and challenge the power structures that perpetuate intersectional oppressions, and adopt a data justice lens when considering issues related to privacy, consent, personal agency, and data sovereignty.
Our courses were funded by grants from Fundación Avina; the Inter-American Development Bank; the Organisation of American States; the HDI network and IRDC Canada, and were supported by the ILDA, the Centre for Open Education (NREA) at the University of La República and Agency for Electronic Government and the Information and Knowledge Society in Uruguay; the Presidency and the University of Costa Rica; the Government of Mendoza (Argentina) and, the University of Chile.Several civil society organisations, academics and research centres contributed with their experiences to co-create the curricula including colleagues from ILDA; School of Data; Abriendo Datos Costa Rica; Data Wheel; Data Uruguay; Grupo de Investigación en Gobierno, Administración y Políticas Públicas ; Data for Development; University College London; University of Augsburg; the Data Justice Lab; Monithon EU and A Scuola di Open Coesione; LATAM digital; Open Contracting; Open Data Charter; Open Knowledge International and the Interdisciplinary Centre for Data Science and Machine Learning as well as the Open Education UNESCO chair at the University of la República in Uruguay, the UNESCO Chair in Open and Distance Education at the University of Brasilia and the UNESCO coalition in Open Education.
3.4.3.2017 -Open Data Practices in the HE sector -Chile This course promoted the adoption of open practices in education, government and sciences practices and data in academic practice.It was designed with a focus on individual and collective reflection about the value of open-research, public information and OER, considering that Chile is an early adopter of OD principles and charters, to enhance local teaching and research practices.

Figure 1
Figure 1 Review of competences pre and post-course

Figure 2
Figure 2Pedagogic and research skills

Table 1
Curriculum framework by level and proficiency

Table 2
The academic CPD experiences

Table 3
Competence self-assessment rubric

Table 4
Recommended activities and dynamics