The European public sector faces increasingly complex socio-economic problems that require innovative solutions. Big data and data-informed decision-making offer a pathway to better solutions, greater transparency and enhanced trust in public administration. However, implementing these technologies needs skills, experience and appropriate digital tools, which are often lacking.
Introducing the EU’s Big Data Test Infrastructure
To tackle these challenges, the European Commission launched the Big Data Test Infrastructure (BDTI) under the Digital Europe Programme in 2019. The BDTI aims to promote the reuse of public sector data and facilitate a data-informed public sector across EU member states. It provides public administrations with a comprehensive set of mainstream open-source tools for data storage, processing and analytics, all hosted in the cloud and free of charge.
This service empowers civil servants at all government levels to conduct pilot projects independently, demonstrating the value of data in policymaking and administration. Public administrations can utilise the BDTI to prototype solutions before implementing them in live environments.
The platform comprises open-source tools and necessary cloud infrastructure like virtual machines, analytics clusters, storage facilities and networking capabilities. After completing a pilot project, users can transfer the source code and data to continue their work on their preferred cloud or other resources.
The BDTI offers a variety of tools to facilitate data-driven projects, including data storage and processing tools for handling large datasets, analytics tools for analysis and visualisation, and cloud infrastructure for running data-intensive tasks. Its open-source nature ensures flexibility, allowing public administrations to explore different solutions cost-effectively.
How public administrations are using the BDTI
Several public administrations have utilised and continue to use the BDTI to enhance decision-making processes. For example, the Valencia health services previously employed BDTI to apply text mining techniques for extracting valuable insights from health data. Similarly, the Norwegian Digitalisation Agency used BDTI to optimise public procurement processes through data analytics, while the city of Florence, Italy, utilised BDTI to understand the impact of COVID-19 on the city through data analytics.
BDTI offers a variety of tools to facilitate data-driven projects
Currently, numerous public administrations are leveraging BDTI. For instance, the University of Macedonia in Greece transformed data gathered through the MITOS API (Greek National Registry of Administrative Public Service) into linked data aligned with the EU standard model to support more public services. The city of Bochum, Germany, is using BDTI to analyse the health of trees, providing valuable insights for urban planning and environmental management.
Additionally, Naples, in Italy, is employing BDTI to collect data related to public spaces and mobility, streamlining citizen participation and creating a transportation dataset tailored for public administration in compliance with the open data directive.
Resources for public administrations
To further support public administrations, the BDTI team has developed a BDTI Essentials course, available for free. This course consists of five one-hour webinars, targeting public administration officials to help them better understand how to use the BDTI with concrete examples.
This educational resource aims to build the necessary skills and knowledge to effectively leverage the BDTI for data-driven projects. Additionally, the BDTI team has recently launched a new online course in collaboration with the interoperability academy, to help public sector actors advance their data literacy journey and understand how BDTI can help shortcut their data maturity efforts.
In line with their commitment to inclusivity, the BDTI team has also taken the initiative to translate key BDTI materials into the 24 European official languages. This effort allows individuals from diverse linguistic backgrounds to fully understand and engage with the content.
Public administrations interested in using the BDTI can subscribe to the Joinup page or to ‘The BDTI Kitchen: Baking Data Stories’ newsletter for updates on future events and initiatives.
Transforming public services
The BDTI represents a significant step towards a data-informed public sector in the EU, offering public administrations the tools and infrastructure needed to harness the power of big data. Through pilot projects and real-life use cases, the BDTI demonstrates the transformative potential of data analytics in improving public services and policymaking.
By enabling experimentation and innovation in a cost-effective manner, the BDTI helps public administrations overcome data challenges, fostering a more transparent, efficient and trustworthy public sector.
Are you a civil servant interested in developing your own use case on the BDTI or a student and researcher eager to support a public administration? Apply for a BDTI pilot project, or reach out to the team via EC-BDTI-PILOTS@ec.europa.eu.