Transforming Data Management for Next-Generation Mobility Research

As the BERTHA project advances the frontiers of Connected, Cooperative, and Automated Mobility (CCAM), managing the growing volume and complexity of research data has become a top priority. To meet this challenge, BERTHA has developed and validated a cutting-edge data infrastructure built on NiFi, with Kubernetes as the target operational environment, setting a new benchmark for secure, scalable, and efficient data sharing across its European consortium.

From Manual Transfers to Automated, High-Speed Data Workflows

Previously, data transfers relied on manual scripts and Python-based solutions, which posed limitations in scalability and efficiency. The transition to a NiFi-based solution represents a major step forward for BERTHA’s data management, enabling automated, high-speed workflows that can handle increasing data volumes without delays. The integration of Kubernetes orchestration further enhances operational efficiency, providing a robust and flexible environment for all project partners.

Security and Governance at the Core

Security and data governance are central to BERTHA’s infrastructure upgrade. The project has validated SSH parameters for large file transfers and defined IP-based access control to safeguard sensitive research data. A “declaration of honour” protocol ensures that only authorized users can access the platform, reinforcing trust and compliance across the consortium. These measures align with the highest standards of data protection and support the FAIR data principles (Findable, Accessible, Interoperable, Reusable).

Future-Proofing with the Phoenix Cluster

Looking ahead, BERTHA is currently migrating its data and services to the Phoenix cluster, a move that promises to sustain project outputs well beyond the initial funding period. This strategic investment is designed to ensure that valuable datasets, workflows, and research tools remain accessible to the scientific community, supporting long-term impact and collaboration in CCAM research.

Why This Matters for CCAM and Research Collaboration

By embracing NiFi and Kubernetes, BERTHA demonstrates its commitment to operational excellence, data security, and scalable research infrastructure. This approach positions BERTHA as a strong reference for scalable and secure data infrastructure for other mobility and transport research initiatives across Europe.