The CernVM File System (CernVM-FS) provides a scalable and reliable software distribution service. It was developed at CERN to assist High Energy Physics (HEP) collaborations in deploying software on the worldwide distributed computing infrastructure for data processing applications, but it can also be used in other domains.
CernVM-FS is implemented as a POSIX read-only file system in user space (FUSE) and uses content-addressed storage and Merkle trees for maintaining file data and meta-data. Files are stored remotely on standard web servers and are retrieved and cached on-demand through outgoing HTTP connections only, avoiding most of the firewall issues of other network file systems.
For writing, CernVM-FS follows a publish-subscribe pattern with a single source of new content that is propagated to a large number of readers.
This talk focuses on a new publication architecture, capable of accommodating dynamic strategies for content publishing and improving the scalability of the publication phase. We chose Erlang/OTP for implementing certain components of the new architecture, due to its suitability for making scalable and fault-tolerant systems.
Radu Popescu is currently a CERN fellow, working on the CernVM-FS distributed file system project, a mission critical system of the CERN physics collaboration. He really enjoys distributed systems, expressive type systems and functional programming. He believes studying programming language design makes for a better programmer, regardless of the tools used. Radu has worked on a multitude of other interesting projects, including a 3D editor for photogrammetry data - Pix4Dmapper and libraries for high performance computing (HPC) - Trilinos, LifeV. His relationship with HPC culminated in a PhD degree obtained at the Ecole Polytechnique Federale Lausanne, Switzerland.