Simulation and data management environment
From February 1, 2022 to June 30, 2022
I worked on a project for SIREHNA, a company that runs computational fluid dynamics (CFD) simulations to evaluate ship performances (amongst other things). My role in this project was to create a simulation and data management environment that would help SIREHNA team track and retrieve their data, as well as automate low value-added tasks. After working on understanding the users' needs, I worked with the team on defining the environment architecture. We chose to use a web server as the corner stone of the architecture. We highlighted the functional needs such as User authentication, User management and data storage, and then benchmarked technologies. I used Girder library to instantiate a web server dedicated to managing users, their authentication and permissions. Using this library allowed us to gain time in implementing user authentication and management functionalities. To answer the large data storage need - coming from geometries involved in the CFD simulations that can weight tens of GB - I set up a MinIO instance. As MinIO uses Amazon Simple Storage Service (S3) standards, I could easily connect it to the Girder-based web server. Another main part of the environment, which was very time consuming, was to automated some of the CFD simulation processes. As a GitLab server was hosted internally, I used its pipelines features (which are mostly designed for CI/CD purpose) to fully automate simulations. The idea was to use GitLab pipeline's scheduler to separate the main steps of a simulation. The actual computation scripts were executed through Makefile scripts, while communications with the Girder-based web server were done via Python scripts. In order to execute computations efficiently, we installed a GitLab Runner on a device with a large computational power. The final piece of the puzzle was creating a user-friendly web interface using Girder web components, allowing users to easily start automated computations, monitor their progress, and keep track of their results.