Satellite Signal Processing with Monte Carlo Markov Chains (MCMC)

The project

The Almanac is used by Leiden Observatory in order to extract and interpret cosmological data about phenomena such as dark matter and gravity from noisy satellite observations. This is achieved by cap-turing the statistical structure of the universe through generating posterior distributions of power spectra Cℓ’s and decomposition coefficients aℓm’s. It concerns a huge missing-data problem, as it is defined on 17 million dimensions.

The workflow operates in four sequential phases: burn-in , step-size tuning, leapfrog tuning and Hamiltonian Monte Carlo (HMC) sampling. The last phase utilises a Markov Chain Monte Carlo (MCMC) algorithm to propose trajectories through the state space. The trajectory length is determined by the number of steps and the step size, with a target acceptance rate of 0.65 for proposed steps. Originally, step-sizes were randomly selected from a predefined set of values, which proves to be a limitation.

The goal of the project is to achieve an improvement in the Effective Sample Size (ESS), resulting in a higher rate of independent samples compared to the original baseline, while converging faster to the target acceptance rate of 0.65 during the HMC running phase. Additionally, another goal is to maintain larger step sizes, as measured by the Expected Squared Jumping Distance (ESJD), a metric that quantifies the length of the entire trajectory.

Real eyes realise real lies, and our ADAM implementation never dies.

The customer

The project was created by Associate Professor Dr Elena Sellentin and her PhD candidate Kutay Nazli, who are affiliated with the Mathematical Institute and Leiden Observatory (Sterrewacht; STRW). Their research expertise lies in Analysis and Dynamical Systems applied to Astrophysics. Dr Sellentin has contributed to the design of data analysis methods for the Euclid satellite mission of the European Space Agency (ESA). Euclid is a space telescope which reveals details about gravity, dark matter and dark energy by examining how the Universe has expanded and formed throughout cosmic history. At Leiden University, Dr. Sellentin teaches the course Modern Astrostatistics, which equips students with the techniques for discovering and categorizing astronomical objects, as well as identifying subtle signals within noisy data.

Her research focuses on randomness and data analysis, particularly in areas such as covariance and Markov Chain Monte Carlo techniques. Current research efforts by Dr. Elena Sellentin and Kutay Nazli aim to improve the sampling efficiency of the Almanac codebase, which is used for the extraction and interpretation of satellite imaging of the sky.

Does anyone understand how Leapfrog actually works?

The team

Team404 consisted of four Data Science and AI students (Gabriela Czapska, Tome Jolevski, Odysseas Lydakis Simantiris and Mateusz Oleksa) and two Computer Science students (Lianne van den Bosch and Joy Liu). During the project, each team member took on individual commitments to both increase workflow and efficiency, as well as to adhere to everyone’s strengths.

Lianne van den Bosch: Developer (C++ and Python), Teaching Assistant Contact Person

Gabriela Czapska: Documentation Coordinator, Weekly Assignment Supervisor, Developer (Python)

Tome Jolevski: General Secretary (meeting notes), Developer (Python)

Joy Huxinyuan Liu: Lead C++ Developer (Code Master)

Odysseas Lydakis Simantiris: Product Owner, Client Contact Person, Developer (Python)

Mateusz Oleksa: SCRUM Master, Developer (Python)

Both client and TA meetings were held once every two weeks on an alternating basis, ensuring open communication and the opportunity for both parties to ask questions or provide feedback whenever necessary. Informally, the team also aimed to meet twice a week to work on established tasks, set outlines for future sprint goals, track each member’s progress and hold each other accountable whenever setbacks would occur. In order to ensure efficient logging of meetings and project documentation, we used a combination of Docs and GitHub, with our GitHub project board containing a detailed overview of which tasks have been completed, are yet to be completed, or were dropped entirely.

The biggest challenge during this project was the integration of our optimizer implementation into the massive codebase that makes up Almanac. Specifically, this was due to the very specialized nature of the codebase and concepts at hand, and the fact that the codebase itself was written entirely in C++, which the majority of our team had no prior experience with. Overcoming these obstacles proved quite challenging at first, however, through a diligent work ethic, a collaborative atmosphere and frequent team meetings (as well as beneficial client feedback), we managed to produce results that we were all satisfied with.

ESS is fine, but ESS/s is too far - one too many S's

The technologies

• Introduction of the performance metrics ESJD and ESS to assess the effectiveness of our implementations for each elapsed run.
• Implementation of a gradient-based ADAM optimizer applied during the step-size tuning phase, with 2 alternatives that each tune different hyperparameters: one uses the acceptance rate as the optimisation metric (aiming at an acceptance rate of 0.65), whereas the other aims at maximising the ESJD during the optimisation process.