For a complete presentation, see the pdf version of the offer available below.
Context
Stochastic Differential Equations (SDEs) are popular models in many fields including spatial ecology, climate science, biology. Diffusion SDEs with additive noise are commonly found. When proposing such a model for observed trajectories at discrete times, the next step consists in estimating the SDE parameters from those observed data. This is a critical task from which one can gain understanding on the underlying process mechanics. One classical parameter estimation approach is that of maximum likelihood. Many approaches have been proposed over the last decades, all with strengths and weaknesses.
Another feature of SDEs has been under-used in the estimation context. Namely, the Fokker-Planck Equation (FPE) is a Partial Differential Equation (PDE) which describes the evolution of the probability density function of the stochastic process described by the SDE. The FPE is parameterized by the same set of paramaters as the SDE. Thus, solving the FPE would provide an implicit expression of the marginal likelihood of each observation , which is a first step towards the maximum likelihood estimation of the parameters.
In the past few years, the emergence of Physics-Informed Neural Networks (PINNs) has led to a fundamental rethinking of traditional approaches to solving Partial Differential Equations (PDEs). In a few words, the PINNs approach seeks to find the best neural network representing the solution of the PDE. A recent line of research uses PINNs for simulation or parameter estimation in SDEs via their FPE. In this context, building on the previous articles, this internship will explore the connection between SDEs, their FPE, and the Physics-Informed Neural Network (PINN) methodology.
Goal of this internship: parameter estimation in SDEs with PINNs
The internship aims at proposing an efficient neural network architecture and optimisation scheme to accurately solve a FPE and perform parameter estimation by using observational data that are assumed to be generated by the corresponding SDE. Since the solution to the FPE is a normalized probability density, an interesting line of research considers using Normalizing Flows (NFs), as these architectures inherently encode the normalization constraint of probability densities. The Temporal NF with KR-net seems particularly well suited for this task. Such an architecture has been combined with a new loss function for training the PINN in where the author proposes to switch from the standard loss function of PINNs to a loss function involving the Feynman-Kac formula. The advantages resides in the fact that it has been shown to be well suited for non-stationary FPEs and to resolve some of the convergence issues of the vanilla PINN framework.
In this internship, we plan to test those new sophisticated approaches, since the classical PINN framework fails on more intricate FPEs.
An important part of the internship resides in the comparison of the developed PINN approach with other state-of-the-art approaches for parameter estimation in SDEs. Despite the fact that a proposed PINN model would lack theoretical guarantees (such as convergence guarantees), we expect that the PINN exhibits better accuracy in the estimation for a reduced computational time. This should be particularly true for high dimensional stochastic processes, as PINN training via the FPE does not require to linearize the equation, and benefits from optimized computations on GPUs. The validation of the models and parameter estimation approaches will first be carried on synthetic data before considering observational data from spatial ecology or climate sciences.