RBMs are generative models capable of fitting complex dataset's probability distributions. Thanks to their simple structure, they are particularly well suited for interpretability and pattern extraction, a feature particularly appealing for scientific use. In this talk, we show that RBMs operate in two distinct regimes, depending on the procedure followed to estimate the log-likelihood gradient during the training. Short sampling times fit machines that are trained to reproduce exactly the dynamics followed to train them, long samplings (as compared to the MCMC mixing time) are need to learn a good model for the data. The non-equilibrium regime should be used to generate high quality samples in short learning and sampling times, but cannot be used to extract the unnormalized data probability of the data necessary for interpretability. In practice, it is hard to extract good equilibrium models for structured datasets (which is the typical case in biological applications) due to a divergence of the Monte Carlo mixing times. In this work, we show this barrier can be surmounted using biased Monte Carlo methods. [-]

I will first introduce a general class of mean-field-like spin systems with random couplings that comprises both the Ising model on inhomogeneous dense random graphs and the randomly diluted Hopfield model. I will then present quantitative estimates of metastability in large volumes at fixed temperatures when these systems evolve according to a Glauber dynamics, i.e. where spins flip with Metropolis rates at inverse temperature $\beta $. The main result identifies conditions ensuring that with high probability the system behaves like the corresponding system where the random couplings are replaced by their averages. More precisely, we prove that the metastability of the former system is implied with high probability by the metastability of the latter. Moreover, we consider relevant metastable hitting times of the two systems and find the asymptotic tail behaviour and the moments of their ratio. This result provides an extension of the results known for the Ising model on the the Erdos-Renyi random graph. Our proofs use the potential-theoretic approach to metastability in combination with concentration inequalities.
Based on a joint work in collaboration with Anton Bovier, Frank den Hollander, Saeda Marello and Martin Slowik.[-]