Skip to main content

Research Repository

Advanced Search

Regularized joint mixture models

Perrakis, Konstantinos; Lartigue, Thomas; Dondelinger, Frank; Mukherjee, Sach

Regularized joint mixture models Thumbnail


Thomas Lartigue

Frank Dondelinger

Sach Mukherjee


Regularized regression models are well studied and, under appropriate conditions, offer fast and statistically interpretable results. However, large data in many applications are heterogeneous in the sense of harboring distributional differences between latent groups. Then, the assumption that the conditional distribution of response Y given features X is the same for all samples may not hold. Furthermore, in scientific applications, the covariance structure of the features may contain important signals and its learning is also affected by latent group structure. We propose a class of mixture models for paired data pX, Y q that couples together the distribution of X (using sparse graphical models) and the conditional Y | X (using sparse regression models). The regression and graphical models are specific to the latent groups and model parameters are estimated jointly. This allows signals in either or both of the feature distribution and regression model to inform learning of latent structure and provides automatic control of confounding by such structure. Estimation is handled via an expectation-maximization algorithm, whose convergence is established theoretically. We illustrate the key ideas via empirical examples. An R package is available at


Perrakis, K., Lartigue, T., Dondelinger, F., & Mukherjee, S. (2023). Regularized joint mixture models. Journal of Machine Learning Research, 24, 1-47

Journal Article Type Article
Acceptance Date Nov 10, 2022
Online Publication Date Jan 1, 2023
Publication Date 2023
Deposit Date Nov 21, 2022
Publicly Available Date May 2, 2023
Journal Journal of Machine Learning Research
Print ISSN 1532-4435
Electronic ISSN 1533-7928
Publisher Journal of Machine Learning Research
Peer Reviewed Peer Reviewed
Volume 24
Article Number 19
Pages 1-47
Publisher URL
Related Public URLs


Published Journal Article (1.3 Mb)

Publisher Licence URL

Copyright Statement
&copy; 2023 Konstantinos Perrakis, Thomas Lartigue, Frank Dondelinger and Sach Mukherjee.<br /> License: CC-BY 4.0, see Attribution requirements are provided<br /> at

You might also like

Downloadable Citations