A distribution-free method for change point detection in non-sparse high dimensional data

Drikvandi, Reza; Modarres, Reza

Abstract

We propose a distribution-free distance-based method for high dimensional change points that can address challenging situations when the sample size is very small compared to the dimension as in the so-called HDLSS data or when non-sparse changes may occur due to change in many variables but with small significant magnitudes. Our method can detect changes in mean or variance of high dimensional observations as well as other distributional changes. We present efficient algorithms that can detect single and multiple high dimensional change points. We use nonparametric metrics, including a new dissimilarity measure and some new distance and difference distance matrices, to develop a procedure to estimate change point locations. We also introduce a nonparametric test to determine the significance of estimated change points. We provide theoretical guaranties for our method and demonstrate its empirical performance in comparison with some of the recent methods for high dimensional change points. An R package called HDDchangepoint is developed to implement the proposed method.

Citation

Drikvandi, R., & Modarres, R. (2024). A distribution-free method for change point detection in non-sparse high dimensional data. Journal of Computational and Graphical Statistics, https://doi.org/10.1080/10618600.2024.2365733

• Journal Article Type: Article
• Acceptance Date: May 30, 2024
• Online Publication Date: Jun 12, 2024
• Publication Date: Jun 12, 2024
• Deposit Date: May 31, 2024
• Publicly Available Date: Jun 12, 2024
• Journal: Journal of Computational and Graphical Statistics
• Print ISSN: 1061-8600
• Electronic ISSN: 1537-2715
• Publisher: American Statistical Association
• Peer Reviewed: Peer Reviewed
• DOI: https://doi.org/10.1080/10618600.2024.2365733
• Public URL: https://durham-repository.worktribe.com/output/2468672