Exploiting regularity in sparse Generalized Linear Models

Abstract : Generalized Linear Models (GLM) are a wide class of regression and classification models, where the predicted variable is obtained from a linear combination of the input variables. For statistical inference in high dimensions, sparsity inducing regularization have proven useful while offering statistical guarantees. However, solving the resulting optimization problems can be challenging: even for popular iterative algorithms such as coordinate descent, one needs to loop over a large number of variables. To mitigate this, techniques known as screening rules and working sets diminish the size of the optimization problem at hand, either by progressively removing variables, or by solving a growing sequence of smaller problems. For both of these techniques, significant variables are identified by convex duality. In this paper, we show that the dual iterates of a GLM exhibit a Vector AutoRegressive (VAR) behavior after sign identification, when the primal problem is solved with proximal gradient descent or cyclic coordinate descent. Exploiting this regularity one can construct dual points that offer tighter control of optimality, enhancing the performance of screening rules and helping to design a competitive working set algorithm.
Type de document :
Communication dans un congrès
Liste complète des métadonnées

Littérature citée [36 références]  Voir  Masquer  Télécharger

https://hal-univ-bourgogne.archives-ouvertes.fr/hal-02288859
Contributeur : Imb - Université de Bourgogne <>
Soumis le : dimanche 13 octobre 2019 - 19:49:37
Dernière modification le : lundi 14 octobre 2019 - 14:07:24

Fichier

spars2019.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-02288859, version 1

Citation

Mathurin Massias, Samuel Vaiter, Alexandre Gramfort, Joseph Salmon. Exploiting regularity in sparse Generalized Linear Models. SPARS, Jul 2019, Toulouse, France. ⟨hal-02288859⟩

Partager

Métriques

Consultations de la notice

53

Téléchargements de fichiers

16