Exploiting regularity in sparse Generalized Linear Models

Abstract : Generalized Linear Models (GLM) are a wide class ofregression and classification models, where the predictedvariable is obtained from a linear combination of the in-put variables. For statistical inference in high dimensions,sparsity inducing regularization have proven useful whileoffering statistical guarantees. However, solving the result-ing optimization problems can be challenging: even forpopular iterative algorithms such as coordinate descent, oneneeds to loop over a large number of variables. To mitigatethis, techniques known asscreening rulesandworking setsdiminish the size of the optimization problem at hand, eitherby progressively removing variables, or by solving a growingsequence of smaller problems. For both of these techniques,significant variables are identified by convex duality. In thispaper, we show that the dual iterates of a GLM exhibit aVector AutoRegressive (VAR) behavior after sign identifi-cation, when the primal problem is solved with proximalgradient descent or cyclic coordinate descent. Exploitingthis regularity one can construct dual points that offertighter control of optimality, enhancing the performance ofscreening rules and helping to design a competitive workingset algorithm.
Type de document :
Communication dans un congrès
Liste complète des métadonnées

Littérature citée [36 références]  Voir  Masquer  Télécharger

https://hal-univ-bourgogne.archives-ouvertes.fr/hal-02288859
Contributeur : Imb - Université de Bourgogne <>
Soumis le : dimanche 13 octobre 2019 - 19:49:37
Dernière modification le : mercredi 8 janvier 2020 - 01:37:15
Archivage à long terme le : mardi 14 janvier 2020 - 13:49:55

Fichier

spars2019.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-02288859, version 1

Citation

Mathurin Massias, Samuel Vaiter, Alexandre Gramfort, Joseph Salmon. Exploiting regularity in sparse Generalized Linear Models. SPARS 2019 - Signal Processing with Adaptive Sparse Structured Representations, Jul 2019, Toulouse, France. ⟨hal-02288859⟩

Partager

Métriques

Consultations de la notice

68

Téléchargements de fichiers

26