Fast-rate PAC-Bayes generalization bounds via shifted rademacher processes

Publikation: Bidrag til tidsskriftKonferenceartikelForskningfagfællebedømt

The developments of Rademacher complexity and PAC-Bayesian theory have been largely independent. One exception is the PAC-Bayes theorem of Kakade, Sridharan, and Tewari [21], which is established via Rademacher complexity theory by viewing Gibbs classifiers as linear operators. The goal of this paper is to extend this bridge between Rademacher complexity and state-of-the-art PAC-Bayesian theory. We first demonstrate that one can match the fast rate of Catoni's PAC-Bayes bounds [8] using shifted Rademacher processes [27, 43, 44]. We then derive a new fast-rate PAC-Bayes bound in terms of the “flatness” of the empirical risk surface on which the posterior concentrates. Our analysis establishes a new framework for deriving fast-rate PAC-Bayes bounds and yields new insights on PAC-Bayesian theory.

TidsskriftAdvances in Neural Information Processing Systems
StatusUdgivet - 2019
Eksternt udgivetJa
Begivenhed33rd Annual Conference on Neural Information Processing Systems, NeurIPS 2019 - Vancouver, Canada
Varighed: 8 dec. 201914 dec. 2019


Konference33rd Annual Conference on Neural Information Processing Systems, NeurIPS 2019
SponsorCitadel, Doc.AI, et al., Lambda, Lyft, Microsoft Research

Bibliografisk note

Publisher Copyright:
© 2019 Neural information processing systems foundation. All rights reserved.

ID: 361431893