Convolutional Networks with Adaptive Inference Graphs
Research output: Contribution to journal › Journal article › Research › peer-review
Standard
Convolutional Networks with Adaptive Inference Graphs. / Veit, Andreas; Belongie, Serge.
In: International Journal of Computer Vision, Vol. 128, No. 3, 01.03.2020, p. 730-741.Research output: Contribution to journal › Journal article › Research › peer-review
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - JOUR
T1 - Convolutional Networks with Adaptive Inference Graphs
AU - Veit, Andreas
AU - Belongie, Serge
N1 - Publisher Copyright: © 2019, Springer Science+Business Media, LLC, part of Springer Nature.
PY - 2020/3/1
Y1 - 2020/3/1
N2 - Do convolutional networks really need a fixed feed-forward structure? What if, after identifying the high-level concept of an image, a network could move directly to a layer that can distinguish fine-grained differences? Currently, a network would first need to execute sometimes hundreds of intermediate layers that specialize in unrelated aspects. Ideally, the more a network already knows about an image, the better it should be at deciding which layer to compute next. In this work, we propose convolutional networks with adaptive inference graphs (ConvNet-AIG) that adaptively define their network topology conditioned on the input image. Following a high-level structure similar to residual networks (ResNets), ConvNet-AIG decides for each input image on the fly which layers are needed. In experiments on ImageNet we show that ConvNet-AIG learns distinct inference graphs for different categories. Both ConvNet-AIG with 50 and 101 layers outperform their ResNet counterpart, while using 20 % and 38 % less computations respectively. By grouping parameters into layers for related classes and only executing relevant layers, ConvNet-AIG improves both efficiency and overall classification quality. Lastly, we also study the effect of adaptive inference graphs on the susceptibility towards adversarial examples. We observe that ConvNet-AIG shows a higher robustness than ResNets, complementing other known defense mechanisms.
AB - Do convolutional networks really need a fixed feed-forward structure? What if, after identifying the high-level concept of an image, a network could move directly to a layer that can distinguish fine-grained differences? Currently, a network would first need to execute sometimes hundreds of intermediate layers that specialize in unrelated aspects. Ideally, the more a network already knows about an image, the better it should be at deciding which layer to compute next. In this work, we propose convolutional networks with adaptive inference graphs (ConvNet-AIG) that adaptively define their network topology conditioned on the input image. Following a high-level structure similar to residual networks (ResNets), ConvNet-AIG decides for each input image on the fly which layers are needed. In experiments on ImageNet we show that ConvNet-AIG learns distinct inference graphs for different categories. Both ConvNet-AIG with 50 and 101 layers outperform their ResNet counterpart, while using 20 % and 38 % less computations respectively. By grouping parameters into layers for related classes and only executing relevant layers, ConvNet-AIG improves both efficiency and overall classification quality. Lastly, we also study the effect of adaptive inference graphs on the susceptibility towards adversarial examples. We observe that ConvNet-AIG shows a higher robustness than ResNets, complementing other known defense mechanisms.
KW - Convolutional neural networks
KW - Gumbel-Softmax
KW - Residual networks
UR - http://www.scopus.com/inward/record.url?scp=85068103742&partnerID=8YFLogxK
U2 - 10.1007/s11263-019-01190-4
DO - 10.1007/s11263-019-01190-4
M3 - Journal article
AN - SCOPUS:85068103742
VL - 128
SP - 730
EP - 741
JO - International Journal of Computer Vision
JF - International Journal of Computer Vision
SN - 0920-5691
IS - 3
ER -
ID: 301823501