We characterize the singular values of the linear transformation
associated with a standard 2D multi-channel convolutional layer,
enabling their efficient computation. This characterization also leads
to an algorithm for projecting a convolutional layer onto an
operator-norm ball. We show that this is an effective regularizer; for
example, it improves the test error of a deep residual network using
batch normalization on CIFAR-10 from 6.2\% to 5.3\%.
- Tags
-