SPLIT: Out-of-distribution Generalization with Granularity Biased Variational Autoenscoder

A variational autoencoder (VAE) model that captures global and local variations in its representation can improve generalization performance in downstream tasks. We propose the Separated Paths for Local and Global Information (SPLIT) framework that modifies standard VAE models, enabling them to disentangle global and local visual features explicitly. Our framework requires a subset of the latent variables to generate an auxiliary set of observable data that is obtained by a hand-crafted transformation. This additional generative assumption primes the latent variables to local information and encourages the other latent variables to represent global information. We find that our SPLIT models successfully learn representations that decouple global and local variations in classical datasets. Furthermore, we apply our framework to solve several downstream tasks: clustering, unsupervised object recognition, and visual reinforcement learning. The learned representations are found to be useful for these tasks and improve out-of-distribution generalization performances. The code for our experiments is at https://github.com/51616/split-vae.