During sampling, they propose to use a sequential sampling mechanism by adding a newly sampled dimension to the observed sets and running the network in an autoregressive manner. Previous attempts to learn probability distributions conditioned on arbitrary subsets of known covariates include the Universal Marginalizer, which is trained as a feed-forward network to approximate the marginal posterior distribution of each unobserved dimension conditioned on the observed ones. We describe them in detail below.įor notation simplicity, we drop the subscripts X and Z in the following sections. Since the missing pattern could be arbitrary, we require the transformation to learn a large range of possible dependencies.Īiming at solving those challenges, we propose several conditional transformations that leverage the conditioning information in x o and b and can be adapted to x u with arbitrary dimensionality. Another challenge is that different missing patterns require the transformation to capture different dependencies. One challenge comes from requiring the transformation to have adaptive outputs that can adapt to different dimensionality of x u. However, the fact that x u could have varying dimensionality and arbitrary missing dimensions makes it challenging to define q x o, b’s across different bitmasks b. Where q x o, b is a transformation on the unobserved covariates x u with respect to the observed covariates x o and binary mask b as demonstrated in Fig. Checkerboarded boxes in (b) belong to unobserved dimensions, but are used as conditioning in the affine coupling transformation. Grayed out boxes represent missing covariates. (a) general formulationįigure 1: Conditional transformations used in AC-Flow.
4) We run extensive empirical studies and show that AC-Flow achieves state-of-the-art performance for both missing feature imputation and image inpainting on benchmark real-world datasets. 3) We propose a novel penalty to generate a single imputed “best guess” for models without an analytically available mean. 2) We strengthen a flow-based model by using a novel autoregressive conditional likelihood. Our method is the first to develop invertible transformations that operate on an arbitrary set of covariates. 1) We propose a novel extension of flow-based generative models to model the conditional distribution of arbitrary unobserved covariates in data imputation tasks. We focus on the use of AC-Flow for the purpose of imputation, where we infer possible values of x u, given observed values x o both in general real-valued data and images (for inpainting). Yield tractable (analytically available) conditional likelihoods p ( x u ∣ x o ) of an arbitrary subset of covariates, x u, given the remaining observed covariates x o. In this work, we propose a framework, arbitrary conditioning flow models (AC-Flow), to construct generative models that However, most generative approaches are solely focused on the joint distribution of features, p ( x ), and are opaque in the conditional dependencies that are carried among subsets of features. Generative models have a multitude of potential applications, including image restoration, agent planning, and unsupervised representation learning. These models learn an approximation of the underlying data distribution and are capable of drawing realistic samples from it. Spurred on by recent impressive results, there has been a surge in interest for generative probabilistic modeling in machine learning. Imputation across image inpainting and feature imputation in synthetic and Our models achieve state-of-the-art performance in both single and multiple Extensive empirical evaluations show that Imputation by introducing an auxiliary objective that provides a principled We apply AC-Flow to the imputation ofįeatures, and also develop a unified platform for both multiple and single (AC-Flow), that can be conditioned on arbitrary subsets of observed covariates, Of variables based) flow generative models, arbitrary conditioning flow models Yielding all conditional distributions p(x_u | x_o) (for arbitrary x_u) Instead, in this work we develop a model that is capable of
Traditional conditional approaches provide a modelįor a fixed set of covariates conditioned on another fixed set of observedĬovariates. Models where it is intractable to obtain the conditional distribution of someĪrbitrary subset of features x_u given the rest of the observed covariates However, a majority of generative modelingĪpproaches are focused solely on the joint distribution p(x) and utilize Understanding the dependencies among features of a dataset is at the core of
0 kommentar(er)
