Transactions of the Association for Computational Linguistics (Jan 2023)

Transformers for Tabular Data Representation: A Survey of Models and Applications

  • Gilbert Badaro,
  • Mohammed Saeed,
  • Paolo Papotti

DOI
https://doi.org/10.1162/tacl_a_00544
Journal volume & issue
Vol. 11
pp. 227 – 249

Abstract

Read online

AbstractIn the last few years, the natural language processing community has witnessed advances in neural representations of free texts with transformer-based language models (LMs). Given the importance of knowledge available in tabular data, recent research efforts extend LMs by developing neural representations for structured data. In this article, we present a survey that analyzes these efforts. We first abstract the different systems according to a traditional machine learning pipeline in terms of training data, input representation, model training, and supported downstream tasks. For each aspect, we characterize and compare the proposed solutions. Finally, we discuss future work directions.