Parallel t-SNE Applied to Data Visualization in Smart Cities

Abstract

The growth of smart city applications is increasingly around the world, many cities invest in the development of these systems intending to improve the management and life of their residents. This increase is mainly due to the emergence of new technologies such as Big Data and Internet of Things (IoT). Some of the biggest challenges in applying these systems, relate to the processing, visualization, and analysis of the generated data, since most systems tend to work connected, thus generating a large mass of data that deviates from the standard of previously used systems. For data visualization, one of the main devices used is the reduction of dimensionality, in an attempt to bring data from one dimension N to two or three dimensions and thus be noticeable to human eyes. There are several algorithms used for dimensionality reduction, the linear ones that as the name implies, solve linearly separable problems and so these are very limited and the nonlinear ones, that solve more complex problems, but usually have an excessive runtime, making them or often inappropriate to apply. This article presents the parallel implementation of the nonlinear dimension reduction algorithm t-Distributed Stochastic Neighbor Embedding (t-SNE), showing better results than its conventional version in terms of runtime, thus showing that parallelism can make the problem of dimensionality reduction treatable, bringing greater scalability and delivering results within an acceptable time frame.

Keywords

• Big data
• data visualization
• dimensionality reduction
• parallel t-sne
• smart cities