Abstract: The recent rapid growth of the dimension of many datasets means that many
approaches to dimension reduction (DR) have gained significant attention.
High-performance DR algorithms are required to make data analysis feasible for
big and fast data sets. However, many traditional DR techniques are challenged
by truly large data sets. In particular multidimensional scaling (MDS) does not
scale well. MDS is a popular group of DR techniques because it can perform DR
on data where the only input is a dissimilarity function. However, common
approaches are at least quadratic in memory and computation and, hence,
prohibitive for large-scale data.
We propose an out-of-sample embedding (OSE) solution to extend the MDS
algorithm for large-scale data utilising the embedding of only a subset of the
given data. We present two OSE techniques: the first based on an optimisation
approach and the second based on a neural network model. With a minor trade-off
in the approximation, the out-of-sample techniques can process large-scale data
with reasonable computation and memory requirements. While both methods perform
well, the neural network model outperforms the optimisation approach of the OSE
solution in terms of efficiency. OSE has the dual benefit that it allows fast
DR on streaming datasets as well as static databases.