Abstract
Modern information retrieval systems use distributed and parallel algorithms to meet their operational requirements, and commonly operate on sparse vectors; but dimensionality-reducing techniques produce dense and relatively short feature vectors. Motivated by this relevance of dense vectors, we have parallelized the vector space model for dense matrices and vectors. Our algorithm uses a hybrid partitioning splitting documents and features and operates on a mesh of hosts holding a block partitioned corpus matrix. We show that the theoretic speed-up is optimal. The empirical evaluation of an MPI-based implementation reveals that we obtain a super-linear speed-up on a cluster using Nehalem Xeon CPUs.
Keywords
MSC
15-04, 65F99, 68P20
Cite this article
@article{cai-110200,
author = {Tobias Berka and Marian Vajteršic},
title = {Parallel Retrieval of Dense Vectors in the Vector Space Model},
journal = {Computing and Informatics},
volume = {30},
number = {2},
pages = {247--265},
year = {2011},
issn = {1335-9150},
url = {https://cai.type.sk/content/2011/2/parallel-retrieval-of-dense-vectors-in-the-vector-space-model/}
}
TY - JOUR AU - Tobias Berka AU - Marian Vajteršic TI - Parallel Retrieval of Dense Vectors in the Vector Space Model JO - Computing and Informatics VL - 30 IS - 2 SP - 247 EP - 265 PY - 2011 SN - 1335-9150 UR - https://cai.type.sk/content/2011/2/parallel-retrieval-of-dense-vectors-in-the-vector-space-model/ KW - Vector space model KW - symmetric multiprocessing KW - dense vector computations KW - message passing interface ER -
Berka, T., & Vajteršic, M. (2011). Parallel Retrieval of Dense Vectors in the Vector Space Model. Computing and Informatics, 30(2), 247–265. https://cai.type.sk/content/2011/2/parallel-retrieval-of-dense-vectors-in-the-vector-space-model/