HotCSE Seminar
Computational Science & Engineering
Wednesday Feb 22, 12pm-1pm, 1116-E Klaus

Accelerating Inference in Fully-Connected Layers

Marat Dukhan
Advisor: Prof. Richard Vuduc


The CANDECOMP/PARAFAC (CP) decomposition is a leading method for the analysis of multi-way data. The standard alternating least squares algorithm for the CP decomposition (CP-ALS) involves a series of highly overdetermined least squares problems. We show that, by extending randomized least squares ("sketching") methods to tensors, the workload of CP-ALS can be drastically reduced without a sacrifice in quality. We introduce techniques for efficiently preprocessing, sampling, and computing randomized least squares on a dense tensor of arbitrary order, as well as an efficient sampling-based technique for checking the stopping condition. We also show more generally that the Khatri-Rao product (used within the CP iteration) produces conditions favorable for direct sampling without any preprocessing. In numerical results, we see improvements in speed, storage, and robustness.


Marat is a Ph.D. student in CSE working with Richard Vuduc. His research interests are in high-performance computing, machine learning, and their interaction. He previously received his Bachelors in Applied Mathematics and Physics from the Moscow Institute of Physics and Technology and a Masters in Economics from the New Economic School in Moscow.