Do Randomized Algorithms Improve the Efficiency of Minimal Learning Machine?

• Main Authors: Joakim Linja, Joonas Hämäläinen, Paavo Nieminen, Tommi Kärkkäinen
• Format: Article
• Language: English
• Published: MDPI AG 2020-11-01
• Series: Machine Learning and Knowledge Extraction
• Subjects: machine learning; supervised learning; distance–based regression; minimal learning machine; approximate algorithms; ordinary least–squares
• Online Access: https://www.mdpi.com/2504-4990/2/4/29

Minimal Learning Machine (MLM) is a recently popularized supervised learning method, which is composed of distance-regression and multilateration steps. The computational complexity of MLM is dominated by the solution of an ordinary least-squares problem. Several different solvers can be applied to the resulting linear problem. In this paper, a thorough comparison of possible and recently proposed, especially randomized, algorithms is carried out for this problem with a representative set of regression datasets. In addition, we compare MLM with shallow and deep feedforward neural network models and study the effects of the number of observations and the number of features with a special dataset. To our knowledge, this is the first time that both scalability and accuracy of such a distance-regression model are being compared to this extent. We expect our results to be useful on shedding light on the capabilities of MLM and in assessing what solution algorithms can improve the efficiency of MLM. We conclude that <inline-formula><math display="inline"><semantics><mrow><mo>(</mo><mi>i</mi><mo>)</mo></mrow></semantics></math></inline-formula> randomized solvers are an attractive option when the computing time or resources are limited and <inline-formula><math display="inline"><semantics><mrow><mo>(</mo><mi>i</mi><mi>i</mi><mo>)</mo></mrow></semantics></math></inline-formula> MLM can be used as an out-of-the-box tool especially for high-dimensional problems.