Information Theory and Machine Learning

• Format: eBook
• Language: English
• Published: MDPI - Multidisciplinary Digital Publishing Institute 2022
• Subjects: History of engineering and technology; Technology: general issues; analytical error probability; anomaly detection; artificial general intelligence; atypicality; closed-loop transcription; component overlap; deep neural network; distribution and federated learning; empirical risk; entropy; fairness; feature extraction; finite state machines; generalization error; HGR maximal correlation; hidden Markov models; independence criterion; independent and non-identically distributed features; information criteria; information theoretic learning; information theory; information-theoretic bounds; interpretability; K-means clustering; Lempel-Ziv algorithm; linear discriminative representation; local information geometry; long short-term memory; lossless compression; merging mixture components; meta-learning; minimax game; minimum error entropy; model compression; model-based clustering; overfitting; pattern dictionary; population risk; rate distortion theory; rate reduction; recurrent neural networks; reservoir computers; separation criterion; spiking neural network; supervised classification; time series prediction; vector quantization
• Online Access: Open Access: DOAB: description of the publication; Open Access: DOAB, download the publication

 The recent successes of machine learning, especially regarding systems based on deep neural networks, have encouraged further research activities and raised a new set of challenges in understanding and designing complex machine learning algorithms. New applications require learning algorithms to be distributed, have transferable learning results, use computation resources efficiently, convergence quickly on online settings, have performance guarantees, satisfy fairness or privacy constraints, incorporate domain knowledge on model structures, etc. A new wave of developments in statistical learning theory and information theory has set out to address these challenges. This Special Issue, "Machine Learning and Information Theory", aims to collect recent results in this direction reflecting a diverse spectrum of visions and efforts to extend conventional theories and develop analysis tools for these complex machine learning systems.