A STUDY OF SOUND GENERATION WITH TWO APPROACHES

• Main Author: TIAS KURNIATI
• Other Authors: Chuan-Kai Yang
• Format: Others
• Language: en_US
• Published: 2018
• Online Access: http://ndltd.ncl.edu.tw/handle/tgh5w8

碩士 === 國立臺灣科技大學 === 資訊管理系 === 106 === Nowadays, sound generation has become one of the directions in multimedia research. People are searching new methods to generate interesting sounds. Therefore, in this research we address the problem of making multimedia system for sound production from a given image through two different approaches including color based segmentation and object detection. We use jQuery audiosynth.js to generate the sound of notes in the color mapping sonification system, while YOLOv3 is used in object detection for sonification. The system will play suitable sound from a local database that matches with the object detected by the system. We choose to implement the systems in a web-based platform using JavaScript associated by node.js with modern web browsers that support Web Audio APIs. In this case, Mozilla Firefox and Google Chrome have already supported this feature. In addition, the web-based sonification system can still be used in different platforms such Android and Windows because it doesn’t depend on the chosen platform. The purpose of the research is to generate a pleasing sound for an image through two approaches presented. A user study was performed to evaluate the systems by using online programs and questionnaires. The results indicate that most of the users agree that the sonification systems presented were interesting and unique.