Terrain Adaptive Walking of Biped Neuromuscular Virtual Human Using Deep Reinforcement Learning
There have been some biomechanics-based control systems that have achieved better realistic virtual human motion. Yet their abilities to adapt the changing environments are weaker than the traditional control systems with characters driven by proportional derivative actuators directly. In our method...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2019-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/8758114/ |
| id |
doaj-54704c9f65474e13885b370f0171f624 |
|---|---|
| record_format |
Article |
| spelling |
doaj-54704c9f65474e13885b370f0171f6242021-03-29T23:28:16ZengIEEEIEEE Access2169-35362019-01-017924659247510.1109/ACCESS.2019.29276068758114Terrain Adaptive Walking of Biped Neuromuscular Virtual Human Using Deep Reinforcement LearningJianpeng Wang0https://orcid.org/0000-0002-9716-1809Wenhu Qin1Libo Sun2https://orcid.org/0000-0002-7838-9410School of Instrument Science and Engineering, Southeast University, Nanjing, ChinaSchool of Instrument Science and Engineering, Southeast University, Nanjing, ChinaSchool of Instrument Science and Engineering, Southeast University, Nanjing, ChinaThere have been some biomechanics-based control systems that have achieved better realistic virtual human motion. Yet their abilities to adapt the changing environments are weaker than the traditional control systems with characters driven by proportional derivative actuators directly. In our method, we build a hierarchical neuromuscular virtual human (NMVH) motion control system that consists of a low-level spine reflex layer and a high-level policy control layer. The spine reflex layer uses a feedback net to map sensory information to excitations, which stimulate muscles to generate joint torques. The policy control layer includes a deep neural network, which provides a learned action policy to spine reflex layer for achieving terrain-adaptive motion skills. The particle swarm optimization algorithm is used to optimize the gain factors of the feedback net for finding out a basic policy to make the virtual human walk on the flat terrain autonomously. The proximal policy optimization algorithm is employed to train the deep neural network in policy control layer for learning how to modulate the actions to adapt to the changing terrain. The simulation results in Matlab show that virtual human can walk smoothly and better adapt to the given terrain changes. It demonstrates that our control system improves the terrain-adaptive walking skill of the neuromuscular virtual human.https://ieeexplore.ieee.org/document/8758114/Biped walkinghierarchical controlneuromuscular virtual humanproximal policy optimization |
| collection |
DOAJ |
| language |
English |
| format |
Article |
| sources |
DOAJ |
| author |
Jianpeng Wang Wenhu Qin Libo Sun |
| spellingShingle |
Jianpeng Wang Wenhu Qin Libo Sun Terrain Adaptive Walking of Biped Neuromuscular Virtual Human Using Deep Reinforcement Learning IEEE Access Biped walking hierarchical control neuromuscular virtual human proximal policy optimization |
| author_facet |
Jianpeng Wang Wenhu Qin Libo Sun |
| author_sort |
Jianpeng Wang |
| title |
Terrain Adaptive Walking of Biped Neuromuscular Virtual Human Using Deep Reinforcement Learning |
| title_short |
Terrain Adaptive Walking of Biped Neuromuscular Virtual Human Using Deep Reinforcement Learning |
| title_full |
Terrain Adaptive Walking of Biped Neuromuscular Virtual Human Using Deep Reinforcement Learning |
| title_fullStr |
Terrain Adaptive Walking of Biped Neuromuscular Virtual Human Using Deep Reinforcement Learning |
| title_full_unstemmed |
Terrain Adaptive Walking of Biped Neuromuscular Virtual Human Using Deep Reinforcement Learning |
| title_sort |
terrain adaptive walking of biped neuromuscular virtual human using deep reinforcement learning |
| publisher |
IEEE |
| series |
IEEE Access |
| issn |
2169-3536 |
| publishDate |
2019-01-01 |
| description |
There have been some biomechanics-based control systems that have achieved better realistic virtual human motion. Yet their abilities to adapt the changing environments are weaker than the traditional control systems with characters driven by proportional derivative actuators directly. In our method, we build a hierarchical neuromuscular virtual human (NMVH) motion control system that consists of a low-level spine reflex layer and a high-level policy control layer. The spine reflex layer uses a feedback net to map sensory information to excitations, which stimulate muscles to generate joint torques. The policy control layer includes a deep neural network, which provides a learned action policy to spine reflex layer for achieving terrain-adaptive motion skills. The particle swarm optimization algorithm is used to optimize the gain factors of the feedback net for finding out a basic policy to make the virtual human walk on the flat terrain autonomously. The proximal policy optimization algorithm is employed to train the deep neural network in policy control layer for learning how to modulate the actions to adapt to the changing terrain. The simulation results in Matlab show that virtual human can walk smoothly and better adapt to the given terrain changes. It demonstrates that our control system improves the terrain-adaptive walking skill of the neuromuscular virtual human. |
| topic |
Biped walking hierarchical control neuromuscular virtual human proximal policy optimization |
| url |
https://ieeexplore.ieee.org/document/8758114/ |
| work_keys_str_mv |
AT jianpengwang terrainadaptivewalkingofbipedneuromuscularvirtualhumanusingdeepreinforcementlearning AT wenhuqin terrainadaptivewalkingofbipedneuromuscularvirtualhumanusingdeepreinforcementlearning AT libosun terrainadaptivewalkingofbipedneuromuscularvirtualhumanusingdeepreinforcementlearning |
| _version_ |
1724189421504299008 |