<italic>DStore</italic>: A Holistic Key-Value Store Exploring Near-Data Processing and On-Demand Scheduling for Compaction Optimization

Log-structured merge tree (LSM-tree)-based key-value stores are widely deployed in largescale storage systems. The underlying reason is that the traditional relational databases cannot reach the high performance required by big-data applications. As high-throughput alternatives to relational databas...

Full description

Bibliographic Details
Main Authors: Hui Sun, Wei Liu, Zhi Qiao, Song Fu, Weisong Shi
Format: Article
Language:English
Published: IEEE 2018-01-01
Series:IEEE Access
Subjects:
Online Access:https://ieeexplore.ieee.org/document/8481427/
id doaj-7c5ef7ea13f348b98791c260bec1bff9
record_format Article
spelling doaj-7c5ef7ea13f348b98791c260bec1bff92021-03-29T21:21:31ZengIEEEIEEE Access2169-35362018-01-016612336125310.1109/ACCESS.2018.28735798481427<italic>DStore</italic>: A Holistic Key-Value Store Exploring Near-Data Processing and On-Demand Scheduling for Compaction OptimizationHui Sun0https://orcid.org/0000-0003-1811-1318Wei Liu1Zhi Qiao2Song Fu3Weisong Shi4School of Computer Science and Technology, Anhui University, Hefei, ChinaSchool of Computer Science and Technology, Anhui University, Hefei, ChinaComputer Science and Engineering, University of North Texas, Denton, TX, USAComputer Science and Engineering, University of North Texas, Denton, TX, USADepartment of Computer Science, Wayne State University, Detroit, MI, USALog-structured merge tree (LSM-tree)-based key-value stores are widely deployed in largescale storage systems. The underlying reason is that the traditional relational databases cannot reach the high performance required by big-data applications. As high-throughput alternatives to relational databases, LSM-tree-based key-value stores can support high-throughput write operations and provide high sequential bandwidth in storage systems. However, the compaction process triggers write amplification and is confronted with the degraded write performance, especially under update-intensive workloads. To address this issue, we design a holistic key-value store to explorer near-data processing (NDP) and on-demand scheduling for compaction optimization in an LSM-tree key-value store, named DStore. DStore makes full use of various computing capacities in the host-side and device-side subsystems. DStore dynamically divides the whole host-side compaction tasks into the above two-side subsystems according to two-side different computing capabilities. Meanwhile, the device must be featured with an NDP model. The divided compaction tasks are performed by the host and the device in parallel. In DStore, the NDP-based devices exhibit low-latency and high-bandwidth performance, thus facilitating key-value stores. DStore not only accomplishes compaction for key-value stores but also improves the system performance. We implement our DStore prototype in a real-world platform, and different kinds of testbeds are employed in our experiment. LevelDB and a static compaction optimization using the NDP model (called Co-KV) are used to compare with the DStore in our evaluation. Results show that DStore achieves about 3.7&#x00D7; performance improvement over LevelDB under the db_bench workload. In addition, DStore-enabled key-value stores outperform LevelDB by a factor of about 3.3&#x00D7; and 77% in terms of throughput and latency under YCSB benchmark, respectively.https://ieeexplore.ieee.org/document/8481427/LSM-treekey-value storeon-demand schedulingnear-data processing
collection DOAJ
language English
format Article
sources DOAJ
author Hui Sun
Wei Liu
Zhi Qiao
Song Fu
Weisong Shi
spellingShingle Hui Sun
Wei Liu
Zhi Qiao
Song Fu
Weisong Shi
<italic>DStore</italic>: A Holistic Key-Value Store Exploring Near-Data Processing and On-Demand Scheduling for Compaction Optimization
IEEE Access
LSM-tree
key-value store
on-demand scheduling
near-data processing
author_facet Hui Sun
Wei Liu
Zhi Qiao
Song Fu
Weisong Shi
author_sort Hui Sun
title <italic>DStore</italic>: A Holistic Key-Value Store Exploring Near-Data Processing and On-Demand Scheduling for Compaction Optimization
title_short <italic>DStore</italic>: A Holistic Key-Value Store Exploring Near-Data Processing and On-Demand Scheduling for Compaction Optimization
title_full <italic>DStore</italic>: A Holistic Key-Value Store Exploring Near-Data Processing and On-Demand Scheduling for Compaction Optimization
title_fullStr <italic>DStore</italic>: A Holistic Key-Value Store Exploring Near-Data Processing and On-Demand Scheduling for Compaction Optimization
title_full_unstemmed <italic>DStore</italic>: A Holistic Key-Value Store Exploring Near-Data Processing and On-Demand Scheduling for Compaction Optimization
title_sort <italic>dstore</italic>: a holistic key-value store exploring near-data processing and on-demand scheduling for compaction optimization
publisher IEEE
series IEEE Access
issn 2169-3536
publishDate 2018-01-01
description Log-structured merge tree (LSM-tree)-based key-value stores are widely deployed in largescale storage systems. The underlying reason is that the traditional relational databases cannot reach the high performance required by big-data applications. As high-throughput alternatives to relational databases, LSM-tree-based key-value stores can support high-throughput write operations and provide high sequential bandwidth in storage systems. However, the compaction process triggers write amplification and is confronted with the degraded write performance, especially under update-intensive workloads. To address this issue, we design a holistic key-value store to explorer near-data processing (NDP) and on-demand scheduling for compaction optimization in an LSM-tree key-value store, named DStore. DStore makes full use of various computing capacities in the host-side and device-side subsystems. DStore dynamically divides the whole host-side compaction tasks into the above two-side subsystems according to two-side different computing capabilities. Meanwhile, the device must be featured with an NDP model. The divided compaction tasks are performed by the host and the device in parallel. In DStore, the NDP-based devices exhibit low-latency and high-bandwidth performance, thus facilitating key-value stores. DStore not only accomplishes compaction for key-value stores but also improves the system performance. We implement our DStore prototype in a real-world platform, and different kinds of testbeds are employed in our experiment. LevelDB and a static compaction optimization using the NDP model (called Co-KV) are used to compare with the DStore in our evaluation. Results show that DStore achieves about 3.7&#x00D7; performance improvement over LevelDB under the db_bench workload. In addition, DStore-enabled key-value stores outperform LevelDB by a factor of about 3.3&#x00D7; and 77% in terms of throughput and latency under YCSB benchmark, respectively.
topic LSM-tree
key-value store
on-demand scheduling
near-data processing
url https://ieeexplore.ieee.org/document/8481427/
work_keys_str_mv AT huisun italicdstoreitalicaholistickeyvaluestoreexploringneardataprocessingandondemandschedulingforcompactionoptimization
AT weiliu italicdstoreitalicaholistickeyvaluestoreexploringneardataprocessingandondemandschedulingforcompactionoptimization
AT zhiqiao italicdstoreitalicaholistickeyvaluestoreexploringneardataprocessingandondemandschedulingforcompactionoptimization
AT songfu italicdstoreitalicaholistickeyvaluestoreexploringneardataprocessingandondemandschedulingforcompactionoptimization
AT weisongshi italicdstoreitalicaholistickeyvaluestoreexploringneardataprocessingandondemandschedulingforcompactionoptimization
_version_ 1724193099900518400