<italic>DStore</italic>: A Holistic Key-Value Store Exploring Near-Data Processing and On-Demand Scheduling for Compaction Optimization
Log-structured merge tree (LSM-tree)-based key-value stores are widely deployed in largescale storage systems. The underlying reason is that the traditional relational databases cannot reach the high performance required by big-data applications. As high-throughput alternatives to relational databas...
Main Authors: | , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
IEEE
2018-01-01
|
Series: | IEEE Access |
Subjects: | |
Online Access: | https://ieeexplore.ieee.org/document/8481427/ |
id |
doaj-7c5ef7ea13f348b98791c260bec1bff9 |
---|---|
record_format |
Article |
spelling |
doaj-7c5ef7ea13f348b98791c260bec1bff92021-03-29T21:21:31ZengIEEEIEEE Access2169-35362018-01-016612336125310.1109/ACCESS.2018.28735798481427<italic>DStore</italic>: A Holistic Key-Value Store Exploring Near-Data Processing and On-Demand Scheduling for Compaction OptimizationHui Sun0https://orcid.org/0000-0003-1811-1318Wei Liu1Zhi Qiao2Song Fu3Weisong Shi4School of Computer Science and Technology, Anhui University, Hefei, ChinaSchool of Computer Science and Technology, Anhui University, Hefei, ChinaComputer Science and Engineering, University of North Texas, Denton, TX, USAComputer Science and Engineering, University of North Texas, Denton, TX, USADepartment of Computer Science, Wayne State University, Detroit, MI, USALog-structured merge tree (LSM-tree)-based key-value stores are widely deployed in largescale storage systems. The underlying reason is that the traditional relational databases cannot reach the high performance required by big-data applications. As high-throughput alternatives to relational databases, LSM-tree-based key-value stores can support high-throughput write operations and provide high sequential bandwidth in storage systems. However, the compaction process triggers write amplification and is confronted with the degraded write performance, especially under update-intensive workloads. To address this issue, we design a holistic key-value store to explorer near-data processing (NDP) and on-demand scheduling for compaction optimization in an LSM-tree key-value store, named DStore. DStore makes full use of various computing capacities in the host-side and device-side subsystems. DStore dynamically divides the whole host-side compaction tasks into the above two-side subsystems according to two-side different computing capabilities. Meanwhile, the device must be featured with an NDP model. The divided compaction tasks are performed by the host and the device in parallel. In DStore, the NDP-based devices exhibit low-latency and high-bandwidth performance, thus facilitating key-value stores. DStore not only accomplishes compaction for key-value stores but also improves the system performance. We implement our DStore prototype in a real-world platform, and different kinds of testbeds are employed in our experiment. LevelDB and a static compaction optimization using the NDP model (called Co-KV) are used to compare with the DStore in our evaluation. Results show that DStore achieves about 3.7× performance improvement over LevelDB under the db_bench workload. In addition, DStore-enabled key-value stores outperform LevelDB by a factor of about 3.3× and 77% in terms of throughput and latency under YCSB benchmark, respectively.https://ieeexplore.ieee.org/document/8481427/LSM-treekey-value storeon-demand schedulingnear-data processing |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Hui Sun Wei Liu Zhi Qiao Song Fu Weisong Shi |
spellingShingle |
Hui Sun Wei Liu Zhi Qiao Song Fu Weisong Shi <italic>DStore</italic>: A Holistic Key-Value Store Exploring Near-Data Processing and On-Demand Scheduling for Compaction Optimization IEEE Access LSM-tree key-value store on-demand scheduling near-data processing |
author_facet |
Hui Sun Wei Liu Zhi Qiao Song Fu Weisong Shi |
author_sort |
Hui Sun |
title |
<italic>DStore</italic>: A Holistic Key-Value Store Exploring Near-Data Processing and On-Demand Scheduling for Compaction Optimization |
title_short |
<italic>DStore</italic>: A Holistic Key-Value Store Exploring Near-Data Processing and On-Demand Scheduling for Compaction Optimization |
title_full |
<italic>DStore</italic>: A Holistic Key-Value Store Exploring Near-Data Processing and On-Demand Scheduling for Compaction Optimization |
title_fullStr |
<italic>DStore</italic>: A Holistic Key-Value Store Exploring Near-Data Processing and On-Demand Scheduling for Compaction Optimization |
title_full_unstemmed |
<italic>DStore</italic>: A Holistic Key-Value Store Exploring Near-Data Processing and On-Demand Scheduling for Compaction Optimization |
title_sort |
<italic>dstore</italic>: a holistic key-value store exploring near-data processing and on-demand scheduling for compaction optimization |
publisher |
IEEE |
series |
IEEE Access |
issn |
2169-3536 |
publishDate |
2018-01-01 |
description |
Log-structured merge tree (LSM-tree)-based key-value stores are widely deployed in largescale storage systems. The underlying reason is that the traditional relational databases cannot reach the high performance required by big-data applications. As high-throughput alternatives to relational databases, LSM-tree-based key-value stores can support high-throughput write operations and provide high sequential bandwidth in storage systems. However, the compaction process triggers write amplification and is confronted with the degraded write performance, especially under update-intensive workloads. To address this issue, we design a holistic key-value store to explorer near-data processing (NDP) and on-demand scheduling for compaction optimization in an LSM-tree key-value store, named DStore. DStore makes full use of various computing capacities in the host-side and device-side subsystems. DStore dynamically divides the whole host-side compaction tasks into the above two-side subsystems according to two-side different computing capabilities. Meanwhile, the device must be featured with an NDP model. The divided compaction tasks are performed by the host and the device in parallel. In DStore, the NDP-based devices exhibit low-latency and high-bandwidth performance, thus facilitating key-value stores. DStore not only accomplishes compaction for key-value stores but also improves the system performance. We implement our DStore prototype in a real-world platform, and different kinds of testbeds are employed in our experiment. LevelDB and a static compaction optimization using the NDP model (called Co-KV) are used to compare with the DStore in our evaluation. Results show that DStore achieves about 3.7× performance improvement over LevelDB under the db_bench workload. In addition, DStore-enabled key-value stores outperform LevelDB by a factor of about 3.3× and 77% in terms of throughput and latency under YCSB benchmark, respectively. |
topic |
LSM-tree key-value store on-demand scheduling near-data processing |
url |
https://ieeexplore.ieee.org/document/8481427/ |
work_keys_str_mv |
AT huisun italicdstoreitalicaholistickeyvaluestoreexploringneardataprocessingandondemandschedulingforcompactionoptimization AT weiliu italicdstoreitalicaholistickeyvaluestoreexploringneardataprocessingandondemandschedulingforcompactionoptimization AT zhiqiao italicdstoreitalicaholistickeyvaluestoreexploringneardataprocessingandondemandschedulingforcompactionoptimization AT songfu italicdstoreitalicaholistickeyvaluestoreexploringneardataprocessingandondemandschedulingforcompactionoptimization AT weisongshi italicdstoreitalicaholistickeyvaluestoreexploringneardataprocessingandondemandschedulingforcompactionoptimization |
_version_ |
1724193099900518400 |