UPI: A Primary Index for Uncertain Databases

Uncertain data management has received growing attention from industry and academia. Many efforts have been made to optimize uncertain databases, including the development of special index data structures. However, none of these efforts have explored primary (clustered) indexes for uncertain databas...

Full description

Bibliographic Details
Main Authors: Kimura, Hideaki (Author), Madden, Samuel R. (Contributor), Zdonik, Stanley B. (Contributor)
Other Authors: Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science (Contributor)
Format: Article
Language:English
Published: Very Large Data Base Endowment Inc. (VLDB Endowment), 2012-09-27T15:52:19Z.
Subjects:
Online Access:Get fulltext
LEADER 02761 am a22002413u 4500
001 73422
042 |a dc 
100 1 0 |a Kimura, Hideaki  |e author 
100 1 0 |a Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science  |e contributor 
100 1 0 |a Madden, Samuel R.  |e contributor 
100 1 0 |a Zdonik, Stanley B.  |e contributor 
700 1 0 |a Madden, Samuel R.  |e author 
700 1 0 |a Zdonik, Stanley B.  |e author 
245 0 0 |a UPI: A Primary Index for Uncertain Databases 
260 |b Very Large Data Base Endowment Inc. (VLDB Endowment),   |c 2012-09-27T15:52:19Z. 
856 |z Get fulltext  |u http://hdl.handle.net/1721.1/73422 
520 |a Uncertain data management has received growing attention from industry and academia. Many efforts have been made to optimize uncertain databases, including the development of special index data structures. However, none of these efforts have explored primary (clustered) indexes for uncertain databases, despite the fact that clustering has the potential to offer substantial speedups for non-selective analytic queries on large uncertain databases. In this paper, we propose a new index called a UPI (Uncertain Primary Index) that clusters heap files according to uncertain attributes with both discrete and continuous uncertainty distributions. Because uncertain attributes may have several possible values, a UPI on an uncertain attribute duplicates tuple data once for each possible value. To prevent the size of the UPI from becoming unmanageable, its size is kept small by placing low-probability tuples in a special Cutoff Index that is consulted only when queries for low-probability values are run. We also propose several other optimizations, including techniques to improve secondary index performance and techniques to reduce maintenance costs and fragmentation by buffering changes to the table and writing updates in sequential batches. Finally, we develop cost models for UPIs to estimate query performance in various settings to help automatically select tuning parameters of a UPI. We have implemented a prototype UPI and experimented on two real datasets. Our results show that UPIs can significantly (up to two orders of magnitude) improve the performance of uncertain queries both over clustered and unclustered attributes. We also show that our buffering techniques mitigate table fragmentation and keep the maintenance cost as low as or even lower than using an unclustered heap file. 
520 |a National Science Foundation (U.S.) (Grant IIS-0448124) 
520 |a National Science Foundation (U.S.) (Grant IIS-0905553) 
520 |a National Science Foundation (U.S.) (Grant IIS-0916691) 
546 |a en_US 
655 7 |a Article 
773 |t Proceedings of the VLDB Endowment