This is the mobile-friendly web version of the original article.

Performance Comparison of Operations in the File System and in Embedded Key-Value Databases

Jesse Hines School of Computing, Southern Adventist University, Collegedale, USA, *jessehines ‘at’ southern.edu

Nicholas Cunningham School of Computing, Southern Adventist University, Collegedale, USA, nicholascunningham ‘at’ southern.edu

German H. Alf´erez ´ School of Computing, Southern Adventist University, Collegedale, USA harveya ‘at’ southern.edu

Video: Southern Adventist University, An Honest Campus Tour

1. Abstract—A common scenario when developing local PC applications such as games, mobile apps, or presentation software is storing many small files or records as application data and needing to retrieve and manipulate those records with some unique ID. In this kind of scenario, a developer has the choice of simply saving the records as files with their unique ID as the filename or using an embedded on-disk key-value database. Many file systems have performance issues when handling large numbers of small files, but developers may want to avoid a dependency on an embedded database if it offers little benefit or has a detrimental effect on performance for their use case. Despite the need for benchmarks to enable informed answers to this design decision, little research has been done in this area. Our contribution is the comparison and analysis of the performance for the insert, update, get, and remove operations and the space efficiency of storing records as files vs. using key-value embedded databases including SQLite3, LevelDB, RocksDB, and Berkeley DB.
2. Index Terms—databases, file systems, database performance

Abstract—A common scenario when developing local PC applications such as games, mobile apps, or presentation software is storing many small files or records as application data and needing to retrieve and manipulate those records with some unique ID. In this kind of scenario, a developer has the choice of simply saving the records as files with their unique ID as the filename or using an embedded on-disk key-value database. Many file systems have performance issues when handling large numbers of small files, but developers may want to avoid a dependency on an embedded database if it offers little benefit or has a detrimental effect on performance for their use case. Despite the need for benchmarks to enable informed answers to this design decision, little research has been done in this area. Our contribution is the comparison and analysis of the performance for the insert, update, get, and remove operations and the space efficiency of storing records as files vs. using key-value embedded databases including SQLite3, LevelDB, RocksDB, and Berkeley DB.

Index Terms—databases, file systems, database performance