Published July 31, 2021
| Version v1
Dataset
Open
Wikipedia Compressed SQLite database
Creators
Description
Starting point:
We downloaded the 40.3 GiB SQLite database of English Wikipedia and created a compressed version of it with sqlite_zstd_vfs, our read/write Zstandard compression layer for SQLite3. The compressed version is 10.4 GiB (26%) and remains randomly accessible over the web in the spirit of the original (although we don't yet have a WebAssembly build of the compression layer).
sqlite_zstd_vfs is a building block of our Genomics Extension for SQLite, which bundles it along with other domain-specific features. This demonstrates a general-purpose use case of the compression layer intended for large-scale genomics data storage.
Files
Files
(11.2 GB)
Name | Size | Download all |
---|---|---|
md5:eb73fb7b3a21a326a5ab780280b7af32
|
11.2 GB | Download |
md5:8e77f40455649859290286713b579f19
|
8.6 MB | Download |