This seems to be a key value store built atop object storage. Which is to say, it seems completely redundant. Not sure if there's some feature I'm missing, but all of the six features mentioned on the front page are things you'd have if you used the key value store directly (actually, you get more because then you get multiple writers).
I was excited at first and thought this was SQL atop S3 et al. I've jerryrigged a solution to this using SQLite with a customized VFS backend, and would suggest that as an alternative to this particular project. You get the benefit of ACID transactions across multiple tables and a distributed backend.
This is a low-level embedded db that would be used by sql databases/query engines/streaming engines/etc rather than something that's likely to make sense for you to use directly. It sits in a similar space to RocksDB and LMDB.
You generally can't use object storage directly for this stuff; if you have a high volume of writes, it's incredibly slow (and expensive) to write them individually to s3. Similarly, on the read side you want to be able to cache data on local disk & memory to reduce query latency and cost.
Using an s3 object per key would be too expensive for many use cases.
The website is a bit fancy but the readme seems to pretty straightforwardly explain why you might want this. It seems to me like a nice little (13k loc) project that doesn't fit my needs but might come in handy for someone else?
This seems to be a key value store built atop object storage. Which is to say, it seems completely redundant. Not sure if there's some feature I'm missing, but all of the six features mentioned on the front page are things you'd have if you used the key value store directly (actually, you get more because then you get multiple writers).
I was excited at first and thought this was SQL atop S3 et al. I've jerryrigged a solution to this using SQLite with a customized VFS backend, and would suggest that as an alternative to this particular project. You get the benefit of ACID transactions across multiple tables and a distributed backend.
This is a low-level embedded db that would be used by sql databases/query engines/streaming engines/etc rather than something that's likely to make sense for you to use directly. It sits in a similar space to RocksDB and LMDB.
You generally can't use object storage directly for this stuff; if you have a high volume of writes, it's incredibly slow (and expensive) to write them individually to s3. Similarly, on the read side you want to be able to cache data on local disk & memory to reduce query latency and cost.
Using an s3 object per key would be too expensive for many use cases.
The website is a bit fancy but the readme seems to pretty straightforwardly explain why you might want this. It seems to me like a nice little (13k loc) project that doesn't fit my needs but might come in handy for someone else?
If you want SQLite backed by S3, maybe something like SQLite in :memory: mode with Litestream would work?
Edit: actually not sure if you can use :memory: mode since Litestream uses the WAL (IIRC), so maybe a ramfs instead
There are many solutions. The particular example I was using SQLite via webassembly and then resorting to HTTP's fetch api for a read-only solution.