We just have to get a little more fed up with all of these services and then the initial cost of setting it up in the first place will be worth it. Any day now...
For example, it's important that I can add this comment, and I can't delete your comment, but the moderators can. The "storage" software would have to know something about the business logic of a web forum to make that happen.
If I want to leave a comment on an article and then delete it, I can publish the comment in my pod, and I can also publish the deletion. If you've got your browser in a mode where it's interested in my comments, it can pull in the data from my pod and render it in context with the article--whether or not the article's author cared to provide a comments section.
If you drop the idea that anyone is authoritative about how it all comes together on the viewer's screen, you can also dispense with the headaches of being that authority (e.g. services).
K8s PersistentVolume is another decent shot at storage abstraction, only a bit raw.
Finally, more and more tools expect you to have a Postgres they can plug into as backend.
All above assumes you want to treat the library data as a big unknown blob. Once data start being corrupted and need bespoke repair, things are less fun. Access and retention is another fun rabbit hole.
Data is complicated.
That's more or less a description of SQL.
A clean room implementation would likely yield different results but there appears to be some appetite for a solution.
So, until some years ago there was complete nonsense and anarchy in Linux networking management. That is until we got the "ip" program. There's still nonsense and anarchy, because the "ip" program doesn't cover everything, but it's on the right track to organize everything Linux knows about networking under one roof. So, vendors today, like, say, Melanox (i.e. NVidia) choose to interface with "ip" and work with that stack rather than invent their own interfaces.
When it's extendable in predictable and convenient ways, user will extend and enrich functionality.
Now, compare this to Linux storage... I want to scream and kill somebody every time I have to deal with any aspect of it because of how poorly mismanaged it is. There's no uniformity, plenty of standards where at most one is necessary, duplication upon duplication, layers... well, forget layers. Like, say, you wanted a RAID0, well, you have MD RAIDs, you have LVM RAIDs, you have ZFS RAIDs, you have multipassing with DM (is that a RAID, well sorta' depends on what you expected...) also, well, Ceph RDB are also kind of like RAIDs, DRBD can also sort of be like a RAID...
Do you maybe also want snapshots? How about encryption? -- Every solution will end up so particularly tailored to the needs of your organization that even an experienced admin in this very area your org is specializing will have no clue what's going on with your storage.
Needs can be studied, understood, catalogued, rolled into some sort of a hierarchy or some other structure amenable to management. We haven't solved this problem. But we have an even bigger one: no coordination and no desire to coordinate even within Linux core components, forget third-party vendors.
For the comparison being referenced here, if you want to compare RAM, the storage medium that backs compute, to modern persistent storage, here it is:
* 40 GB/s per DIMM vs 5-10 GB/s per NVMe SSD. At most one order of magnitude off, but you can pack enough disks into a computer that the throughput ratio is almost 1:1. AWS EBS is about 1 order of magnitude different here, and that is with network-attached storage.
* 100-200 ns latency (RAM) vs 10-50 us (fast SSD) - about 2 orders of magnitude, but also possible to hide with batching.
Personally, I am working on a modern Python ORM for PostgreSQL and PostgreSQL alone.
If the library is designed to send sql to another storage library...
No, not all services come connected with a database. Alternatively, often times a database is an artifact of tenancy and the need to manage users which would not be needed, had the functionality be exposed as a library.
More importantly, whether users realize this or not, a library is more beneficial for them than a service in majority of cases. Much in the same way how it's almost always better to own something than to rent it.
Just to give some examples of the above: all the Internet of crap stuff, all sorts of "smart" home nonsense which requires that you subscribe to a service, install an app on your phone and send all your private data unsupervised to some shady Joe Shmo who you know nothing about. To be more specific, take something like Google Nest thermostat. There's no reason this contraption should ever go on the Internet, nor should it require to know your street address, nor your email etc. In fact, the utility it brings is very marginal (saves you few steps you'd have to make to reach for the boiler's controls to program it). It absolutely could've been designed in such a way that it doesn't connect to the Internet, or, at least, to never leave the local area network, and yet it's a cloud service...
The library uses only the interface to work with whatever orm/db connector exists in the client project.
If services at any given company all use a standard db library, it could even directly interface assuming your using that. I don't think we're talking about public apis and packages here.
zeromq is a library
That’s all the storage you need