There’s a trend of doing impressive things by pushing many of the hard problems into S3 and assuming S3 “just works”, then not bothering to explain how S3 works. I guess we do the same thing all the time with other miraculous solutions to hard problems, like assuming that superscalar out-of-order processors “just work”, but in cases like this it makes for an unsatisfying explanation. Especially because S3 is a proprietary product, not an algorithm.
The product is proprietary, but the interface and semantics are documented and freely available. "S3-compatible object store" is a commodity at this point, so it makes sense to treat it as a primitive building block for a distributed system. Of course, each implementation will have its own set of idiosyncrasies and limitations, so you'll still have to make sure that any such divergent behavior doesn't interfere with your own system requirements.
I'd argue that S3-compatible isn't exactly what you'd need though. You'd need S3-compatible with similar latency characteristics for read and write, with four nines of availability and 10+ nines for durability. [0]. Their metrics are, quite honestly, absolutely impossible for most engineering teams and companies to ever achieve, no matter how hard they work at it. In essence, what I mean is Cursor could migrate Origin out of Amazon's S3, but they'd lose a lot of reliability and/or performance in the process.
Less of a commodity at this point than a few years ago, what with Minio immolating itself to be reborn as, excuse my French, AIStor.
Which is not to say that TFA’s approach is invalid or uninteresting, but the mechanics of how “an S3” works are really obscure to those of us not in the know.
If you're looking specifically for self hosted S3-compatible object stores, then yes, the selection of available options is smaller. But an object store's ability to self-host is orthogonal to its compatibility with S3's interface and semantics.
This was the thing that caught my attention. It does feel like a lot of "scaling git was hard and complicated, so we're just offloading a significant to a different layer"
I've got no dog in this fight and no specific rationale, but it smells like a shortcut that's going to make some assumptions that might surprise later (maybe not at the global level, but a local level)
at this point, s3 just means object storage and doesnt mean it actually has to be hosted on aws. theres plenty of other companies that provide s3 compatible storage apis.
Last time I looked many of the alternatives didn't have as good support for some of the transaction-friendly things like conditional writes.
Hopefully if you're trying to build something complex on top of S3 and have any expectation of potentially needing to run on top of other infra in the future, you know which magic features you need and can see which of them are supported by what vendors/OSS projects. :)
But it's not really that different than doing cool stuff by pushing a bunch of things into Postgres or Kafka or [other thing] and not explaining how those things work.
Yes a good object storage service provider would. It all depends on what you are optimizing for. If you are only optimizing for cost then there are several cheap options that probably don't make sense for hot or warm storage. But if you care about availability, consistency, and latency guarantees then there are different service providers available.
