steeleduncan parent
The other conclusion to draw is "Git is a fantastic choice of database for starting your package manager, almost all popular package managers began that way."
I think the conclusion is more that package definitions can still be maintained on git/GitHub but the package manager clients should probably rely on a cache/db/a more efficient intermediate layer.
Mostly to avoid downloading the whole repo/resolve deltas from the history for the few packages most applications tend to depend on. Especially in today's CI/CD World.
This is exactly the right approach. I did this for my package manager.
It relies on a git repo branch for stable. There are yaml definitions of the packages including urls to their repo, dependencies, etc. Preflight scripts. Post install checks. And the big one, the signatures for verification. No binaries, rpms, debs, ar, or zip files.
What’s actually installed lives in a small SQLite database and searching for software does a vector search on each packages yaml description.
Semver included.
This was inspired by brew/portage/dpkg for my hobby os.
yaml definitions compiled to sqlite sounds pretty similar to winget, which is scaling very well. vector search is a cool idea - how much storage does it use per package?
This is how WinGet works. It has a small SQLite db it downloads from a hosted url. The DB contains some minimal metadata and a url path to access the full metadata. This way WinGet only has to make API calls for packages it's actually interacting with. As a package manager, it has plenty of problems still, but it's a simple, elegant solution for the git as a DB issue.
Indeed. Nixpkgs wouldn't have been as successful if it hadn't been using Git (or GitHub).
Sure, eventually you run into scaling issues, but that's a first world problem.
I actually find that nixpkgs being a monorepo makes it even better. The code is surprisingly easy to navigate and learn if you've worked in large codebases before. The scaling issues are good problems to have, and git has gotten significantly better at handling large repos than it was a decade ago, when Facebook opted for Mercurial because git couldn't scale to their needs. If anything, it's GitHub issues and PRs that are probably showing its cracks.
Git isn't a fantastic choice unless you know nothing about databases. A search would show plenty of research on databases and what works when/why.
For the purposes of the article, git isn't just being used as a database, it's being used as a protocol to replicate the database to the client to allow for offline operation and then keep those distributed copies in sync. And even for that purpose you can do better than git if you know what you're doing, but knowledge of databases alone isn't going to help you (let alone make your engineering more economical than relying on free git hosting).
Exactly. It's not just about the best solution to the problem, it's also heavily about the economics around it. If I wanted to create a new package manager today, I could get started by utilizing Git and existing git hosting solutions with very little effort, and effort translates to time, and time is a scarce resource. If you don't know whether your package manager will take off or not, it may not be the best use of your scarce resources to invest in a robust and optimized solution out of the gate. I wish that weren't the case, I would love to have an infinite amount of time, but wishing is not going to make it happen
Git is an absolute shit database for a package manager even in the beginning. It’s just that GitHub subsidizes hosting and that is hard to pass up.
What's a better option? One that keeps track of history and has a nice review interface?
Sure, but can you back up the expletive with some reason why you think that?
As it is, this comment is just letting out your emotion, not engaging in dialogue.
The article enumerates many such reasons.
Can we please stop the tone-policing, please? It's not helpful. Not everything needs Wikipedia-style citation, and this particular rhetorical trick is extremely passive-aggressive.
No. No, no, no. Git is a fantastic choice if you want a supply chain nightmare and then Leftpad every week forever.