After seeing the Docker image for vllm jump +5Gb (to 10Gb!) over the past five months, I grew suspicious of vllm's development practices [1]. It's not easy, for sure, to deal with all those flaky python modules [2].
But having the CUDA packages four times in different layers is questionable! [3]
Yet again, as a college mate of mine used to say, "Don't change it. It works."
But having the CUDA packages four times in different layers is questionable! [3]
Yet again, as a college mate of mine used to say, "Don't change it. It works."
--
[1]: https://hub.docker.com/r/vllm/vllm-openai/tags
[2]: https://github.com/vllm-project/vllm/issues/13306
[3]: These kinds of workarounds tend to end up accumulating and never get reviewed back:
- https://github.com/vllm-project/vllm/commit/b07d741661570ef1...
- https://github.com/vllm-project/vllm/commit/68d37809b9b52f4d... (this one in particular probably accounts for +3Gb)