Statistics.
Not all smokers get lung cancer. But with large enough samples, smokers turn out to be much more likely to get lung cancer than non-smokers. That's why we say that smoking causes lung cancer. It's not as if someone has actually watched a particle of tobacco interact with someone's lung cells to turn them cancerous in real time.
Smoking isn't a contagious pathogen.
Second- and third-hand smoke mimic contagion.
That the root of the confusion for me. I haven't wrapped my head around how we can know a specific pathogen causes disease if a large number of people can be found to have that pathogen present without showing symptoms.