I think you are spot on. What makes Chiang remarkable is that he never just builds a clever world and then leaves it at that. The counterfactuals are always put under pressure until their human consequences show through. That is why the religious stories work so well. He does not mock belief from the outside, he imagines a world where belief is literally true and then forces us to face the consequences. The horror comes from taking doctrines at face value and discovering that they are not comforting at all.

Exhalation is a good example of the same method applied to physics. The narrator dissects himself and his world with patient clarity, and in the process he reveals the same fragility that we face. The beauty of the story is that it does not rage against entropy or wrap it up in metaphors. It accepts the facts of decline and finds meaning in understanding them. That is why it feels sealed and perfect, as you say. The restraint is what gives it emotional weight.