This is a totally valid concern. A mitigation is to have some other device or mechanism to generate a device that's part of sync fabric in escrow somewhere else. Of course, this introduces two new problems: Where do you keep it in escrow, and how do you authenticate to that escrow service given that your authenticators are gone. Safe deposit boxes are one alternative: They're authenticated by your presence and government-issued ID. Apple uses a 24-character key and keeps the encrypted record in its HSMs. Crypto wallets tend to use a 24-word recovery phrase to derive a key. These last two options you could also escrow with a third party or maybe even memorize.

For what it's worth, it's not an inherent problem with hardware backed security, at least not in the context that I was talking about. If you were using pure software implementations of WebAuthn, you could also authenticate to that sync fabric only using WebAuthn and you'd have that exact same problem you're describing.