You are absolutely correct, and because it doesn't scale well, you have to keep it highly targeted.
It all depends on what you are designing. Something DSP focused like a MJPEG encoder can be entirely simulated as it can be abstracted away from anything external. Most you would need would be a simple model of your external DRAM controller for a framebuffer. And this would definitely be both the fastest and easiest way to develop it.
On the other hand, I did a job where I had to design a secure enclave accessible as a typical LBA-accessible SD card through a standard SD reader. I designed a pcb specifically to let me sniff the protocol both with a real card and my own IP.
Of course, I started with implementing everything in the SD spec to the letter. It still required a crazy week of in situ debugging with dozens of card readers and cards to see why both ends violated the spec but worked anyway.
In the end, I was able to design the flash translation layer fully in simulation but the SD link and phy layer were developed almost fully with the signaltap debug and compile loop.
It all depends on what you are designing. Something DSP focused like a MJPEG encoder can be entirely simulated as it can be abstracted away from anything external. Most you would need would be a simple model of your external DRAM controller for a framebuffer. And this would definitely be both the fastest and easiest way to develop it.
On the other hand, I did a job where I had to design a secure enclave accessible as a typical LBA-accessible SD card through a standard SD reader. I designed a pcb specifically to let me sniff the protocol both with a real card and my own IP. Of course, I started with implementing everything in the SD spec to the letter. It still required a crazy week of in situ debugging with dozens of card readers and cards to see why both ends violated the spec but worked anyway. In the end, I was able to design the flash translation layer fully in simulation but the SD link and phy layer were developed almost fully with the signaltap debug and compile loop.