The Witness is a 3d engine made from scratch. Not all AAA companies use Unreal or Unity.
Having a 3D engine does not a AAA make. The Witness is a beautiful looking game, but the amount of state and interactions it has to deal with is orders of magnitude less than GTA: San Andreas. It is closer to the complexity a Myst remake would have.
My impression is that the Blow/Muratori style works well if you're the only person working on a game, or part of a very limited team, which is fair enough, but it naturally limits the scope of what you can achieve.