This is the same argument you could use against any open-source projects. Software is much easier to open-source because the tools needed and barrier to entry are relatively minuscule. Hardware requires a lot of resources that take time and money to acquire. TSMC is fifty years ahead because they have had billions in research funds and have acquired the brightest minds of the past few generations, this still doesn’t mean that the technology of today is limited to highly advanced fabs the same way fifty years from now. Arguably all it takes is a dedicated team of highly-skilled hobbyists to make leaps toward open-source hardware more suitable for today’s requirements.
OP said hobbyists will never be able to make open-source hardware close to today’s scale, but it’s entirely possible for future generations to do just that.