They are people who aren't very online, and their concern has nothing to do with "knowing he's unfit" and everything to do with his viability as a candidate for president.
I was surprised that every single person at the BBQ thought he needed to drop out! They aren't all lawyers, btw
There is and has been a backup plan for if Biden should die or be unable to govern for the last four years.
What if instead of freaking out in public for all the reasons mentioned, we wait 2 weeks & see how any appearances Biden makes go & whatever behind the scenes stuff plays out.