Adam Smith's invisible hand seems to be right in many respects, but a huge problem with capitalist theory comes from psychology and mental illness. The problem is how malignant narcissists become obsessed with accumulating money at any cost. As these psychopaths lack empathy and humanity, they have no compunction against abusing or enslaving people to boost their wealth. This includes corrupting government systems. The wealthy class largely consist of these psychopaths, and society hasn't figured out a way to rein them in. Medical science doesn't have any solutions for narcissism and society has no means to force a treatment if it existed. Thus, they go about destroying society because they can.
I've always thought that people put way too much importance on "The invisible hand" mentioned by Adams, as he only mentioned it once. Personally, I see it as a cheat, where he didn't actually know the real cause and forces behind certain economic movements, so he just made up the idea of the invisible hand, which kind of smacks of supernatural forces at work, which I don't believe in at all.
I pretty much agree with the points you made though.