Why does it seem that most 'rich' people come from backrounds where they had a hard childhood?

I was watching a show about 'rich' people, and it seemed like a vast majority of them had a ruff childhood (Immigrants, Abuse, Homeless). Never just your average Joe who had a pretty good childhood? Why is it that? P.S if this is offending to anyone, its certainly wasn't meant for it to be. Thanks.