I think Navy seals are over estimated in their skills and that is probably more to do with daft Hollywood films building them up as some kind of super human when they clearly are not. I would have to say that I would expect a cave man to be much stronger than a modern day man.
Seeing as Humans lived out in the open or under the cover of trees (depending on where they lived) until H. Sapiens came along, I don't see how that's possible. What I'm trying to say is cavemen and modern humans are 1 and the same, it's a difference in tech.