Some opponents of ev pscyh go so far as to state that the larger size and strength of human males did not evolve to be that way. The idea that evolution didn't determine the relative sizes of our ancestors a million years ago contradicts evolutionary theory.
The default mammal body is female. For most mammals, the males have evolved to be bigger and stronger than females. Being bigger comes at a big price, first for the mother, who has to provide more nutrients for the unborn male and more milk for the juvenile. The cost carries on to the male, who needs to eat more his whole life. Being bigger is expensive. Any mother who could produce males that were of regular size and strength (that is, like females), could produce more of them faster and cheaper, and the individual males would fare better maintaining a regular-size body. When organism pays a price, evolution must be offering some benefit that makes the price worth it. For humans living a million years ago, greater male size couldn't have been merely a benign vestige of our prehuman past. Greater male size is too expensive to be vestigial. The difference in male size represents evolution in action, for bull elephant seals and for humans.
So why bigger males? If males and females, on average, have the same number of young, why spend all this effort in making one sex bigger? The answer reveals an even larger phenomenon. Thanks to the different dynamic of reproduction between males (who provide sperm) and females (who provide egg, womb, nutrients for fetus, milk for young, parental care), males are a high-risk, high-reward endeavor, while females are more of a safe bet.
Female reproductive outcomes are more even than for males. Females bring so much to the reproductive table (egg, womb, nutrients, milk, care) that most females mate about as often as they can and bear about as many young as they can. Fitter females will do better, but it's common for them to do OK and really hard to do tremendously well.
Male mammals, on the other hand, promise little other than their sperm. They are eager to mate because it's such a freaking good deal for them. "I get laid. She raises the kids. OK!" The females are choosy. If the only thing they're getting is sperm, that had better be some high-quality semen, Mister! A mammal male has basically nothing to offer that the next male can't provide. If he never mates, no one will miss his contribution because he basically doesn't contribute.
So males compete. Rams buck each other, bucks ram each other, lions fight over prides of lionesses, etc. Bigger males generally defeat smaller ones. The males that "win" come out way ahead, mating with plenty of females and having way more than their share of young (genes) in the next generation. Aggression evolved along with greater size. There's little point for a male to be big enough to win a fight if he's not in a mood to fight and win.
In this mating economy, bigger males do far better than average males. A successful male can mate with multiple females while an unsuccessful male mates with none. The stakes are high. The pressure is on. There's no point in bearing male offspring that are average size. They'll just get outcompeted and left behind. So mammals are in something of an arms race, with those who can breed big strong males doing far better than those who can breed average males.
With this dynamic in place, mammals are stuck breeding bigger males just to stay even with their conspecifics (other organisms of the same species). The costs of bearing bigger males are so high that there's a limit to how big males get, but bull elephant seals show that the promise of tremendous mating opportunity is enough to drive the males of some species really big.
Biologists track a correlation between male competition and male size. The fiercer the competition, the bigger the males. You can look at our ancestors from a million years ago and see that our males were getting relatively smaller. Humans invented the pair-bond (love). Pair-bonding allows the male to offer something other than sperm to his offspring, gets more males into the mating economy, relieves pressure on male size. Human males are less divergent from the female norm than our ancestral males were because the male sex strategy is more like the females: less competition, more parental investment.
But we did stick with some physical differentiation. For a million years, it made sense to put a little more nutrient and time into males than into females, making them a little bigger and stronger. And there's still a bigger difference between a top male breeder and an average one than there is between a top female breeder and an average one. As long as that difference is there, evolution will favor the males that compete the best.
Like I said, there's a bigger phenomenon here than just size. While pair-bonding has made males act more like females (esp. care for young), the basic mammal sex economy is still in effect. Raising females is a safe bet. You don't have to put extra resources into daughters to make them valuable and successful. They have wombs and mammary glands, making even an unexceptional female valuable for rearing the next generation. A female that plays it safe will still bear young. But for males, the rewards of being at the top of the leader board in mating success are really big. Statistically and genetically, it's worth some amount of real risk on a male's part to score multiple mates. So males are the high-risk, high-reward gender. Male mammals are more likely to die without issue (either pushed out of the mating pool or actually killed) but they are also more likely to leave behind a really large number of young.
While the evolution of greater male size is easy to see and about as far beyond dispute as the theory of evolution itself, ev psych points to the greater risk-taking behavior on the part of men as our genetic heritage. Every human has tendencies both to play it safe and to take risks (we're flexible), but males take risks more willingly than females because, in evolutionary terms, they had so much more to gain by coming out ahead. Ev psych points to men taking risky jobs (not just combat, but open-sea fishing, etc.) and engaging in risky behavior (fighting, drug use, car racing, etc.) as manifestations of evolution. Greater male aggressiveness also goes hand-in-hand with greater size and competition. The human male penchant for risk-taking and aggression served us in the past, and now we're stuck with it, for good or ill. How we accommodate these tendencies is a question of culture, but the tendency is built in.