I saw
this comic recently.
Somewhere in the house is a box containing the plans I drew up around age 12 for pretty much exactly this. (Without the Oculus Rift part, but instead with a complicated multiple controller setup for large-scale team play.) As I recall, my drawings were heavy on the concepts and light on engineering details. :)
My childhood vision began as "Battlestar Galactica fleet vs. Cylon Basestar fleet in the sky", but I gradually expanded it to include at least two additional very distinct teams. I don't think I quite recognized at the time just how far out of reach my ideas were for the technology of the day, but it's awesome to think that someone could conceivably do it for real right now.