Huh. For some reason, MacOS X has the surround sound channels set differently than what VLC expects.
Data is going out over MiniDisplay port to HDMI.
I had to open Audio Midi Setup and set the following channels:
Left Front: 1
Center: 5
Right Front: 2
Left Surround: 3
Right Surround: 4
Sub-Woofer: 6?
Left Rear Surround: 7?
Right Rear Surround: 8?
Downloading Dolby's Channel Check Trailer. Hopefully it'll help me set the last three.
http://www.demo-world.eu/trailers/high-definition-trailers.php