Nothing is as irritating as stuttering image or sound in the video you just put up on your big-screen TV. In the Wiidiaplayer, this stottering can come from 2 sources. Either the Wii doesn’t get the video and audio-data fast enough; this might be either the fault of the network, of the video-source, or the encoding program. Of these problems, only the encoding speed is of interest to me, the network is fast enough (11MBit), and the video source is a local video file. The problems in file-encoding wil be discussed in a separate blog entry.

The second reason for stuttering video or audio is the decoding and rendering speed on the Wii. Flash is extremely slow for showing video; on my Linux box, a flv video playing full screen in a browser might take 80% CPU (flash 9), while playing the same file in mplayer full screen results in less than 1 percent CPU load (and that is without optimized videocard drivers which will take even more load off the CPU). There might be a thousand good (or not) reasons for this, however this doesn’t change the fact that playing video in flash is slow. (There are two extra reasons to assume flash-video is slow. Firstly, the Wii plays mjpeg video’s without a problem, secondly, Adobe recently announced Moviestar, which (among other things) brings hardware acceleration to flash video (implying that it has little/none of the now).

Since the Wii has only limited processing power, it is essential to tweak our videos to make the best use of this power. It was clear from the start that 1080p videos on the Wii would not work. But as I stated before, although better image quality does look nicer, the movie usually does not improve. For me personally, what the Wii can bring is good enough.

There are several parameters that will infuence flv playback speed. I discovered the optimal values for most of these parameters through extensive testing on the Wii. Probably most tunings will work likewise on other platforms, however I never tested that. Note again, that I only tested for decoding-and-viewing performance on the Wii; The suggestions below are not valid when network bandwidth is an issue. I also do not claim to be an expert on audio or video-image quality, so while my settings are acceptable for me, others might be able to improve on them. Also, since your content is different from mine, your results might vary as well. If you get different results, please post them in the comments.

You will notice that on a lot of issues I substitute CPU on the Wii for CPU on the server (of which I claimed I had enough). These options probably need different tweaking when the server CPU is less powerfull.

Right now, I’m getting 400 x 224 video @ ~18 fps.

Testing method 

Since I do not have a reliable method to determine load on the Wii, I varied different parameters and played the file on the Wii with a too high framerate. As soon as the Wii reaches full capacity, frames will be dropped and the reported framerate will stabilize on a lower value than the actual video framerate. The flash program will regularly report the actual framerate (from the NetStream.currentFPS) back to the server; a higher framerate means that the load of each frame is lower.

Resolution

Obviously the resolution of the video file matters. A little experimenting led to a best resolution of 400 x 224. This is not exedentally exactly half of the 800 x 450 resolution that will fill the screen in Opera (adjusted for FLV resolutions which always need to be a multitude of 2).

Keyframes 

Logical reasoning would suggest that making each frame a keyframe, would be the most effective, and indeed, I seem to get a 1 or 2%  increase in framerate whe using only keyframes.

Video Framerate

Ideally, the framerate of the source video is equal to the framerate that is being displayed. When this is not the case, extra overhead is introduced by discarding frames. Also, the resulting viewing experience is less. If our video is 20 fps, and we can only show 19 fps, the result will be 18 images that are 50 milliseconds apart, followed by one that has a 100 millisecond gap. Having a continuous stream of images with 53 ms gaps looks far better.

What I found was that when video was encoded at 25 fps, the Wii would show only about 16.5 fps. Recoding the video to 18 fps allowed the Wii to play most of the video at 18 fps. However some parts of the video (especially those with heavy zooming and panning) still play at only 14-15fps. Encoding the whole video on 16 fps, would allow everything to play at 16 fps.

For now, I opt  for the higher setting of 18 fps.

Bitrate

My initial guess was that a higher bitrate would result in a lower CPU usage, since a higher bitrate would mean less compression (so less work to decompress). However, the opposite turned out to be true. Forcing the video bitrate to remain under the 2500kbps, avoids framedrops and stuttering on high-motion parts.

Sound

FLV supports mp3 and proprietary Nellymoser codec for sound. I didn’t look into the Nellymoser codec too much (don’t even know if any open source products can encode into it), some things can be tweaked on the mp3 side though. Leaving out the sound at all increases the framerate with about 2 fps compared to using 320kbps stereo mp3. Here as well it seems that a lower bitrate improved performance, so I settled for 128 kbps stereo.

Extra encoding options

There are numerous extra encoding options in mplayer, most of these seem to be concerned with exchanging bitrate for CPU on the encoding side. None of the extra options I tried increased the framerate measurably, nor could I see any improvement in image quality (and they do produce higher load on the server), so I use none of them

Overlays 

Perhaps more than obvious, but it tricked me for a while. Using extra stuff in your flash file on top of your video decreases performance, especially if these are nice semi-transparent layers. I had such a layer printing the current FPS for a while, not realizing that just that layer was keeping the fps down. A single transparent layer, occupying about 20% of the screen cost me about 25% or 4 fps performance. Having a lot of flash objects doesn’t seem to be a problem, as long as they are invisible (and, honestly, as soon as I pop up my menu, I don’t care that much that my framerates plumet).

SWF framerate

It is possible to set a framerate for SWF files. This decides how often a frame is drawn to provide nice swf animations, however seems to be totally independent of video framerate. An swf framerate of 2 fps, resulted in a really choppy interface, but the video still played at the same  framerate. I could not find any video-performance differences between 2 and 25 swf-fps (obviously, when doing a lot of swf-animations on an overlay, or having a (lot of) heavy  eventhandlers on “onframeenter”, keeping down the swf fps will improve video performance).

SWF scaling

I have experimented breefly with SWF scaling; making an swf of dimension p*q, and then telling the HTML page to resize the swf to x*y, but as far as I could see, it only decreased performance slightly.

SWF window mode

In the HTML page, it’s possible to set a SWF window mode, though the wmode attribute. The value of the wmode attribute seems to have no effect on the performance

SWF video smoothing 

There is a attribute for the video tag in flash called “Smoothing. Apparently, it exchanges CPU for quality in flash video playback. Since it only works for flash 8+, I expected the setting to have no effect, and I haven’t been able to measure any either.

Wii/Opera scaling 

From the WiiCR project, I got the suggestion that it could increase performance to make the swf-movie smaller (in dimensions/resolution) and use the Opera zooming/scaling functions to return it to full-screen size. Indeed, having the swf at only 400 x 224, and then zooming to full-screen, the performance seemed to improve about 1%. The problem however is that now I need to do manual zooming and positioning every time I want to watch a video, so I’m not sure whether this is worth it.

Lockups 

When doing frequent reloads of the wiidiplayer page on the Wii, sometimes the console locks up, any I need to press the power-button at the front of the console for 5 seconds to reset; during normal operation of the wiidiaplayer, however, I have not been able to crash it.

I’ll describe my mencoder settings in the next blog.