Update 2009-07-28: In the final project another solution was pointed out to me, which ended up being better in most cases, see the commands here: http://code.google.com/p/wiidiaplayer/source/browse/trunk/wiidiaserver/rtmp/flvstreamprovider/convertvideoffmpeg.sh

In my previous blog I explained the format I want the flash video in (and the reasons why), however, getting my video in that format did pose some problems, most of which I hope I have tackled by now.

The actual command for converting the video itself is not so complicated:

 /usr/bin/mencoder "$1"
        -of lavf -lavfopts format=flv
        -af resample=44100:0:1 -af channels=2 -oac mp3lame -lameopts cbr:br=128 -mc 0
        -ovc lavc -lavcopts vcodec=flv:vbitrate=2500:autoaspect:vratetol=1000:keyint=1 -ofps $FPS
        -o "$2"

On the first line, it shows the command and inputfile, the second line defines handles the output container format; The third line handles all audio (44.1kHz (the next two numbers define the conversion type), use two channels, convert with lame to a constant bitrate of 128kbps. The final option disallows any shifting between video and audio frames.

The forth line defines the video options (codec flv, bitrate (max) 2500kbps (max 1000kbps tolerance), per 1 frame 1 keyframe (ie: use only keyframes), and the required framerate), the final line sets the output filename.

Now, before we continue, it’s necessary to note that I have installed 3 different versions of mplayer in the last week, and each version needed a slightly different command. Currently I’m using MPlayer SVN-r24130, as provided by Gentoo ebuild media-video/mplayer-1.0.20070824, if you have a different version, you might need to adjust some parameters. I’m using some unversioned version of ffmpeg, which is provided by media-video/ffmpeg-0.4.9_p20070330; however my experiences with ffmpeg is that it is much more stable and compatible between versions, so any recent version should probably do.

The problem with the command above is that it only works on certain video files, and obviously we would like to convert any video file that mplayer can play. It seems that there are several bugs in mplayer/mencoder that disallow this (try the command above on your average avi file and mencoder will crash with a segfault), so we have to take a work-around. We will convert the video in two steps: first convert the source to an intermediary format, next convert this format with the command above to the final result. To avoid quality loss and extra CPU load, the intermediary format is the raw, uncompressed video format. This format takes about 4.5MB per second, so we don’t want to store it on disk, but pipe it directly from one mencoder into the next. To accomplish this we need to redirect any other output from mencoder to somewhere else (so that only the audio/video data reaches the next mencoder, not the status information). Furthermore, we need to keep the first mencoder from wanting to seek in the file it writes; When writing avi-files, mencoder updates the avi header every couple of hundred megabytes, by seeking to the beginning of the file and writing the header – this obviously cannot be done with a pipe. The -noodml tag fixes this problem, but another problem persists: the second mencoder (behind the pipe) expects the maximum avi size to be 4GB (which is reached in 15 minutes), so after 15 minutes encoding stops. The only solution I’ve found to this is not using the avi container, but the asf container (the mpeg container and other containers have other problems, only asf seems to work).

So the encoding part of the file looks like this:

/usr/bin/mencoder "$1"
                -of lavf -lavfopts format=asf
                -oac pcm -af resample=44100:0:1
                -ovc raw -vf scale=400:224 -ofps $FPS
                -o /dev/fd/3 3>&1 >/var/log/mencoder/1 2>&1 |
        /usr/bin/mencoder /dev/stdin
                -of lavf -lavfopts format=flv
                -af resample=44100:0:1 -af channels=2 -oac mp3lame -lameopts cbr:br=128 -mc 0
                -ovc lavc -lavcopts vcodec=flv:vbitrate=2500:autoaspect:vratetol=1000:keyint=1 -ofps $FPS
                -o "$2" > /var/log/mencoder/2 2>&1

The first mencoder sets the correct resoltion, framerate and audio options, and outputs it to a filedescriptor called /dev/fd/3. The little filedescriptor magic afterwards redirects this filedescriptor 3 to stdout, while directing stdout en stderr to a logfile. Both the input and the output are files. For the wiidiaplayer all sources are (for now) files, and I render to a temporary directory as well, to improve caching and enable seeking in the video.

To this we add some extra padding that lowers the priority of the encoders (to avoid the computer locking up when coding), and some code that makes sure all child processes end when the parent process is killed, and that’s the full conversion script:


function getmencoderchildids {
    ps --ppid $$ | awk '$4=="mencoder" {print $1}'

function stop_encoding {
 echo "now stopping"
 for mypid in $PIDS; do
 	kill $mypid > /dev/null 2&>1;
 sleep 2;
 for mypid in $PIDS; do
 	kill -9 $mypid > /dev/null 2&>1;

trap stop_encoding TERM;
trap stop_encoding EXIT;


nice -n 2 /usr/bin/mencoder "$1"
	        -of lavf -lavfopts format=asf
                -oac pcm -af resample=44100:0:1
                -ovc raw -vf scale=400:224 -ofps $FPS
                -o /dev/fd/3 3>&1 >/var/log/mencoder/1 2>&1 |
        nice -n 1 /usr/bin/mencoder /dev/stdin
        		-of lavf -lavfopts format=flv
        		-af resample=44100:0:1 -af channels=2 -oac mp3lame -lameopts cbr:br=128 -mc 0
        		-ovc lavc -lavcopts vcodec=flv:vbitrate=2500:autoaspect:vratetol=1000:keyint=1 -ofps $FPS
        		-o "$2" > /var/log/mencoder/2 2>&1 &
while [ -n "$(getmencoderchildids)" ]; do
 echo "$(getmencoderchildids)"
 sleep 1;

Nothing is as irritating as stuttering image or sound in the video you just put up on your big-screen TV. In the Wiidiaplayer, this stottering can come from 2 sources. Either the Wii doesn’t get the video and audio-data fast enough; this might be either the fault of the network, of the video-source, or the encoding program. Of these problems, only the encoding speed is of interest to me, the network is fast enough (11MBit), and the video source is a local video file. The problems in file-encoding wil be discussed in a separate blog entry.

The second reason for stuttering video or audio is the decoding and rendering speed on the Wii. Flash is extremely slow for showing video; on my Linux box, a flv video playing full screen in a browser might take 80% CPU (flash 9), while playing the same file in mplayer full screen results in less than 1 percent CPU load (and that is without optimized videocard drivers which will take even more load off the CPU). There might be a thousand good (or not) reasons for this, however this doesn’t change the fact that playing video in flash is slow. (There are two extra reasons to assume flash-video is slow. Firstly, the Wii plays mjpeg video’s without a problem, secondly, Adobe recently announced Moviestar, which (among other things) brings hardware acceleration to flash video (implying that it has little/none of the now).

Since the Wii has only limited processing power, it is essential to tweak our videos to make the best use of this power. It was clear from the start that 1080p videos on the Wii would not work. But as I stated before, although better image quality does look nicer, the movie usually does not improve. For me personally, what the Wii can bring is good enough.

There are several parameters that will infuence flv playback speed. I discovered the optimal values for most of these parameters through extensive testing on the Wii. Probably most tunings will work likewise on other platforms, however I never tested that. Note again, that I only tested for decoding-and-viewing performance on the Wii; The suggestions below are not valid when network bandwidth is an issue. I also do not claim to be an expert on audio or video-image quality, so while my settings are acceptable for me, others might be able to improve on them. Also, since your content is different from mine, your results might vary as well. If you get different results, please post them in the comments.

You will notice that on a lot of issues I substitute CPU on the Wii for CPU on the server (of which I claimed I had enough). These options probably need different tweaking when the server CPU is less powerfull.

Right now, I’m getting 400 x 224 video @ ~18 fps.

Testing method 

Since I do not have a reliable method to determine load on the Wii, I varied different parameters and played the file on the Wii with a too high framerate. As soon as the Wii reaches full capacity, frames will be dropped and the reported framerate will stabilize on a lower value than the actual video framerate. The flash program will regularly report the actual framerate (from the NetStream.currentFPS) back to the server; a higher framerate means that the load of each frame is lower.


Obviously the resolution of the video file matters. A little experimenting led to a best resolution of 400 x 224. This is not exedentally exactly half of the 800 x 450 resolution that will fill the screen in Opera (adjusted for FLV resolutions which always need to be a multitude of 2).


Logical reasoning would suggest that making each frame a keyframe, would be the most effective, and indeed, I seem to get a 1 or 2%  increase in framerate whe using only keyframes.

Video Framerate

Ideally, the framerate of the source video is equal to the framerate that is being displayed. When this is not the case, extra overhead is introduced by discarding frames. Also, the resulting viewing experience is less. If our video is 20 fps, and we can only show 19 fps, the result will be 18 images that are 50 milliseconds apart, followed by one that has a 100 millisecond gap. Having a continuous stream of images with 53 ms gaps looks far better.

What I found was that when video was encoded at 25 fps, the Wii would show only about 16.5 fps. Recoding the video to 18 fps allowed the Wii to play most of the video at 18 fps. However some parts of the video (especially those with heavy zooming and panning) still play at only 14-15fps. Encoding the whole video on 16 fps, would allow everything to play at 16 fps.

For now, I opt  for the higher setting of 18 fps.


My initial guess was that a higher bitrate would result in a lower CPU usage, since a higher bitrate would mean less compression (so less work to decompress). However, the opposite turned out to be true. Forcing the video bitrate to remain under the 2500kbps, avoids framedrops and stuttering on high-motion parts.


FLV supports mp3 and proprietary Nellymoser codec for sound. I didn’t look into the Nellymoser codec too much (don’t even know if any open source products can encode into it), some things can be tweaked on the mp3 side though. Leaving out the sound at all increases the framerate with about 2 fps compared to using 320kbps stereo mp3. Here as well it seems that a lower bitrate improved performance, so I settled for 128 kbps stereo.

Extra encoding options

There are numerous extra encoding options in mplayer, most of these seem to be concerned with exchanging bitrate for CPU on the encoding side. None of the extra options I tried increased the framerate measurably, nor could I see any improvement in image quality (and they do produce higher load on the server), so I use none of them


Perhaps more than obvious, but it tricked me for a while. Using extra stuff in your flash file on top of your video decreases performance, especially if these are nice semi-transparent layers. I had such a layer printing the current FPS for a while, not realizing that just that layer was keeping the fps down. A single transparent layer, occupying about 20% of the screen cost me about 25% or 4 fps performance. Having a lot of flash objects doesn’t seem to be a problem, as long as they are invisible (and, honestly, as soon as I pop up my menu, I don’t care that much that my framerates plumet).

SWF framerate

It is possible to set a framerate for SWF files. This decides how often a frame is drawn to provide nice swf animations, however seems to be totally independent of video framerate. An swf framerate of 2 fps, resulted in a really choppy interface, but the video still played at the same  framerate. I could not find any video-performance differences between 2 and 25 swf-fps (obviously, when doing a lot of swf-animations on an overlay, or having a (lot of) heavy  eventhandlers on “onframeenter”, keeping down the swf fps will improve video performance).

SWF scaling

I have experimented breefly with SWF scaling; making an swf of dimension p*q, and then telling the HTML page to resize the swf to x*y, but as far as I could see, it only decreased performance slightly.

SWF window mode

In the HTML page, it’s possible to set a SWF window mode, though the wmode attribute. The value of the wmode attribute seems to have no effect on the performance

SWF video smoothing 

There is a attribute for the video tag in flash called “Smoothing. Apparently, it exchanges CPU for quality in flash video playback. Since it only works for flash 8+, I expected the setting to have no effect, and I haven’t been able to measure any either.

Wii/Opera scaling 

From the WiiCR project, I got the suggestion that it could increase performance to make the swf-movie smaller (in dimensions/resolution) and use the Opera zooming/scaling functions to return it to full-screen size. Indeed, having the swf at only 400 x 224, and then zooming to full-screen, the performance seemed to improve about 1%. The problem however is that now I need to do manual zooming and positioning every time I want to watch a video, so I’m not sure whether this is worth it.


When doing frequent reloads of the wiidiplayer page on the Wii, sometimes the console locks up, any I need to press the power-button at the front of the console for 5 seconds to reset; during normal operation of the wiidiaplayer, however, I have not been able to crash it.

I’ll describe my mencoder settings in the next blog.