- Native multimedia: why, what, and how?
- Codecsthe horror, the horror
- Rolling custom controls
- Multimedia accessibility
- Summary
This chapter is from the book
This chapter is from the book
This chapter is from the book
Codecs—the horror, the horror
Early drafts of the HTML5 specification mandated that all browsers should at least have built-in support for multimedia in two codecs: Ogg Vorbis for audio and Ogg Theora for movies. Vorbis is a codec used by services like Spotify, among others, and for audio samples in games like Microsoft Halo, it's often used with Theora for video and combined together in the Ogg container format.
However, these codecs were dropped from the HTML5 spec after Apple and Nokia objected, so the spec makes no recommendations about codecs at all. This leaves us with a fragmented situation. Opera and Firefox support Theora and Vorbis. Safari doesn't, preferring instead to provide native support for the H.264 video codec and MP3 audio. Microsoft has announced that IE9 will also support H.264, which is also supported on iPhone and Android. Google Chrome supports Theora and H.264 video, and Vorbis and MP3 audio. Confused?
As we were finishing this book, Google announced it is opensourcing a video codec called VP8. This is a very high-quality codec, and when combined with Vorbis in a container format based on the Matroska format, it's collectively known as "webM".
Opera, Firefox and Chrome have announced it will support it. IE9 will, if the codec is separately installed. VP8 will be included in Adobe's Flash Player and every YouTube video will be in webM format.
Like Theora, it's a royalty-free codec. In this chapter, you can substitute .ogv examples with .webm for high quality video, once browser support is there.
The rule is: provide both royalty-free (webM or Theora) and H.264 video in your pages, and both Vorbis and MP3 audio so that nobody gets locked out of your content. Let's not repeat the mistakes of the old "Best viewed in Netscape Navigator" badges on websites.
Multiple <source> elements
To do this, you need to encode your multimedia twice: once as Theora and once as H.264 in the case of video, and in both Vorbis and MP3 for audio.
Then, you tie these separate versions of the file to the media element. Instead of using the single src attribute, you nest separate <source> elements for each encoding with appropriate type attributes inside the <audio> or <video> element and let the browser download the format that it can display.
Note that in this case we do not provide a src attribute in the media element itself:
1<video controls>2<source src=leverage-a-synergy.ogv type='video/ogg; codecs="theora, vorbis"'>3<source src=leverage-a-synergy.mp4 type='video/mp4; codecs="avc1.42E01E, mp4a.40.2"'>4<p>Your browser doesn't support video.5Please download the video in <a href=leverage-a-synergy.ogv>Ogg</a> or <a href=leverage-a-synergy.mp4>mp4</a> format.</p>6</video>
Line 1 tells the browser that a video is to be inserted and to give it default controls. Line 2 offers an Ogg Theora video and uses the type attribute to tell the browser what kind of container format is used (by giving the file's MIME type) and what codec was used for the encoding of the video and the audio stream. We could also offer a WebM video here as a high-quality royaltyfree option. Notice that we used quotation marks around these parameters. If you miss out on the type attribute, the browser downloads a small bit of each file before it figures out that it is unsupported, which wastes bandwidth and could delay the media playing.
Line 3 offers an H.264 video. The codec strings for H.264 and AAC are more complicated than those for Ogg because there are several profiles for H.264 and AAC. Higher profiles require more CPU to decode, but they are better compressed and take less bandwidth.
Inside the <video> element is our fallback message, including links to both formats for browsers that can natively deal with neither video type but which is probably on top of an operating system that can deal with one of the formats, so the user can download the file and watch it in a media player outside the browser.
OK, so that's native HTML5 video for all users of modern browsers. What about users of legacy browsers—including Internet Explorer 8 and older?
Video for legacy browsers
Older browsers can't play native video or audio, bless them. But if you're prepared to rely on plugins, you can ensure that users of older browsers can still experience your content in a way that is no worse than they currently get.
Remember that the contents of the <video> element can contain markup, like the text and links in the previous example? Because the MP4 file type can also be played by the Flash player plugin, you can use the MP4 movie in combination as a fallback for Internet Explorer 8 and older versions of other browsers.
The code for this is as hideous as you'd expect for a transitional hack, but it works everywhere a Flash Player is installed—which is almost everywhere. You can see this nifty technique in an article called "Video for Everybody!" by its inventor, Kroc Camen http://camendesign.com/code/video_for_everybody.
Alternatively, you could host the fallback content on a video hosting site and embed a link to that between the tags of a video element:
<video controls><source src=leverage-a-synergy.ogv type='video/ogg; codecs="theora, vorbis"'><source src=leverage-a-synergy.mp4 type='video/mp4; codecs="avc1.42E01E, mp4a.40.2"'><embed src="https://www.youtube.com/v/cmtcc94Tv3A&hl=en_GB&fs=1&rel=0" type="application/x-shockwave-flash"allowscriptaccess="always" allowfullscreen="true" width="425" height="344"></video>
You can use the html5media library http://static.etianen.com/html5media/ to hijack the <video> element and automagically add necessary fallback by adding one line of JavaScript in the head of your page.
Sending differently-compressed videos to handheld devices
Video files tend to be large, and sending very high-quality video can be wasteful if sent to handheld devices where the small screen sizes make high quality unnecessary. There's no point in sending high-definition video meant for a widescreen monitor to a handheld device screen. Compressing a video down to a size appropriate for a small screen can save a lot of bandwidth, making your server and—most importantly—your mobile users happy.
HTML5 allows you to use the media attribute on the source element, which queries the browser to find out screen size (or number of colours, aspect ratio, and so on) and send different files that are optimised for different screen sizes.
This functionality and syntax is borrowed from the CSS Media Queries specification dev.w3.org/csswg/css3-mediaqueries/) but is part of the markup, as we're switching source files depending on device charateristics. In the following example, the browser is "asked" if it has a min-device-width of 800px—that is, does it have a wide screen. If it does, it receives hi-res.ogv; if not, it is sent lo-res.ogv:
<video controls><source src=hi-res.ogv ...media="(min-device-width: 800px)><source src=lo-res.ogv></video>
Also note that you should still use the type attribute with codecs parameters and fallback content previously discussed. We've just omitted those for clarity.