OTT Guide to Codecs, Container Formats and Transcoding

If the title of this article doesn’t already make your head hurt, then the rest of it definitely will. Well, hopefully I can explain it in a way that makes sense to the non-technical person. I am writing this article about video/audio codecs becauseI found myself confused when trying to explain it to a friend one day.

What exactly is a codec? I know you need them to play a movie here and there, but what the heck is it really? What really confused me were all the terms I’ve heard of, but never really understood: H.264, DivX, MP4, AVI, MPEG-2, AVCHD, AAC, OGG, MKV, etc, etc. Were these all codecs? After much reading and learning, I realized that the world of audio and video codecs and container formats is very confusing. Actually, I really didn’t even know the difference between a codec and a container format until I started researching.

I won’t be going into much detail, just because the topic starts to become very terse very fast. Do you really care if 1 hour of RAW uncompressed 1080p 60 frames per second video eats up almost 500 GB of storage space? Probably not.

What is a Codec?

So what exactly is a codec and what does it do? In the most simple terms possible, a codec is an encoder/decoder or, in it’s original form, a compressor (co) – decompressor (dec). So that is exactly what a codec does: it takes a digital file and compresses it (or encodes it) for storage and decompresses it (or decodes it) for viewing or transcoding. I’ll tell you what transcoding is a little bit later on, let’s not confuse ourselves this early on.

The important thing to understand is that a codec is kind of like a specification of how the bitstream should look. What is a bitstream? To not get very technical, it’s how the sequence of bytes should be arranged for that particular codec. Each codec has a specific set of guidelines. For example, it can specify the width, height, aspect ratio, etc. Check out this page, but don’t read too much or your head will hurt:

http://www.cs.cf.ac.uk/Dave/Multimedia/node262.html

Because it’s a specification, there can me many ways to convert data into that format. Hence, there can be many encoders for one codec. I’ll explain more about encoders in the Popular Codecs section below.

So why do we even need all this compression and decompression? Well like I mentioned above, a RAW uncompressed 1080p file is almost a whopping 500GB for just one hour. You would need your own data center to store all your family videos if everything was recorded in that format. That’s where compressions comes into play. There are different types of compression, which translates into different types of codecs. So what are some popular codecs?

Popular Codecs

H.264 (MPEG-4 Part 10 AVC) – This is also officially known as MPEG-4 Part 10, but H.264 is what you commonly hear. Yes, there are also multiple names for the same thing. H.264 is by far the most popular format for encoding videos. H.264 offers the best of all worlds: smaller file sizes with higher quality.

Again, remember, H.264 is just the codec, you won’t find any files with .h264 extensions. That’s what container formats are for, which I’ll explain in the section below. This codec is used all over the place from smartphones to digital camcorders to Blu-ray discs. It’s also used for delivering a lot of web video today because of it’s ability to give you great image quality with a high compression ratio and a low bit rate, meaning less strain on streaming servers.

It’s also highly supported and probably will be the most popular codec for a long time to come. It’s supported by Apple, YouTube, HTML 5, and even in Adobe Flash.

The only downside to H.264 is that it’s algorithms for compression are so good, it’s a lot slower to encode a video into this format. You’ll find out more about that when we talk about transcoding down below.

As I had mentioned above, there are multiple encoders for a single codec. For the H.264 codec, one of the most popular encoders is x264 from VideoLAN, the same people who create VLC Media Player. x264 is free and open-source, but there are many commercial H.264 encoders also. It doesn’t matter how the video stream gets converted into this format, any H.264 decoder (like VLC) will be able to view the file.

MPEG-2 – Still have a large DVD collection? That’s all done using the MPEG-2 codec. It’s pretty old, but very common. The other place where you find MPEG-2 compression is when you watch those HD channels broadcast over-the-air. That’s all MPEG-2 compression. Interesting stuff to know, right?

Since the algorithm for compression using MPEG-2 is vastly inferior to H.264, it’s got the advantage of being a lot faster to encode. However, it never worked for web streaming because at a lower bitrate, the quality would quickly nosedive and you’d end up with pixelated video. That’s why they came up with MPEG-4 Part 2.

MPEG-4 Part 2 – You probably remember the days when everything talked about DivX and Xvid right? Those were encoders for the MPEG-4 Part 2 format, aka H.263. A lot of movies that you downloaded back in the day were compressed using this codec because it gave you a good file size and maintained reasonable quality. However, the only major problem is that high-definition content still suffered in terms of image quality and that’s where H.264 that I mentioned above took over.

Windows Media Video – This is basically Microsoft’s version of the MPEG-4 codecs. WMV 7 was introduced in 1999 and was a copy of MPEG-4 Part 2. Later on, WMV 8 and 9 came, followed by VC-1, a codec that mostly copies MPEG-4 Part 10 and is currently used in Blu-ray discs too. The only other places you see WMV and VC-1 are mostly in Microsoft related products like Windows Movie Maker, Silverlight, HD DVD and Microsoft Expression Encoder, etc.

These are just the popular video codecs, you also have your audio codecs, which come into play when we talk about containers below. Popular audio codecs include FLAC, AC3, Dolby Digital Plus, DTS-HD, ALAC, etc. As with video, audio codecs have their own container formats like AIFF, WAV, etc.

These are not by any means all the codecs out there, it’s just a list of some of the most popular ones. Now let’s talk about container formats.

Container Formats

When you play a file in Windows Media Player or VLC or Quicktime or whatever your media player of choice, you’re usually opening a container format. A container format or wrapper is basically package of one or more codecs, video or audio or both. A container is used to add audio along with video and to make sure the audio and video are synchronized perfectly. The container will also contain other information needed by a streaming server or by the media player. The video and audio bitstreams are just wrapped in another bitstream.

Container formats are where you see the common file extensions we are all used too like MP4, MOV, WMV, AVI, etc. Let’s go through some of the most common containers:

MP4 – You’ve probably downloaded a song that had an .MP4 extension, which is a wrapper for an many video and audio codecs. Most commonly, it’s used to wrap H.264 video and AAC encoded audio. It also supports MPEG-4 Part 2 and MPEG-2 video codecs also. Also, audio can be encoded using other codecs other than AAC.

AVI – This is a Microsoft container format from 1992. It was very popular back in my day and it’s still around quite a bit. If you do any encoding, you should never use the AVI container format anymore. Firstly, it doesn’t support newer codecs like H.264. It also has a few other major problems which is why it’s not used anymore.

ASF – The Advanced Systems Format by Microsoft don’t normally use the .ASF file extension. Instead you’ll see most ASF containers use .WMA or .WMV. You’ll see these files around if you’re using all Microsoft products. Once you leave the Microsoft world, you’ll run into issues playing back the files, especially if you use the H.264 codec.

AVCHD – AVCHD is the most common container format for HD camcorders. The video is normally going to be H.264 with AC3 (Dolby Digital) audio or Linear PCM.

MKV – Download a movie lately? It’s probably in the MKV container format, which houses H.264 video files. Quite a few tools support MKV like Boxee, PS3 Media Server, XMBC, VLC, etc, but it’s not as widely supported as other container formats just yet.

FLV – This is Adobe Flash, which supports many different codecs, most common being H.264 and AAC. Flash video is one of the most popular ways videos are streamed over the Internet. It’s losing ground because of HTML 5 and the fact that Apple won’t let flash run on any of it’s iDevice’s, but it’s still very common.

There are other container formats like the QuickTime File Format, OGG, WebM, etc, but you kind of get the picture now.

Transcoding

The last thing to understand about all this is that you probably will have to convert that AVCHD video you downloaded from your camcorder to a different format that you can then import into iTunes and play on your iPhone. Since there are so many codecs and container formats and each container format can have different combinations of video and audio codecs, you need a transcoder to get you the type of file you want for your particular device.

There are many transcoders aka video converters out there: HandBrake, FFmpeg, SUPER, VirtualDub, etc. Some are paid and some are free open-source tools. Some tools, like HandBrake, give you presets so you can simply choose your source file and pick an output format like iPod or iPhone and click Start. It’ll choose all the settings, container, codec, etc for you. However, hopefully after reading this post, you now understand a little bit more and can go about converting videos between formats and understand the underlying process a little better. Enjoy!