Vidding with high-res source
This is mainly a post-mortem describing how I made Bukowski, my House vid from this year's VVC, using extremely high-resolution source. I'll apologize in advance -- I really can't provide individual help for anyone trying to do this or troubleshoot it, because I just don't have time. I hope this will be useful as a starting point and a leg-up for people trying to figure it out for themselves.
Any corrections and suggestions are very welcome! I was working under time pressure, so each of the steps described below is pretty much just the first way I managed to get things to work, not at all necessarily the best or even a very good way.
I. What high-res source is and why to bother.
What I mean by high resolution source is the AVI files increasingly becoming available online that have been encoded from HDTV captures at what is actually a higher resolution than DVD. They're usually labeled as "HR" somewhere in the filename and are about 650-700 MB large apiece. DVD resolution is 720x480, where the resolution of these files is (for one example) 960x528 -- so you can see, it's a lot bigger.
I made Bukowski using these files. All of House S2 is available in HR format, but I had to use the DVDs for a few clips where I needed S1 footage. To give you a sense of the relative visual quality, I'm posting two screencaps. (Linking rather than embedding, so they won't throw off the width of the entire page.)
These are both taken from the same file, the 26 MB XviD version of the vid, at the size that it plays back in using the VLC player; both screenshots taking using VLC. Both are obviously very pretty source; you don't really get a sense of how much better the HR is until you see the video moving, but I tried to use two shots that are fairly close-up of House, so you can get a sense of the difference in the crispness and detail.
II. Requirements
The method I am going to describe takes a lot of disk space, a fast CPU, and a lot of RAM. Also, I use Adobe Premiere Pro 2.0, and I can only tell you the settings for that. If you use another program, you can probably use the first half of my instructions and then try and tweak the second half for your own program.
Disk space: I used about 70 GB for my House vid in total, vidding primarily with one season. There are ways to use less space by instead just converting individual clips, but I'm too lazy to do that, and I'm only going to talk about the way I did it because this process is already insanely long as it is. Most systems that are powerful enough to work with the AVS files at all will probably have the disk space available to do this.
CPU & memory: My current system has an Athlon 64 3500+ (2.21 GHz) and 1 GB of high quality RAM. My computer struggled in places but was mostly okay. I wouldn't want to try it with lower specs.
III. Background info and additional reading
When you download AVIs online, they are normally in some nice compressed format like DivX, which is great for saving disk space and watching, but does not work properly as source for a video editing program. You need to convert the DivX files into frame-accurate source.
When I am working with the more common 350 MB AVIs, I normally use the method I partly describe in this old post here. I basically take an entire DivX episode, dump it into VirtualDubMod, and convert it into Panasonic DV format. A one-hour episode of TV usually runs about 7 GB in size, so, uh, if you are a freak like me you can convert all of, say, SGA and put it on a 300 GB drive. *cough*
Just for fun, I tried to convert one single HR episode (using HuffyUV instead of Panasonic DV): 32 GB. So, yeah. That wasn't going to work. Hence the much more convoluted process below.
ErMaC and absolut3destiny's AMV technical guide goes into HEAPS of useful detail and step-by-step tutorials on how to set up the source for your vid, focusing on footage ripped from DVDs, and it's what I used myself to figure out how to do this. Using downloaded HR AVI files for source is actually a tweaked version of their process of using ripped DVD footage. If you run into trouble trying to follow my minimalist version here, your best bet is to go through the guide, because they go into a lot more detail than I am doing here -- I'm pretty much just going to describe how I did it in this one specific example, because that's the only way that I can provide a step-by-step process.
Also, you need and want their truly awesome AMV App, which will install nearly all of the necessary programs and codecs in one fell swoop.
IV. The Step-By-Step Process
And that's all! See, easy! ahahahaaaaaa.
Any corrections and suggestions are very welcome! I was working under time pressure, so each of the steps described below is pretty much just the first way I managed to get things to work, not at all necessarily the best or even a very good way.
I. What high-res source is and why to bother.
What I mean by high resolution source is the AVI files increasingly becoming available online that have been encoded from HDTV captures at what is actually a higher resolution than DVD. They're usually labeled as "HR" somewhere in the filename and are about 650-700 MB large apiece. DVD resolution is 720x480, where the resolution of these files is (for one example) 960x528 -- so you can see, it's a lot bigger.
I made Bukowski using these files. All of House S2 is available in HR format, but I had to use the DVDs for a few clips where I needed S1 footage. To give you a sense of the relative visual quality, I'm posting two screencaps. (Linking rather than embedding, so they won't throw off the width of the entire page.)
These are both taken from the same file, the 26 MB XviD version of the vid, at the size that it plays back in using the VLC player; both screenshots taking using VLC. Both are obviously very pretty source; you don't really get a sense of how much better the HR is until you see the video moving, but I tried to use two shots that are fairly close-up of House, so you can get a sense of the difference in the crispness and detail.
II. Requirements
The method I am going to describe takes a lot of disk space, a fast CPU, and a lot of RAM. Also, I use Adobe Premiere Pro 2.0, and I can only tell you the settings for that. If you use another program, you can probably use the first half of my instructions and then try and tweak the second half for your own program.
Disk space: I used about 70 GB for my House vid in total, vidding primarily with one season. There are ways to use less space by instead just converting individual clips, but I'm too lazy to do that, and I'm only going to talk about the way I did it because this process is already insanely long as it is. Most systems that are powerful enough to work with the AVS files at all will probably have the disk space available to do this.
CPU & memory: My current system has an Athlon 64 3500+ (2.21 GHz) and 1 GB of high quality RAM. My computer struggled in places but was mostly okay. I wouldn't want to try it with lower specs.
III. Background info and additional reading
When you download AVIs online, they are normally in some nice compressed format like DivX, which is great for saving disk space and watching, but does not work properly as source for a video editing program. You need to convert the DivX files into frame-accurate source.
When I am working with the more common 350 MB AVIs, I normally use the method I partly describe in this old post here. I basically take an entire DivX episode, dump it into VirtualDubMod, and convert it into Panasonic DV format. A one-hour episode of TV usually runs about 7 GB in size, so, uh, if you are a freak like me you can convert all of, say, SGA and put it on a 300 GB drive. *cough*
Just for fun, I tried to convert one single HR episode (using HuffyUV instead of Panasonic DV): 32 GB. So, yeah. That wasn't going to work. Hence the much more convoluted process below.
ErMaC and absolut3destiny's AMV technical guide goes into HEAPS of useful detail and step-by-step tutorials on how to set up the source for your vid, focusing on footage ripped from DVDs, and it's what I used myself to figure out how to do this. Using downloaded HR AVI files for source is actually a tweaked version of their process of using ripped DVD footage. If you run into trouble trying to follow my minimalist version here, your best bet is to go through the guide, because they go into a lot more detail than I am doing here -- I'm pretty much just going to describe how I did it in this one specific example, because that's the only way that I can provide a step-by-step process.
Also, you need and want their truly awesome AMV App, which will install nearly all of the necessary programs and codecs in one fell swoop.
IV. The Step-By-Step Process
- Install the AMV App. This includes VirtualDubMod and Avisynth, among other things.
- Download and install the PICVideo MJPEG Motion Codec. The free trial version sticks a watermark on your video. If you want to get rid of it, it's under $30 for the non-commercial user version.
- Open one of your AVI files in VirtualDubMod and go to File/File Information....
My AVI files are all 960x528, at 23.976 fps. I suspect that for most AVIs made from HDTV broadcasts of live action TV, this is going to be roughly the same. My SGA HR files all match. If yours don't, you might have to tweak some of the settings used below; read the AMV tech guide to figure out how, if the results aren't coming out well.
Close VDub.
- On a drive that has plenty of room (you will need about 60 GB, as noted above) make a directory for your source named something short (eg, I created a folder on my G: drive named just house).
- In this directory, create three folders, called avi, avs, and mjpeg.
- Put all your nice downloaded AVI files into the "avi" directory. I recommend renaming them to something short that is unique and identifiable from the first few letters -- eg, I name all of mine things like "201-Acceptance.avi" and eliminate all the stuff that is typically in the name when you first download, which is usually something like "House-s02e01.Acceptance.HDTV.HR.AC3.5.1.blahblahblah.pirates.avi" or whatever.
- You are now going to create Avisynth (.avs) files for each of the downloaded avi files. Avisynth files are just plain text files that are used kind of as a filter between your downloaded compressed avi files and video editing software that needs frame-accurate source.
- For the first AVI file (eg, 101-epname.avi), create a text file (with notepad or something similar) in the "avs" folder with the same name and a suffix of .avs (eg, for 201-Acceptance.avi, create the file 201-Acceptance.avs).
- In this text file, put the following:
AVISource("G:\house\avi\201-Acceptance.avi", audio=false)
killaudio()
LanczosResize(848,480)
AssumeFPS(24)
ConvertToRGB32()
You will have to change the name of the source file in the first line (G:\house\avi\201-Acceptance.avi) to be whatever the name of the avi file you are using is, instead. For instance, if you followed these instructions but were working on your D: drive with SGA source, your line might instead have something like D:\sga\avi\201-Siege Part 3.avi.
- Now, test the file by opening VirtualDubMod and dragging and dropping in the **avs** file, NOT the avi. (Or you can open it from File/Open video file -- but NOT File/Open video file via Avisynth.) You should be able to scroll through the video of the avi file. If you don't, you probably messed up the name of the source file in the previous step.
If the aspect ratio looks off or the file is not working right -- you are going to have to go to the AMV technical guide and read up on Avisynth to figure out how it works in more detail so you can change the settings. Sorry!
- When you have it working and the video looks right, make a similar AVS file for every single AVI file that you are planning to use. All these files should be in your avs folder. Just copy the original AVS file; the only thing you have to change inside the copies is that one source file line.
Note: after this step, you should ignore the avi directory -- you can't delete it, but those AVI files are basically always going to be filtered through the AVS files, and you will not be using them directly at all. I make a point of this because you will be creating a second set of AVI files later on, MJPEG-encoded, and I want to avoid confusion. (Some confusion, anyway.)
- Now, start up VDub again and open up one of the avs files again. (It doesn't really matter which one.) Go to Video/Compression... and choose the PICVideo MJPEG Codec. Hit the "Configure" button and move the << Compression Quality>> slider to 8. Hit OK in the configuration and compression windows both.
- Still in VDub (NOTE: do NOT close VDub between these steps), go to File/Job Control... and the Job Control window will open up. Go to Edit/Process Directory.... First it will ask you to "select source directory". Choose the avs (NOT avi) directory. Second it will ask you to "select target directory". Choose the mjpeg directory you made in step 5.
- You should now have a long list in the Job Control window showing all the AVS files ready to be converted into AVI files. These AVI files will be compressed with the MJPEG codec that you set up in step 12. I recommend setting this up and then waiting to run it until overnight or while you are going to be away from the computer for a long time, because it will make everything else run slowly. Whenever you are ready, just hit the "Start" button in the Job Control window.
- When this is done, look in the mjpeg directory. It should be full of the converted MJPEG avi files. These run about 1.2 GB for a single one-hour episode.
- Now it is time to create the actual vid project. Remember that I am working in Premiere Pro 2.0 -- if you are using something else, you will have to figure out how to change this to make it work for your program.
- Open up Premiere 2.0 and choose "New Project". In the New Project window, go to the Custom Settings tab. You will have a window with "General", "Capture", "Video Rendering", and "Default Sequence" on the left-hand side.
- Select "General". Fill in the following values in the boxes/pull-down menus to the right:
Editing Mode: Desktop
Timebase: 24.00 frames/second
Frame Size: 848 horizontal, 480 vertical
Pixel Aspect Ratio: Square Pixels (1.0)
Fields: No Fields (Progressive Scan)
- Select "Video Rendering". Fill in the following values in the boxes/pull-down menus to the right:
File Format: Video for Windows
Compressor: PICVideo M-JPEG
Color Depth: Millions of colors
- Name the project file whatever you like and finish creating the project.
- Once the project has opened, import all of the MJPEG video files, and you are ready to go ahead with editing your vid however you want.
- (This is the step where you actually, you know, make the vid. Go ahead, we'll wait. *jeopardy theme plays*)
- Once your vid is made, and completely finalized, you are now ready to export. Set aside 4-6 hours for this process. No, I am not kidding. (You can break it up over a longer period of time, it won't hurt anything.)
This is because you have been editing with these fast, low-quality MJPEG files, which are basically placeholders, and now we are about to swap them out for the high-quality AVS files, which are incredibly slow and CPU-intensive to work with. As soon as you make the swap, your video editing tool will slow to a crawl and become very prone to crashing. It will be much less stressful if you just accept this and go in prepared.
Do not plan on doing anything else on your computer while you are actually doing the exporting. Close all non-essential programs and disconnect from the Internet if possible. Work slowly and be patient -- don't go clicking and scrolling all across your vid, or you will hang and eventually crash. If you click on the timeline, just sit there and wait until the frame loads up in the Program window before you try to do anything else. If you do hang, try to just sit and not touch the computer for 5-10 minutes to see if it will unstick itself before you try restarting Premiere or rebooting your computer.
- Save a copy of your project. In fact, save two copies, in different locations.
- In Premiere, go to Edit/Preferences.../Auto Save and uncheck Automatically Save Projects. Hit OK.
- Go to the Project window (with all your avi files) and select all of the MJPEG avi files. (Make sure you don't select anything else like your sequence or audio.) Right-click on them and choose Make Offline.... Check the Media Files Remain on Disk option (it should be checked by default anyway) and click OK.
- Select the same files again. Right-click and choose Link Media.... Browse to the avs directory, and for each missing file, choose the corresponding AVS file that matches. IE, for 201-Acceptance.avi, I would choose 201-Acceptance.avs.
- As soon as you are done, save a COPY of your project with a different name, something like "projname-with-avs" or similar.
- To avoid crashing, you will have to export your video in small chunks. I exported Bukowski into 23 separate chunks, each one pretty much just 3-7 clips long. Premiere STILL crashed about 7 times during the process, so be forewarned. You can experiment a bit to see whether you can export longer amounts without crashing.
Note: you export only a chunk by setting the Work Area to just those clips; see the Adobe Premiere help if you aren't sure what the Work Area is or how to set it.
- You will only need to do this next step once, the first time you (successfully, without crashing) export a chunk. Starting at the beginning of your vid, set the Work Area to cover a handful of clips. Then go to File/Export Movie... and hit the Settings button.
- Select "General" and set the following:
File Type: Microsoft AVI
Range: Work Area Bar
Export Video: CHECK
Export Audio: UNCHECK <-- IMPORTANT! don't waste time/disk space exporting the audio here!
Add to Project When Finished: CHECK
- Select "Video" and set the following:
Compressor: None
Frame Size: 848 h 480 v
Frame Rate: 24.00 fps
Pixel Aspect Ratio: Square Pixels (1.0)
- Select Keyframe and Rendering and set the following:
Fields: No Fields (Progressive Scan)
Deinterlace Video Footage: UNCHECK
- Export your first chunk, named something like "chunk01.avi" or similar. Hopefully Premiere will not crash! If it doesn't, go ahead and save your project now, so that the settings you just set up will be saved if (when) you do crash. Each of my small chunks was about 200 MB large.
- At this point, just rinse and repeat -- set the Work Area to the next handful of clips, choose File/Export Movie..., save the next chunk with an incremented number, and then save the project itself. If Premiere crashes during this repeated process, as it probably will, don't get frazzled, just get up, have a drink *g*, and restart it.
- Once you have all the chunks exported, thank the higher power of your choice, the pain-in-the-ass part is done. Go to File/New.../Sequence and create a new sequence. Copy over your audio from your working sequence, and then put down all of the chunks in sequential order.
- With the new sequence selected in the timeline, once again go to File/Export Movie... and hit Settings.
- Select "General" and change the following:
Range: Entire Sequence
Export Audio: CHECKED
Everything else can remain the same.
- Select "Audio" and set the following if not already set:
Compressor: Uncompressed
Sample Rate: 48000 Hz
Sample Type: 16-bit
- Export the final, complete vid as a single AVI file. This will be fairly large; Bukowski was 4.3 GB. This AVI file is now ready to be encoded for distribution.
- For online distribution, I use VirtualDubMod to encode it with DivX. (can also use the open-source XviD, it's a very similar process)
A little bit of proselytizing here: please use DivX/XviD for distribution rather than WMV or Real or Quicktime or mp4. It's a less proprietary format, and anyone can buy a Norcent or some similar cheap DivX player, pop the vid on a disk, and watch it on their TV. That goes for any vids, but in particular, vids made from high-res source will be more than good enough to play at full size on a TV.
- For DVD distribution, I use TMPGEnc Plus 2.5 (note: there are newer versions which I just have never bothered to upgrade to, which are probably better to buy if you are starting now) to encode it to M2V/WAV files.
For Vividcon, I encoded it as a 16:9 file with (these settings will mostly mean nothing to you unless you are using the Advanced tab of TMPGEnc Plus 2.5, sorry) NTSC frame rate (29.97 fps), using 2-pass VBR encoding, min of 4500 bps/average and max of 6500 bps, non-interlace/progressive encode mode, 4:2:0 YUV format, 10 bits DC component precision, and highest quality motion search precision.
And that's all! See, easy! ahahahaaaaaa.