How to Optimize Projects in Adobe After Effects CC

This chapter is from the book 

Image Pipeline, Global Performance Cache, and Render Speed

The render pipeline is the order in which operations happen; by controlling it, you can solve problems and overcome bottlenecks. For the most part, render order is plainly displayed in the timeline and follows consistent rules:

• 2D layers are calculated from the bottom to the top of the layer stack—aka those numbered layers in the timeline.

• Layer properties (masks, effects, transforms, paint, and type) are calculated in strict top-to-bottom order within each layer (twirl down the layer to see it).

  NOTES

  

  The Transform effect allows you to position, scale, or rotate a layer before other effects are applied, solely to avoid precomping.

• 3D layers are calculated based on distance from the camera; coplanar 3D layers respect stacking order and should behave like 2D layers relative to one another.

So to review: In a 2D composition, After Effects starts at the bottom layer and calculates any adjustments to it in the order that properties are shown, top to bottom. Then, it calculates adjustments to the layer above it, composites the two of them together, and moves up the stack in this manner (Figure 4.11). Although effects within a given layer are generally calculated prior to transforms, an adjustment layer guarantees that its effects are rendered after the transforms of all layers below it.

Track mattes and blending modes are applied last, after all other layer properties (masks, effects, and transforms) have been calculated, and after their own mask, effect, and transform data are applied. Therefore, you don’t generally need to pre-render a track matte simply because you’ve added masks and effects to it.

Global Performance Cache: Way Faster!

The feature name Global Performance Cache is a generic term for what is, in fact, a set of interrelated technologies:

• a global RAM cache that is smarter about dividing your work to save as many individual processes as possible
• a persistent disk cache that saves those precalculated processes for continual reuse
• a graphics pipeline that makes greater use of OpenGL to present and stream images onscreen

Global Performance Cache is the result of looking at what modern hardware can deliver that simply was not possible a few years ago, and figuring out how to make use of that hardware:

• cheap and plentiful RAM, and the ability of a 64-bit operating system to access far more of it (up to 192 GB on Windows 7, and well in excess of the 2 GB per processor core recommended for After Effects)
• fast attached storage, including SSD drives that routinely double the access speed of even the fastest HDD drive or array
• high-end graphics cards with GPUs that accelerate performance year after year at rates that way, way outstrip Moore’s Law

Best of all, you don’t really have to do anything special, beyond keeping your hardware up to date.

Memory Acceleration: Global RAM Cache

By slicing a clip with its many selections and effects into discrete chunks and storing each of those render steps individually, After Effects greatly reduces the need to re-render cached footage. You can change a given effect setting or range of keyframes without disrupting other parts of the image and clip that are unrelated to that change.

Reusable frames are recognized anywhere on the timeline: when you use loop expressions (Chapters 8 and 10), remap time, or copy and paste keyframes. Duplicated layers or whole duplicated comps are also recognized.

The net result is that you can commit an edit, preview the result without rendering from scratch, and undo the change without penalty. Since this, in essence, is how you spend your working day as an After Effects artist, the resulting 5–15x speed increase ripples throughout the process, allowing you not only to get to a result more quickly but to try more options without worrying about the time cost.

This tends to work a lot better with 2D layers since in 3D, light, reflection, shadows, refraction, and translucency are all influenced by the adjustment of a single element, such as a light or the position of a layer.

Continuous Access: Persistent Disk Cache

Data in the RAM cache is now much less fragile because it is constantly backed up in a persistent disk cache. If you run out of RAM, increment and save to a new version of the project or even quit the application and reopen the project. Its cache is available for instant playback and immediate rendering (Figure 4.12b).

Persistent disk cache is also the most tweakable of the Global Performance Cache options, and the one for which your choice of hardware may make the greatest difference. Here’s a list of the most effective tweaks, followed by a breakdown:

• dedicate fast attached storage to the After Effects cache
• use the Cache Work Area in Background command as you work
• incorporate Dynamic Link with Adobe Premiere Pro
• render locally

Before drawing out the first three points in more detail, note that the persistent disk cache is not at all sharable or portable. Place the cache on a shared drive and point two systems to it, and all you do is introduce instability: The two systems don’t recognize those cached files in the same way, thus introducing conflicts and instability, and will simply continue to generate their own cache data. The data is designed to be accessed instantly and is cleverly designed to track a given comp and layers even as project versions change on a given system (Figure 4.13).

Disk Cache Boost 1: Get Fast Attached Storage

The persistent disk cache can be a little like a gigantic RAM extension, providing much longer memory and far greater capacity. As such, it’s in your interest to maximize its performance and, if possible, capacity. Why? Not only because faster is better; After Effects actively evaluates whether it’s in the application’s best interests to commit a given process to disk. The greater the difference between processor and cache speed, the more likely a frame gets the blue cache indicator, ready to turn green at any time (and the faster it turns green, the faster it is ready for real time).

If you’re working in After Effects on a laptop, the ideal setup is to install an SSD as your boot disk and main cache. This has the added benefits of rebooting and launching all applications more quickly, but means you probably want two internal drives, so that a larger, cheaper one can be used for longer-term storage.

In a desktop system, an SSD boot drive is equally valid, but you also have higher-performing options, such as the Fusion ioFX, which at this writing has just been raised to 1.6 TB of capacity via a PCI Express slot. It’s sort of like having an SSD RAID, and if you have the cash, it may well be a component in the highest-performing After Effects computer you can build today (Figure 4.14).

Even that striped RAID array you have attached to your system can help you a bunch. Any drive other than the internal boot drive will work better, and if you edit footage professionally, you almost certainly already have just such a dedicated drive available.

Disk Cache Boost 2: Commit a Comp

If you really hate waiting for a comp to preview and have a half-decent system and something better to do with your time, you can select a whole set of comps in the Project panel and cache them. Yes, if you’re on a non-CUDA-enabled MacBook Pro and those comps are all full of HD ray-traced 3D animations, your system is going to sound like a jet preparing for takeoff and your laptop will scorch your lap. On the other hand, if you’re on one of those systems that has more processor cores than you can count when you open up their little capacity meters in the system, well, you are finally going to get your money’s worth.

Caveats? Downsides? You gotta pay to play. This is where gobs of low-latency storage is going to be your new best friend, other than the actual best friend that you get to spend time with when you are done for the day and not already burning the midnight oil. But there’s always that CBB.

Disk Cache Boost 3: Rethink Dynamic Link

Adobe Premiere Pro has a unique ability to link directly to an After Effects comp. Dynamic Link is a feature that allows Adobe Premiere Pro to actually look inside an After Effects project for an existing comp that it can import (Figure 4.15), or designate a clip in a sequence as the basis for a new After Effects comp.

With either approach, there is an actual, live After Effects comp sitting in an Adobe Premiere Pro sequence. After Effects invisibly provides the ability to render it in the background. As any change is made to the comp on the After Effects side, it remains up to date in the Adobe Premiere Pro edit.

One drawback to embedding an After Effects composition into the Adobe Premiere Pro timeline in this manner is that the latter application lacks all of the means immediately at your disposal in After Effects to speed up a preview by lowering settings. It’s all or nothing to cache a clip in the Adobe Premiere Pro timeline, without much certainty how long such a preview will even take.

If you’re thinking that Global Performance Cache helps in such a case, you are correct. Suppose you have a heavy comp that requires 10 seconds to render each frame at full resolution. If you cache the comp at full, Adobe Premiere Pro has access to those cached frames even if After Effects isn’t open. Render the sequence and that clip is ready for real-time playback in seconds, not minutes or hours.

Note that you do, however, still have to render to get rid of the red line above that clip, even if it’s completely cached at full resolution. And, when you do so, it doesn’t add to the After Effects cache. The way to make this work is to generate a preview in After Effects. This still requires you to perform an edit, but once you do so, it helps speed up the Adobe Premiere Pro timeline just as it does in After Effects.

Proxies, Previews, and Network Renders

Previous editions of this book advocated the use of proxies and previews as ways to accelerate the previewing and rendering process. This is exactly where Global Performance Cache changes the game, but only as long as you work on the “one artist, one project, one system” model, given that the cache is neither portable nor sharable.

For this reason, the old ways are still valid in any case where a project needs to be moved or shared, even if only for rendering purposes. The good news is that the cached data helps even this process to happen much more efficiently, because it is also used to render on the system that generated it.

Post-Render Options

Tucked away in the Render Queue panel, but easily visible if you twirl down the arrow next to Output Module, is a menu of three post-render actions to incorporate a render into a project. After the render is complete, you can use

• Import simply to bring the result back into the project
• Import & Replace Usage to replace the usage of the source comp in the project without blowing it away
• Set Proxy to add a proxy to the source (the most elegant solution, but the most high maintenance)

The latter two options even let you use the pick whip icon adjacent to the menu to connect whatever item in the Project panel needs replacement. If you’ve already created a pre-render or proxy, you can target that (Figure 4.16).

Proxies and Pre-Renders

Let’s face it, dutifully rendering proxies is boring and will seem completely unnecessary with all of the new cache features—right up until the moment when you’re in a rush and no longer have access to that cache, either when rendering remotely or handing off the project. Are you willing to buy some insurance on that cache? If so, this section is for you.

Any image or clip in your Project panel can be set with a proxy, which is an imported image or sequence that stands in for that item. Its pixel dimensions, color space, compression, even its length and frame rate, can differ from the item it replaces. You can have a quick-and-dirty still or low-res, compressed, low-frame-rate clip stand in for a render-heavy comp.

To create a proxy, context-click an item in the Project panel and choose Create Proxy > Movie (or Still). A Render Queue item is created and, by default, renders at Draft quality and half-resolution; the Output Module settings create a video file with alpha, so that transparency is preserved and Post-Render Action uses the Set Proxy setting.

Figure 4.17 shows how a proxy appears in the Project panel. Although the scale of the proxy differs from that of the source item, it is scaled automatically so that transform settings remain consistent. This is what proxies seem to have been designed to do: allow a low-resolution file to stand in, temporarily and nondestructively, for the high-resolution final.

There’s another use for proxies. Instead of creating low-res temp versions, you can instead generate final quality pre-rendered elements. With a composition selected, choose Composition > Pre-render and change the settings to Best for Quality and Full for Resolution, making certain that Import and Replace Usage is set for Output Module.

Here’s the key: By default, the source file or composition is used to render unless specifically set otherwise in Render Settings > Proxy Use. Choose Use Comp Proxies Only, Use All Proxies, or Current Settings (Figure 4.18) and proxies can be used in the final render. Thus the speed and quality boost that the proxy provided as you worked can now also contribute to your render, even if the project (with its source) travels to another system.

Background Renders

Rendering from the Render Queue ties up the application and much of the machine’s processing power, which used to mean that renders were left until lunchtime or off-hours. On a modern system with multiple processors, you can do much better than that (but take breaks anyway, they’re good for you).

Adobe Media Encoder

It is too often overlooked that Adobe provides a background rendering application. Adobe Media Encoder (AME) is in many ways a superior alternative to the Render Queue. It can write formats such as DNxHD that After Effects can’t, and it can optimize other formats that benefit from multi-pass rendering, in particular H.264. H.264 is a “long GOP” format, which means that it relies on keyframes with lots of image data surrounded by in-between frames that rely on them, and all of the frames must be rendered before it can work its magic. Only Adobe Media Encoder collects frames to compress them instead of writing each frame as it is rendered, and only it includes presets for many common web video services and mobile devices.

Choose Composition > Add To Adobe Media Encoder Queue to send a comp directly, or you can drag and drop an After Effects project into Adobe Media Encoder and look inside the project for renderable comps (Figure 4.19). You then choose render settings either by selecting them from the Preset Browser or customizing the settings (by clicking on the Preset for the render item and specifying your own, which you can then save as a custom preset of your own).

If you can get used to an unfamiliar user interface that doesn’t match the Render Queue, you begin to reap the greatest reward of AME: background rendering. And once you have created the presets you use most often, you may even find that the UI mismatch isn’t such a big deal.

aerender

Background rendering allows a render to occur without the user interface, allowing you to continue working with it. The aerender application is found alongside the After Effects application on your system but runs via a command line (in Terminal Unix shell on Mac OS or the command shell in Windows). You can drag it into the shell window to run it, or press Enter (Return) to reveal its Unix manual pages. This lists the arguments that can be added in quotes to the command aerender and the location string of the project file.

But that’s all such geeky gobbledygook when you have the BG Renderer script, which gives you access to all of these options via a panel in the After Effects UIs, with no need to type any code.

Network Rendering

The aerender command is also used by third-party rendering solutions that work a lot like BG Renderer but are distributed across multiple machines on a network. These programs can manage renders on multiple machines and perform tricky operations like pause a render until an updated element from 3D is done or automatically re-queue failed renders. Because these third-party rendering options—Rush Render Queue, Pipeline’s Qube!, überware’s Smedge, or Muster by Virtual Vertex, to name a few—also support other terminal-friendly applications, such as Maya and Nuke, it’s an investment facilities that are large enough to have a render farm don’t have to think twice about making.

These are not one-click installs, and they’re generally justified only by dedicated machines and a dedicated nerd to manage it all. If that’s beyond your facility at this point, you can still take advantage of all of this technology via the Cloud or via a service such as Render Rocket. You upload your source files and get back rendered output. The downside for compositors is that we generally require a lot of source data to produce final shots, compared with 3D artists who can sometimes create a final cinematic image with virtually no source.

Watch Folder

The slightly dotty granddaddy of network rendering on After Effects is Watch Folder (File > Watch Folder). Watch Folder looks in a given folder for projects ready to be rendered; these are set up using the Collect Files option. The Adobe Help topic “Network rendering with watch folders and render engines” includes everything you need to know.

Watch Folder is kind of okay on small, intimate networks, but it requires much more hands-on effort than dedicated render management software, and it breaks easily, at which point it requires human intervention. Since individual systems have become so powerful, it’s easy to become lazy about taking the trouble required to set up a Watch Folder render, but if you’re up against a deadline, don’t have the dedicated software, and want to maximize multiple machines, it will do the trick.