Creating a segmentation boundary
You use the Roto Brush tool to specify which parts of the clip are in the foreground and which are in the background. You add strokes to distinguish the two, and then After Effects creates a segmentation boundary between the foreground and background.
Creating a base frame
To use the Roto Brush tool to isolate a foreground object, you start by adding strokes to a base frame to identify foreground and background areas. You can start on any frame in the clip, but in this exercise, you’ll use the first frame as the base frame. First, you’ll add the strokes that identify the bird and the rock pool as the foreground objects.
Move the current-time indicator across the time ruler to preview the footage.
Press the Home key to move the current-time indicator to the beginning of the time ruler.
Select the Roto Brush tool (
) in the Tools panel.
You use the Roto Brush tool in the Layer panel, which you’ll open now.
Double-click the Chickadee.mov layer in the Timeline panel to open the clip in the Layer panel.
Choose Fit from the Magnification Ratio pop-up menu at the bottom of the Layer panel if you don’t see the entire image clearly.
By default, the Roto Brush tool creates green foreground strokes. You’ll start by adding strokes to the foreground—the rock pool and the bird. Generally, it’s most efficient to start with broad strokes and then use smaller brushes to refine the border.
Choose Window > Brushes to open the Brushes panel. Set up a hard round 100-pixel brush. (You may need to resize the Brushes panel to see the options.)
When you’re drawing strokes to define the foreground object, follow the skeletal structure of the subject. Unlike traditional rotoscoping, you don’t need to define a precise boundary around the object. Start with broad strokes, and work down to small regions as After Effects determines where the boundary is supposed to be.
Draw a green stroke horizontally across the rock pool.
In the first Layers panel image top, a thick green stroke is drawn using a large brush horizonally across the rock pool at the bottom of the image. The second image shows the resulting selected area in dark pink. The image on the right shows the Brushes panel with a diameter of 100 pixels and a Hardness of 100%. All other options are at their default settings.
Roto Brush tool properties appear in the Effect Controls panel. In the Layer panel, a magenta outline identifies the boundaries After Effects created for the foreground object. After Effects recognized only about half of the pool, because you initially sampled only a small area of the subject. You’ll help After Effects find the boundary by adding some more foreground strokes. First, you’ll make sure you’re using version 3 of the Roto Brush tool.
In the Effect Controls panel, make sure 3.0 is selected for the Version.
Still using a large brush, continue to draw green strokes along the inside edges of the pool. Do not include the branches directly behind the pool and the bird.
Zoom to 200% and use the Hand tool (
) to pan to the bird.
Select the Roto Brush tool again. Then select a smaller brush, and draw a green stroke across the bird’s body.
It can be tricky to get some areas without accidentally adding background as well. It’s okay if you haven’t captured every detail in the foreground. You’ll use background strokes to remove any extraneous areas of the matte.

The first image shows pressing the Alt (Windows) or Option (macOS) to switch to the red brush to the background and apply a stroke to exclude the background area behind the bird (below the tail). The second image shows the result.
Press Alt (Windows) or Option (macOS) to switch to the red background stroke brush. Add red strokes to background areas you want to exclude from the matte.
Switch back and forth between the foreground and background brushes and brush sizes to fine-tune the matte. Don’t forget to deselect the branches directly behind the bird and the pool. In some cases, one click may be all it takes to exclude an object or area from the matte.
When you’re happy with the bird’s matte, zoom back out and fine-tune any remaining areas that need to be addressed with the rock pool.
Don’t worry about being exact about your brush strokes. Just make sure the matte is within 1 to 2 pixels of the edge of the foreground object. You’ll have an opportunity to refine the matte later. However, After Effects uses the information on the base frame to adjust the matte for the rest of the span, so you want the matte to be accurate.

The first image shows the Toggle Alpha view which displays selected area is white against a black background. The second image shows the Toggle Alpha Overlay view which displays the foreground areas in color, and the background with a red overlay. The third image shows the Toggle Alpha Boundary view which displays the outline around the bird and rock pool again.
Click the Toggle Alpha button (
) at the bottom of the Layer panel. The selected area is white against a black background, so you can see the matte clearly.
Click the Toggle Alpha Overlay button (
) at the bottom of the Layer panel. The foreground area appears in color, and the background has a red overlay.
Click the Toggle Alpha Boundary button (
) at the bottom of the Layer panel to see the outline around the bird and rock pool again.
As you use the Roto Brush tool, the Alpha Boundary is the best way to see how accurate your boundary is, because you can see everything in the frame. However, the Alpha and Alpha Overlay options let you see your matte without the distraction of the background.
Refining the boundary across the span
You used the Roto Brush tool to create a base frame, which includes a segmentation boundary that divides the foreground from the background. The Roto Brush span appears below the time ruler at the bottom of the Layer panel. When After Effects has propagated the current boundary for a frame, the bar beneath that frame on the time ruler is green.
As you move forward and backward through the footage, the segmentation boundary moves with the foreground object (in this case, the bird and the rock pool). You’ll step through the frames in the span and make adjustments to the segmentation boundary as necessary.
Because the rock pool is stationary, its track matte is consistent throughout the span. Given that, we’ll focus on the bird, which moves around a lot.
Zoom in on the bird, and then make sure the Roto Brush tool is selected in the Tools panel.
Press the 2 key on your main keyboard (not the numeric keypad) to move forward one frame.
Working from the base frame, After Effects tracks the edge of the object and attempts to follow its movement. Depending on how complex your foreground and background elements are, the boundary may or may not conform exactly to the area you hoped it would.
Using the Roto Brush tool, paint foreground and background strokes to refine the matte for this frame. If the matte is accurate, you don’t need to paint any strokes.
As you progress through this clip, you’ll notice changes to the segmentation boundary with the bird’s beak. Likewise, you may need to refine the segmentation boundary around the feet and legs.
Press the 2 key again to move forward to the next frame.
Use the Roto Brush tool to add to the foreground or subtract from the background as necessary to refine the boundary.
If you make a stroke you don’t like, you can always undo the stroke and try again. As you move through the span, each change you make affects the frames after it. The more you refine your selection, the better the overall results will be. You may find it useful to move forward a few frames to see how changes affect the boundary.
When the bird is moving quickly (such as 0:14 –0:19), you’ll notice that the segmentation boundary becomes more vague, but then “catches up” once the bird slows down again. When a frame displays rapid movement, focus on removing areas that should be excluded (such as the branch behind the bird), rather than adding blurry areas of the bird.

The first image shows the branch behind the bird included with the matte. The second image shows excluding the branch using a small brush. The third image show the result.
Repeat steps 4 and 5 until you reach the end of the layer.
Once you’re satisfied with the bird, zoom out to fit the entire screen, and scroll though the span, making any adjustments as needed.
When you have completed refining the segmentation boundary for the entire layer, choose File > Save to save your work so far.
The three images show the entire screen. first image shows the bird in its initial perched position looking down. The second image shows the bird tipping down to drink from the rock pool. The third image shows the bird back upright and looking to the left. The matte tracks the boundaries correctly throughout.