All Articles Photoshop
Photoshop: Finessing a Mask with Selection Tricks
Iain Anderson on Mon, January 21st 0 comments
Masks are wonderful things in Photoshop... but they rarely are perfect straight out of the box. Iain Anderson shows how to improve them with some selection tricks.

There are many ways to create a mask in Photoshop, and even more ways to improve that mask. We’ve covered some of these before, but here we’ll look at how to move seamlessly between selections and masks, using one to improve the other.


Start from a basic selection

The content will define the strategy you need to use to get your basic selection. Sometimes the Quick Selection Tool will be the best option, sometimes Select > Color Range. Even the tools I love to hate, the Magic Wand Tool and the Magnetic Lasso Tool, can have a place. Mostly, it comes down to a few things:

If the object has crisp edges and is a different color to the background, use the Quick Selection Tool, or the Magic Wand Tool if that fails.

If the object has soft edges and is a different color to the background, use the Color Range Tool, or start by Command-clicking one of the Channels (R/G/B) in your image.

If the object is at least partly the same color as the background, you’ll have to use the Pen Tool to define a clipping path, or paint a selection in Quick Mask or a Layer Mask. That’s where it can get messy.

Cutting out this awesome birthday cake is a bit of a challenge — the Quick Selection Tool makes a mess of it, there are too many colors for Color Range, and the edges are soft — a mask would need to be manually painted.

Iffy selection


Finesse the selection with Refine Edge

If your mask is nearly there, one of the easiest ways to clean up a selection is to use Refine Edge. You can:

  • use Feather to soften the selection.
  • use Contrast to crispen the selection.
  • use Smooth to take sharp direction changes (like spikes) out of the selection edges — but this will ruin soft edges like hair.
  • use Shift Edge to move edges in or out.

For a smoother look, you could use settings like these:

Refine Edge Settings


Alternatively, leave all those settings at 0, then:

  • Use the Refine Radius Tool to paint over transition areas like hair. Here’s what you can do with a little work with the Refine Radius tool:

Refine Edge for hair


  • Export with decontamination, to a new layer.
  • Finally, you should choose to output to a New Layer with Layer Mask, and Decontaminate Colors to correct any for thin edges of your foreground object that have taken on the color of your background. On a Layer Mask, painting with black will hide your image, and painting with white will show it again — this is the key to making the mask perfect.

A layer with a layer mask.

A layer with a layer mask.


Dodge and Burn

Now that you have a Layer Mask, you can paint the selection to make it good, or you can use the Dodge Tool (set to Highlights) and Burn Tool (set to Shadows) to make your job easier. This technique was discussed in an earlier article here so check it out now.

Dodge the Highlights, Burn the Shadows.

Dodge the Highlights, Burn the Shadows.


Using fancy selections to finesse your mask

Even the best mask isn’t perfect. Maybe part of it could be sharper, or maybe you need to punch a hole in it — or reveal the transparent part of a glass. To tweak part of your mask, the easiest way is often to select part of your image with Channels. If you Command-click the Red, Green or Blue channel in your Channels panel, you’ll load it as a selection. Select > Color Range... can also work well here if the areas you want to select are more complex.

Clicking just once on the back of the glass in the main image was enough to select it — use Fuzziness to get it right.

Clicking just once on the back of the glass in the main image was enough to select it — use Fuzziness to get it right.


Enter Quick Mask

Press Q to enter Quick Mask, converting your selection back into a temporary mask. In a Mask, white and black map to selected and not selected, with a grey range connecting the two smoothly. However, it’s unlikely that the areas you want to change on your mask are all white.

Here’s the selection viewed as a Quick Mask, with RGB hidden (and note, it’s possible to invert this display if desired).

Here’s the selection viewed as a Quick Mask, with RGB hidden (and note, it’s possible to invert this display if desired).


Press the eye next to the RGB channel to hide it, and you’ll see your Quick Mask alone, as a greyscale image. Press Command-L to access old-school destructive Levels, then tweak to make the part of the image you want to adjust white and the rest black. Use Command-I to invert the mask if it’s all wrong.

Here’s the old-school Levels command — only good for masks! — in action.

Here’s the old-school Levels command — only good for masks! — in action.


Exit Quick Mask

Press Q again to exit Quick Mask, and you’ll be able to use your new selection to limit your painting. Click on your Layer Mask, then paint with black or white through your selection. Black hides and white reveals your image, and you can press Command-H to hide the marching ants around the selection if they prove distracting.

With the glass interior selected, paint black on the existing Layer Mask to hide it.

With the glass interior selected, paint black on the existing Layer Mask to hide it.


Using the image data like this is ideal for fixing up any problems with your mask. Also, don’t be afraid to invert your selection with Command-Shift-I if it’s the wrong way round.

Masks are wonderful things, yet few masks are perfect out of the box. The more tools you have at your disposal, the better your chances of making a mask as good as you can.

Related Videos
Comments (0)

You must be logged in to comment.

Selection & Masking Techniques
Photoshop CS6 102
Dream It. Do It.
Do you want to learn Selection & Masking Techniques?
Yes, I want to learn!
No Thanks, I just want to read the article.
Feedback
Course Advisor
Don't Know Where To Start?
Ask A Course Advisor
Ask Us!
Copy the link below and paste it into an email, forum, or Facebook to share this with your friends.
Make money when you share our links
Become a macProVideo.com Affiliate!
The current affiliate rate is: 50%
Classes Start Next Week!
Live 8-week Online Certification Classes for: