First Generation Image Quality part 4: Depth Maps and Depth-Map Errors

Depth Maps

A depth map is a layer of data that defines the distance of various surfaces from the lens of the camera.

(In other words, a depth map is like a kind of topographic map, only for pictures instead of terrain.)

The Lytro software uses a special depth map that is key to refocusing pictures and enabling Perspective Shift.

 

Depth-Map errors

Estimating relative distances accurately is one of the most difficult aspects of computing the light field, especially under the following conditions:

  1. Any special-case light behavior - reflective surfaces, transparent surfaces, and so on.
  2. Limited or misleading light information - shadowed areas and similarly colored areas.
  3. Geometrical irregularities - situations where lines intersect, curve, or come in or out of the composition.

 

Depth-Map errors when refocusing pictures

Although it is possible to find depth-map errors in some pictures just by trying to refocus them, this can be tough to replicate unless you click at just the right spot.  

The most common way people notice a depth-map error when refocusing is by clicking part of the picture, say, someone's eye, but the picture still doesn't refocus correctly to that depth level.  (However, when you click on something else in the picture at the same relative depth, maybe the top of their cheek, the picture refocuses correctly.) 

 

Depth-Map errors and Perspective Shift

Pictures that have Perspective Shift enabled tend to show any depth-map errors more prominently.  This is true for several reasons:

  • Almost all of the light information captured in that scene is being presented, so the chances of an error somewhere in the picture are greater than the odds of an error at one particular spot.
  • Exploring Perspective Shift animates the focus layers in the picture, and as they move over and around each other, our eyes are naturally drawn to something that's "not quite right."  
  • As already discussed, depth-map errors are especially common at the boundaries of a surface.  That means the border area around your foreground subject is especially likely to have an error, and that's someplace you're quite likely to look.
Here are two examples of depth-map errors that are especially noticeable as you click and drag your mouse:
  • In the picture on the left, look in the upper left corner and the lower right corner of the picture.
  • In the picture on the right, look between-- and at the tips of-- the pine needles.

      



Avoiding Depth-Map errors

  • For the time being, as a workaround (especially whenever one of the contributing conditions mentioned above are in play), we recommend always taking a few shots of your composition whenever possible.  This increases the chances you'll get at least one shot that's to your liking.
  • Also, since the more light we have to work with, the lower the chance of an error, you might also try taking at least one of those back-up shots using Manual Controls:  
    • Specifically, you might want to consider a longer exposure (in which case, be sure to steady the camera, either with our Tripod Mount or by using something in the environment like a counter surface, plus our Self-Timer feature), as well as experimenting with a higher ISO. 

Future developments

Depth-map errors are definitely something that we would like to improve over time in software.

  • NOTE: All such improvements will require 'reprocessing' those pictures-- i.e. removing and re-adding Perspective Shift.
  • We are also considering editing tools to allow you to correct depth-map errors yourself.  However, this is demanding engineering work, and we don't have a timeline for such improvements.

 

 

Comments

Powered by Zendesk