A Short Expert Level Tutorial on Practical Stereo Panography

[Thomas Sharpless]

Three years ago, stereoscopic 360 photography was a curiosity. Today it is a specialty that has been mastered by dozens of panoramic photographers, who are able to show their work not only as self-published tours built with krpano or Pano2VR, but also on at least 3 mainstream services. 360Cities.net has supported VR viewing on mobile devices for some time. It now has image sets that can be browsed in VR without removing the headset. All VR displays on 360Cities support stereo images. Gala360 publishes photographic tours direct to VR devices such as Gear VR, Google DayDream, Rift and Vive, and fully supports stereo images. Oculus 360Photos, while not now an easily accessible publishing platform, offers high quality display of local stereo images on Gear VR and Rift. Oculus 360Photos and the closely related Facebook 360 Photos service could easily become major publication platforms in future.

In a few more years, as the VR industry matures, I expect stereo 3D to become part of the skill set of most working VR photographers. We already have the tools to make stereoscopic panoramas at a reasonable cost in time and effort, and these tools are only going to get better. What follows is a brief, expert-level tutorial on the workflow I currently use. It is very like the 2D process I have used for a decade, with stereo pairs of photos in place of single ones, and uses familiar tools: Lightroom, PTGui, krpano, Pano2Vr, Photoshop, plus my own stereo stitching helper, PT3D. PT3D works by converting a preliminary PTGui project into a final stitch project that better aligns and fits the images.

The first requisite is a matched pair of small cameras with 180 degree fisheye lenses, and a means to trigger their shutters simultaneaously. I use Sony APS-C format cameras with Samyang 7.5mm (circular fisheye) or 8mm (diagonal fisheye) lenses. Their wired release ports are connected to an isolator box that lets a single RF remote trigger both cameras.

The cameras must be mounted rigidly side by side in portrait orientation, with lens pupils no more than 85mm apart — 65mm is ideal. Lens rings with Arca feet and a horizontal bar work better than schemes involving the cameras’ tripod sockets. For multi-row work I attach the bar to the top arm of a pano head with a right angle bracket, positioned so that the lens pupils are in line with the vertical pivot axis of the pano head. When the cameras are level, the pano head arm points straight down. For single-row I just put the bar on top of a Nodal Nonja R20 mini-head. The vertical rotation axis can be directly under one lens pupil, to ensure one easy 2D stitch, or halfway between the pupils, to minimize the maximum parallax error. I recommend centered but actually use both.

With the circular fisheye I shoot a single row with the camera tilted up about 5 deg. With the diagonal fisheye I shoot two rows tilted up and down 15-20 degrees. Usually I take 6 views around, rarely more than 12, both rows the same number. To avoid the need for nadir patch shots I shoot from a pole with a small swivel foot (and a big level to help me keep it vertical) or from a tiny tripod with a long neck. To minimize the support footprint I “overhang” the cameras a little — pupils slightly in front of the rotation axis — and use an offset pano head for multi-row.

Apart from “center the lens pupil”, all the usual panographic considerations apply. One special one for stereo: put objects close to the camera in the middle of a view, so they will not be cut by stitching seams. Another: mark your cameras and lenses so that you always pair them the same, and always mount the same camera on the left. And a third: if you wind up with a left image and a right image having the same file name and nearly the same time stamp, Lightroom will obstinately refuse to import the second one, on the grounds that it is a “duplicate”. So before starting a shoot, make sure the cams are going to generate different file names.

I shoot raw and use Lightroom to organize and develop the pictures. After importing images from left and right cams into separate folders, I create a LR set holding both (after a day of shooting, that might be a dozen panos worth). Then I tag the left images “green” and the right images “red”. Normally the 2 series are the same size and consist of stereo pairs only; but sometimes they dont….

To ensure matched color, I use a custom color profile for each camera. Other develop settings should be the same left and right. Typically I select the “green” images for a pano, make whatever adjustments I fancy, and export to a stitch folder under sequence-numbered file names beginning with “left-“. Then I copy the develop settings, select the corresponding “red” images, paste the settings, and export to the same folder under names beginning with “right-“. Sequential file names are important because they let the PT3D tool easily recognize the left and right sets. And PT3D really likes that dash in the middle of the file name.

For stitching, PTGui does the heavy lifting: finding control points; warping the images; masking and blending. PT3D handles a few things PTGui is not equipped to do: fine stereo alignment; selecting good control points for a “warp-to-fit” approach to reducing stitch errors; and generating synthetic control points that help keep warp-to-fit from destroying stereo registration.

To do its alignment and registration magic, PT3D needs accurate camera + lens calibrations. You make those with PTGui: mount each camera accurately on-nodal; take a 360 series with heavy overlap; stitch with all lens parameters enabled. Save the project and open it in PT3D as a camera calibration. Once your camera calibrations are in PT3D’s database, you use PT3D to define “rigs” containing two cameras and some details of the mounting geometry; those also go in the database. With a rig, you can stitch.

When you load your images into PTGui, they should be listed with as two consecutive series, all lefts and all rights together and stereo pairs in corresponding poitions. If that is not the case you need to revise your file names. Now it is comparatively easy to select either the left or right pano using the check boxes on the “create panorama” tab. Drag over a series of boxes to select them, hit spacebar to change their state.

Set PTGui’s preferences to generate a lot of control points — 3000 per image, 100000 per pano. PT3D will select a smaller optimum subset; but it works best with a big sample to select from.

Set crop and nominal lens parameters, and invoke “align images”. PTGui should put the left and right image of each pair at about the same position, with about the same small offset at each position. You can now start painting masks to select the contents of your pano. These need not be very precise, since final image alignments may differ a little; but it is good to do the prelimainary masking now. I usually start by masking out the camera support, and anything else I know I don’t want in the pano. Then add green masks on things I know are wanted, and red counter-masks over the corresponding parts of other images. I often decide to discard certain images altogether, and remove them from the project. It is important that the left and right masks agree as to image content.

Save the preliminary project (if using PTGui 11, export it as a PTGUi 10 project) and open it in PT3Dv1. On the Stitch Project tab, verify that the appdv1″ropriate rig definition is selected and that the adjustable parameters are set as you prefer. Then click the RUN button. PT3D quickly generates a new PTGui project file whose name begins with “PT3Dv1”. Open that in PTGui and immediately run the optimizer, for example by pressing the F5 key (in PTGui 11, first open the Optimizer tab and disable all “global lens parameters” at the top). Do not change any setting or invoke “align images”.

The PT3D alignment applies viewpoint and lens-shift to every image, for the purpose of warping them to fit. This almost always un-levels the panorama. You must level it again by dragging in the editor window. Simply make the row(s) of image number labels straight. Right-drag vertically to balance the row left-right, left drag in center to equalize heights. Vertical line control points are not effective in a PT3D stitch and should not be used.

The central parts of the panos should now be free of visible stich errors, but some may remain near nadir (and indoors, near zenith). Those can often be reduced by adding control points in the problem regions, re-optimizing, and re-leveling.

Adjust your masks, taking care that seam lines follow the same paths in the left and right panos. A good deal of art at this stage is in placing seams so that residual errors, both of alignment and stereo registration, are not visible — or at least not obvious. Although warp-to-fit works surprisingly well, it inevitably leaves such errors. Before long it will be replaced by methods that do real depth analysis and achieve much smaller error levels. But as implemented at present, 3D modeling actually creates worse errors than warp-to-fit.

Remember to optimize vignetting. Perhaps tweak exposure offsets. Then select the images of left or right pano, set an appropriate output file name, and stitch. Repeat for the other eye.

Photoshop work to fix remaining defects is much the same as for 2D panos, bearing in mind that the left and right images must agree in detail. I normally make cube faces with krpano and use content-aware fill to erase the support footprint (often including the pole shadow) from the “down” face. For more elaborate patching I use Pano2VR because of its very convenient interface with Photoshop.

Panorama stitching is a complex skill, and stereo adds several new levels of complexity. But with PTGui/PT3D, I can now complete a typical stereo stitch in 1 to 4 hours. This depends mainly on the amount of selection masking required, as in 2D stitching, with a multiplier of about two due to having to paint each mask twice. As the field matures, tools to automate stereo masking will no doubt be developed.

[Author]

Posts