Support Ukraine

Ideally, a VR panorama is built from an image map covering a full 360 by 180 degrees. When NASA set their rovers to photograph the "Greely" and "Presidential" panoramas, though, they concentrated on capturing the view along the horizon. Understandable, as this is where the interesting stuff tend to be.

Still, the effect is like looking out at the world through a bunker slit. Bigshot is about big imposing images of big imposing views, so I set out to do something about it.

​1. The Problem

Faking image data that was never captured in the first place is a difficult task. Photoshop tries to do it with the content-aware fill, but I'd say that tool is more miss than hit. I was also not going to implement anything that complex: The user always has the option of creating a nadir image using whatever tool they fancy, and trying to invent an automatic process that gives such good results would extend the scope of Bigshot way beyond the limits I've set.

What I would like to have is basically some way to fill in blanks like the Mars sky in the "Greely" panorama. It's not that detailed, to say the least.

​2. The Solution

For simplicity, let's just consider the case of filling in a blank regions surrounding the zenith of a VR panorama, such as the Martian sky. What I was looking for was a way to extend the topmost row of pixels of the image map all the way to the zenith point. I started with two assumptions:

1. The zenith point is the average of all the pixels in the topmost row of the image map.
2. The transition from the image map to the filled-in area is smooth. That is, the limit of the algorithm, when approaching the topmost row from above, is the topmost row itself. This means that the pixels just above the topmost line of the image map should be the average of the nearest map pixel and perhaps parts of another pixel.

This gave the algorithm two boundary conditions.

Let  w_(m)  be the width, in pixels, of the image map. Let  phi  be the angle above the horizon, increasing as we go above the horizon toward zenith. We then define  phi_(0)  as the angle of the top row in the image map. Since zenith is at  pi/2 , the distance from the top row to zenith is  pi/2 - phi_(0) . We can then define a function that goes from 0 to 1 on the interval  [phi_(0), pi/2] :

The plan now is to average more and more of the topmost line of pixels as we go toward zenith. At zenith, we average the whole row. At any other point, we'll average the  w(phi)*w_(m)  pixels closest to the sought pixel, with  w(phi)  being rounded up to one if it is zero - we do want at least one source pixel!

​3. Adjustments

The above algorithm works fine, but I wanted to experiment with the  w(phi)  function. Using it as a base, I finally settled for a piecewise defined function:

Some other attempts:

(eq.3)

  W_(phi) = (w(phi))/2  

This one modified one of the boundary conditions, and while I think the transition from the map to the filled area looked nicer, it resulted in a vortex at zenith.

(eq.4)

  W_(phi) = w(phi)^1.5  

(eq.5)

  W_(phi) = w(phi)^2  

(eq.6)

  W_(phi) = w(phi)^3  

All of these had the same problem: The transition from the map to the filled region was too gradual, and it looked like the sky was streched out - this is especially visible in the last image.

​4. Nadir

While the sky is often featureless - as the Martian sky demonstrates - the ground is usually full of detail. This is what the nadir point corresponding to the skies shown above looks like when filled in:

While you may write up the strange clouds in the sky as curious atmospheric phenomena, this one looks filled-in. But I'd still say it's good enough. If you want better nadir points, create a proper nadir image and insert it into the VR image map.

​5. The New Stuff

Bigshot's MakeImagePyramid now has two options for this: --top-cap and --bottom-cap. The former fills in any zenith-blank and the latter any nadir-blank.