On the publishing side:
Most significantly, krpano, has released a version with support for image+depth 360 panoramas. It supports anaglyph html output for the desktop -- and WebVR immersive 6DOF or 3DOF stereoscopic panorama viewing - in supporting browsers (eg. Firefox Reality) - on vr headsets eg. Rift, Vive, Go and Quest.
https://krpano.com/forum/wbb/index.php?page=Thread&postID=79218
https://krpano.com/forum/wbb/index.php?page=Thread&postID=79218
On the 360 depth map creation side:
most significantly, Holokilo has released an alpha version of an application - Deep360 (free) - that uses AI to automatically produce a depth map from monoscopic equirectangular 360 panorama images.
https://holokilo.wikividi.com
The developer Kaj Toet says he is mainly working on a version for 360 mono videos but this works amazingly well for some 360 stills. It outputs depth maps and over-under stereo. (He also has an Android app for any kind of image, not necessarily panoramic -- https://play.google.com/store/apps/details?id=net.kajos.holokilo&hl=en )
More recent versions of his Windows app have a number of controls:
"Limit Stereo Edge" means fade off the stereo at zenith and nadir (to avoid stereo viewing failing there with the Over-Under output -- a universal issue with ODS (Omnidirectional Stereo) formats generally -- but absent with image+depth panos as correct stereo is generated dynamically in any direction with image+depth pano players)
Other ODS issues are problems when your head is "rolled" and very wide angle, even mildly tilted views. Dersch says ODS is "poison" for good panoramic stereo in one of his articles. (Depth maps are not the only way to avoid being poisoned -- light field technologies eg. Google's Welcome to Light Fields demo, has perfect stereo panoramic viewing in any direction as well as some 6DOF movement capability volume. https://store.steampowered.com/app/771310/Welcome_to_Light_Fields/ )
"3D strength" means what it says -- it controls the amount of disparity produced -- which corresponds with the depth map to how contrasty it is. I usually have it set to between 3 and 6.
"Dilate filter" controls how much the depth map edges are expanded past the corresponding edges of the source pano image elements ie. how much it overlaps it. I am not sure why he has this -- it improves the over-under output quality I think. But I usually set it to zero as I am primarily interested in the depth map output and clean contours are more generally useful I think (I can generate stereo formats from image+depth with Stereophoto Maker with one of its depth tools or with After Effects with the Displacement filter).
http://stereo.jpn.org/eng/
btw ... about ODS rendering: https://developers.google.com/vr/jump/rendering-ods-content.pdf
How does Deep360 do its magic! I don't know ... but "Deep" means -- with AI -- of course -- but what kinds of AI? One common AI approach for depth from mono - for one kind of subject - seems to be with the cool-sounding "Manhattan Scene Understanding" concepts:
http://grail.cs.washington.edu/projects/manhattan/manhattan.pdf
"We focus on the problem of recovering depth maps, as opposed to full object models. The key idea is to replace the smoothness prior used in traditional methods with priors that are more appropriate. To this end we invoke the so-called Manhattan-world assumption [10], which states that all surfaces in the world are aligned with three dominant directions, typically corresponding to the X, Y, and Z axes; i.e., the world is piecewise-axis-alignedplanar."
Other methods of producing depth from monocular 360 images
Depth (or modeling) from monocular 360 images is a very popular AI research topic but there are few software implementations that just anyone, without machine learning software compilation know how, can use.
Matterport, the commercial realestate 3d scanning company has 360 image support. They call their technology Cortex AI. It is for generating models from building interiors with 360 photography.
https://matterport.com/360-cameras/
Everpano, a image-based modeling application, is a sister program for krpano image+depth panorama publishing. It is rather basic in the sorts of models it can produce (it is mainly for interiors) -- but it can model directly from equirectangular panoramas. It also has scripting features (the Navigator Plugin) for 6dof navigation around the models Everpano makes from 360 pictures -- with krpano.
https://everpano.com/
Krpano has a utility which will produce a panoramic depth map from Everpano model if your primary output is going to be image+depth. It also has a tool for matching depth map produced scale to the real world .. important for vr exploration.
In the next release, the krpano developer says, with Everpano's Navigator Plugin, we will be able to navigate intuitively in 6Dof, in VR with Quest say, with image+ depth maps alone by clicking on or moving around multiple panoramas.
https://krpano.com/forum/wbb/index.php?page=Thread&threadID=16984
More depth from monocular AI software:
Masuji Suto's (of Stereophoto Maker) implementation of Google Mannequin Challenge software:
"Learning the Depths of Moving People by Watching Frozen People."https://www.youtube.com/watch?v=prMk6Znm4Bc
https://github.com/google/mannequinchallenge
http://stereo.jpn.org/jpn/stphmkr/google/indexe.html
This requires a recent CUDA card to work (I have it working on a PC with a nvidia GTX 1650). This is I think the most basic deep learning capable card it will work on. The output resolution is highly dependent on the amount of video ram on the card.. The maximum it will produce from a panorama with this card (4Gb video memory) is 512*256. (By processing the panorama in sections you might be able to get more resolution but I don't know how well the depth map would stitch.)
Retouching automatic depth from monoscopic panorama depth maps:There are multiple AI depth from mono programs now that are quite specialized in the sort of subjects that they work with. Using different AI for different problem areas in a 360 pano depth map can be an efficient method.
FACES
For instance there is this online service for automatic face models with AI
3D Face Reconstruction from a Single Image https://cvl-demos.cs.nott.ac.uk/vrn/
https://www.engadget.com/2017/09/19/AI-3d-face-single-photo/?guccounter=1
So you cut out the faces in your mono panorama, upload them, download the models and produce the depth maps. How to produce the depth map? -- you could export a depth channel with After Effects ... or use this workflow with Meshlab (free) -- which has nice controls for the depth map tones ..
https://community.glowforge.com/t/tutorial-creating-a-depth-map-from-a-3d-model-for-3d-engraving/6659
http://www.meshlab.net/
... and then fix the faces in your panoramic depth map image.
Or there are mobile phone apps that do a good job producing depth maps of faces and bodies ... popular apps I guess with phone uses whose phone does not do depth maps with hardware .. and who want depth of field control with portraits.
eg. DPTH
"uses AI to predict depth even on single camera phones"
https://play.google.com/store/apps/details?id=ru.roadar.dpth&hl=en
...the paid version lets you save the depth map .. for panoramic images depth map capture you can put the image on a screen and photograph the relevant area with the phone
More generally ..
Often with a panoramic depth map you would want to extract rectilinear views from the area of the source image that needs retouching on the depth map -- with PTGui say, and build a model from that image area, make a depth map from the model and insert that depth map back into the equirectangular depth map image -- again with PTGui.
So, for instance, if you wanted a depth map of a panorama of that amazing giant crystal cave in Mexico you could do it like this in Photoshop -- much more easily in a rectilinear view than in equirectangular:
"Tutorial on manually creating depth-maps in Photoshop for pictures with complex depths."
https://www.youtube.com/watch?v=GUGafT3WWl4
Gimpel3d as a image+depth editor and previewer
Gimpel3d supports 360 cylindrical and dome panoramic format input images -- as well as rectilinear. It only runs on one of my PC's (?)-- but it is very interesting but hard to get a handle on documentation-wise. It has SBS export and depth map input and export, and depth brushes.
https://sourceforge.net/projects/gimpel3d/
http://www.freeterrainviewer.com/G3DSite/index.html
https://www.youtube.com/watch?v=jxnlFqjQIVk
https://www.stereoscopynews.com/hotnews/3d-technology/2d-to-3d-conversion/1647-gimpel3d2-conversion-software.html
http://3dstereophoto.blogspot.com/2013/10/2d-to-3d-conversion-using-gimpel3d.html
It works very well as an interactive anaglyph 6dof explorer player of wide angle image+depth extracts from Deep360 panos -- on the desktop.
https://www.google.com/search?q=depth+from+monoscopic+panoramas:
... one year's (2019) worth of studies:
https://richardt.name/publications/megaparallax/MegaParallax-BertelEtAl-TVCG2019.pdf
https://hal-mines-paristech.archives-ouvertes.fr/hal-
01915197/file/Cinematic_Virtual_Reality_with_Motion_Parallax_from_a_Single_Monoscopic_Omnidirectional_Image.pdf
Post-stitching Depth Adjustment for Stereoscopic Panorama
https://ieeexplore.ieee.org/document/8682724
"For those regions that still suffer from visible depth error after the global correction, we select control points in those depth anomaly patches and utilize the Thin-Plate-Spline technique to warp those features into their target position."
Thin-Plate-Spline" warping is in ImageJ (Fiji). It is a really fast filter.
https://imagej.net/BigWarp
More than you ever wanted to know on image stitching tactics generally:
A survey on image and video stitching.
https://www.sciencedirect.com/science/article/pii/S2096579619300063
https://www.researchgate.net/publication/336304243_Real-Time_Panoramic_Depth_Maps_from_Omni-directional_Stereo_Images_for_6_DoF_Videos_in_Virtual_Reality
https://www.youtube.com/watch?v=XLOcrAZtc7w
This is a very clear talk. They are working with Suometry -- who have a fast realtime stereo pano capture camera technology. They use stereo panos and normal maps to train realtime depth from mono.
https://web.stanford.edu/~jayantt/data/icme16.pdf
DEPTH AUGMENTED STEREO PANORAMA FOR CINEMATIC VIRTUAL REALITY WITH
HEAD-MOTION PARALLAX
.. a cure for ODS .. "The proposed representation supports head-motion parallax for translations along arbitrary directions."
Motion parallax for 360° RGBD video
http://webdiis.unizar.es/~aserrano/projects/VR-6dof
they have 3 video layers to reduce occlusion errors ... there is a demo:
http://webdiis.unizar.es/~aserrano/projects/VR-6dof.html#downloads
https://www.vr-if.org/wp-content/uploads/Reality-Capture-Present-and-Future.pdf
his prediction 2018-2019:
'“3DOF+” is introduced and begins to gain traction in high-end offline productions, supplanting stereoscopic 360 almost entirely' mmm..
