For depth from stereo software workflows the big news for me has been the release of Photoshop 2020. This incorporates Adobe Sensei AI into its selection tools:
https://photoshopcafe.com/photoshop-2020-upgrade-new-features-use/
"The Object Selection tool is very exciting. It comes in 2 flavors, rectangle and lasso. This amazing tool enables you to drag over and object in a photo or image and then Photoshop magically selects it for you, like Select Subject, but more focused. You could select it with the rectangle and then fine tune with the Lasso option. As always Shift Adds to the selection, while Alt/Option subtracts. This is like a Magic wand and Magnetic lasso in one, but its powered by Sensei, Adobe’s AI engine."
This makes masking so much simpler for depth map correction and background removal for better depth map calculation for foreground elements. It is very interesting how it really seems to know what kind of objects you are interested in and it works really quickly so the process is interactive. Btw you can fine tune your contours by slightly going over into the subject's area with the object lasso tool in a second pass -- something that is not intuitive. Topaz AI Mask is still useful I think for semi-transparent detail like hair.
I have been paying attention to AI scene segmentation software developments on Arxiv etc. This is dividing up all of an image into meaningful objects. I am sure this is the future for depth map extraction. There is also the notion of "panoptic scene segmentation". This is not necessarily to do with panoramas -- it is more to do with with a complete analysis of every pixel in the frame -- something we have to do when creating depth maps
"In the panoptic segmentation task we need to classify all the pixels in the image as belonging to a class label, yet also identify what instance of that class they belong to."
https://medium.com/@danielmechea/what-is-panoptic-segmentation-and-why-you-should-care-7f6c953d2a6a
https://scholar.google.com.au/scholar?q=panoptic+scene+segmentation&hl=en&as_sdt=0&as_vis=1&oi=scholart
Stereo panorama photographer and software developer Thomas Sharpless has published an interesting workflow for two row stereo panorama capture and stitching (with PTGui11) with extremely sharp looking results -- on the 360 Stereo Panoramas Facebook group. PTGui11 has powerful warp to fit features. The two row approach serves to constrain drastic warping to the bottom row (closer to the ground and hence more needed) ... and also to better remove vertical parallax for further depth map from stitched stereo panorama pair extraction.
https://www.facebook.com/groups/3dstereopanoramas/permalink/2495865310679441/
https://www.facebook.com/groups/3dstereopanoramas/
Currently my main preoccupation with depth map improvement involves selecting (stereo) layers of foreground elements in the source stereo pairs -- and using depth from stereo (Fusion/KartaVR) on those.Then compositing the depth elements back together. Also I have been investigating guided depth map improvement (from the source -- aka "feature" image). Like I described in my last post with JointWMF:
https://stereopanoramas.blogspot.com/2019/11/depth-from-stereo-depth-map-retouching.html
Now I am working to see if I can get this new "PixTransform" depth map superresolution software working -- it is primarily for smart upscaling depth maps -- with more detail from the source image but I think it might work for depth map improvement generally.
https://arxiv.org/abs/1904.01501
https://github.com/riccardodelutio/PixTransform
https://medium.com/ecovisioneth/guided-super-resolution-as-pixel-to-pixel-transformation-dad13dfc76cb
More news:
Lucid -- who work with Red and eYs3D -- and had an early VR180 type consumer camera -- have released a beta AI phone app for depth map from monocular: LucidPix 3D Photo Creator.
https://www.lucidpix.com/
https://www.marketwatch.com/press-release/lucid-partners-with-eys3d-etrons-subsidiary-to-create-the-first-vr180-depth-camera-module-2019-01-08
Raspberry Pi have a stereo/360 camera solution now -- and there is also a hardware chip for realtime AI depth videos
https://www.raspberrypi.org/blog/stereoscopic-photography-stereopi-raspberry-pi/
https://www.crowdsupply.com/luxonis/depthai
There are two camera types available I think -- one is an M12 lens mount sensor that can take fisheye lenses eg. Entaniya etc -- the other is a smaller sensor. I am not sure if the AI board will work with fisheye input. On 3dphoto.net they say the sync is good for stills but not so good for video in one post. You can stream stereo fisheye views directly from a Raspberry Pi into Oculus Go.
Google says the next version of ARCore for Android phones with its Depth API will provide depth capture for regular Android phones (no special lenses etc required).
https://www.theverge.com/2019/12/9/20999646/google-arcore-augmented-reality-updates-occlusion-physics-depth
Intel have released details of the next version of their Realsense depth camera
the Realsense Lidar -- these are small enough to clip onto stereo rigs etc without being too obstructive
https://www.anandtech.com/show/15220/intel-announces-realsense-lidar-depth-camera-for-indoor-applications
