The Art of SQL Query Optimization (jnidzwetzki.github.io)
2 points by thunderbong 19m ago 0 comments
SaaS Launch Checklist for the Vibecoding Era (lightrains.com)
2 points by niksmac 39m ago 1 comments
The iPhone 15 Pro’s Depth Maps
224 marklit 60 6/4/2025, 5:57:15 PM tech.marksblogg.com ↗
1. The original method uses two cameras on the back, taking a picture from both simultaneously and using parallax to construct a depth map, similar to human vision. This was introduced on the iPhone 7 Plus, the first iPhone with two rear cameras (a 1x main camera and 2x telephoto camera.) Since the depth map depends on comparing the two images, it will naturally be limited to the field of view of the narrower lens.
2. A second method was later used on iPhone XR, which has only a single rear camera, using focus pixels on the sensor to roughly gauge depth. The raw result is low-res and imprecise, so it's refined using machine learning. See: https://www.lux.camera/iphone-xr-a-deep-dive-into-depth/
3. An extension of this method was used on an iPhone SE that didn't even have focus pixels, producing depth maps purely based on machine learning. As you would expect, such depth maps have the least correlation to reality, and the system could be fooled by taking a picture of a picture. See: https://www.lux.camera/iphone-se-the-one-eyed-king/
4. The fourth method is used for selfies on iPhones with FaceID; it uses the TrueDepth camera's 3D scanning to produce a depth map. You can see this with the selfie in the article; it has a noticeably fuzzier and low-res look.
You can also see some other auxiliary images in the article, which use white to indicate the human subject, glasses, hair, and skin. Apple calls these portrait effects mattes and they are produced using machine learning.
I made an app that used the depth maps and portrait effects mattes from Portraits for some creative filters. It was pretty fun, but it's no longer available. There are a lot of novel artistic possibilities for depth maps.
Yes, they will capture them from the main photo mode if there’s a subject (human or pet) in the scene.
> I made an app that used the depth maps and portrait effects mattes from Portraits for some creative filters. It was pretty fun, but it's no longer available
What was your app called? Is there any video of it available anywhere? Would be curious to see it!
I also made a little tool, Matte Viewer, as part of my photo tool series - but it’s just for viewing/exporting them, no effects bundled:
https://apps.apple.com/us/app/matte-viewer/id6476831058
One of the example pictures on TFA is a plant. Given that, are you sure iOS is still only taking depth maps for photos that get the "portrait" icon in the gallery? (Or have they maybe expanded the types of possible portrait subjects?)
My previous comment was about the default Photo mode.
If you have a recent iPhone (iPhone 15 or above iirc) try it yourself - taking a photo of a regular object in the standard Photo mode won’t yield a depth map, but one of a person or pet will. Any photo taken from the Portrait mode will yield a depth map.
You can find out more about this feature by googling “iPhone auto portrait mode”.
Apple’s documentation is less helpful with the terminology; they call it “Apply the portrait effect to photos taken in Photo mode”
https://support.apple.com/guide/iphone/edit-portrait-mode-ph...
Ah, that makes sense, thank you!
I think there might be a few typos of the file format?
- 14 instances of "HEIC"
- 3 instances of "HIEC"
I could probably make a better camera app with the correct aperture math, I wonder if people would pay for it or if mobile phone users just wouldn't be able to tell the difference and don't care.
Those methods come from the world of non-realtime CG rendering though - running truly accurate simulations with the aberrations changing across the field on phone hardware at any decent speed is pretty challenging...
What real optics does:
- The blur kernel is a function of the shape of the aperture, which is typically circular at wide aperture and hexagonal at smaller aperture. Not gaussian, not triangular, and the kernel being a function of the depth map itself, it does not parallelize efficiently
- The blurring is a function of the distance to the focal point, is typically closer to a hyperbola; most phone camera apps just use a constant blur and don't even account for this
- Lens aberrations, which are often thought of as defects, but if you generate something too perfect it looks fake
- Diffraction effects happen at sharp points of the mechanical aperture which create starbursts around highlights
- When out-of-focus highlights get blown out, they blow out more than just the center area, they also blow out some of the blurred area. If you clip and then blur, your blurred areas will be less-than-blown-out which also looks fake
Probably a bunch more things I'm not thinking of but you get the idea
i.e. train a very small ML model on various camera parameters vs resulting reciprocal space transfer function.
However, you could just make the app connect to localhost and hoover up the user's data to monetize and then offer the app for free. That would be much less annoying than showing an ad at launch or after every 5 images taken. Or some other scammy app dev method of making freemium apps successful. Ooh, offer loot boxes!!!
No comments yet
As another commenter pointed out, they used to be captured only in Portrait mode, but on recent iPhones they get captured automatically pretty much whenever a subject (human or pet) is detected in the scene.
I make photography apps & tools (https://heliographe.net), and one of the tools I built, Matte Viewer, is specifically for viewing & exporting them: https://apps.apple.com/us/app/matte-viewer/id6476831058
> Copyright © 2014 - 2025 Mark Litwintschik. This site's template is based off a template by Giulio Fidente [1]
The theme is for the Pelican [2] static site generator.
[1] https://github.com/gfidente/pelican-svbhack
[2] https://getpelican.com/
FWIW I personally hate the display of HDR on iPhones (they make the screen brightness higher than the maximum user-specified brightness) and in my own pictures I try to strip HDR gain maps. I still remember the time when HDR meant taking three photos and then stitching them together while removing all underexposed and overexposed parts; the resulting image doesn't carry any information about its HDR-ness.
It also reads EXIF to "scale" the image's physical dimensions to match the field of view of the original capture, so wide-angle photos are physically much larger in VR-Space than telephoto.
In my opinion, this button and feature alone justifies the $4000 I spent on the device. Seeing photos I took with my Nikon D7 in 2007, in full 3D and correct scale, triggers nostalgia and memories I've forgotten I had for many years. It was quite emotional.
Apple is dropping the ball on not making this the primary selling-point of Vision Pro. It's incredible.
Just in case you were doing 3d modeling work or photogrammetry and wanted to know, like I was.
I did end up getting the Creality Ferret SE (via TikTok for like $100) for scanning small objects, and it's amazing.
I’ve also heard good things about Canvas (requires LiDAR) and Scaniverse (LiDAR optional.)
I’d be fine with paying for it, but it’s clear that they want to employ basic dark patterns and false advertising.
Anyways, never heard of oiiotool before! Super cool
Chimera
The old gpu is an aberration and odd place to skimp. If he upgraded to a newer nvidia gpu it would have linux driver support and he could ditch windows entirely
And if he wasn’t married to arcgis he could just get a mac studio
ArcGIS is the best general purpose solution.
If doing mission planning including flight guidance for aerial operations, FalconView is the least worst.
Telling an experienced GIS user to drop ArcGIS is like telling a somehow-still-living B.B. King to stop using Gibson guitars.
And by "better supported" he probably meant "isn't a huge pain in the ass that breaks every other kernel upgrade due to dkms or the package manager shitting the bed" when it comes to Nvidia drivers. Doesn't matter if it's a 1080ti or a 5090.
Is it in any way relevant?
The labels on your food are +/-20%. If you are analyzing all of your meals via camera, it's probably not too far off that over a week.
First: the image recognition model is unlikely to have seen very many depth maps. Seeing one alongside a photo probably won't help it recognize the image any better.
Second: even if the model knew what to do with a depth map, there's no reason to suspect that it'd help in this application. The lack of accuracy in a image-to-calorie-count app doesn't come from problems which a depth map can answer like "is this plate sitting on the table or raised above it"; they come from problems which can't be answered visually like "is this a glass of whole milk or non-fat" or "are these vegetables glossy because they're damp or because they're covered in butter".
> I'm running Ubuntu 24 LTS via Microsoft's Ubuntu for Windows on Windows 11 Pro
this is like hearing someone buying yet another automatic super car.
The details he documented don’t seem relevant for what he did and nothing performed would seem to stress even a low end ancient system.
Definitely felt like a brag to me.
Only thing that makes me think otherwise is he also documented the line counts of the scripts. Seems more like a bizarre obsession with minutiae… (would have been more meaningful to document the git commit/branch for a GitHub project instead of the line count!!)