Photos does a lot of extra work on import (merging RAW+JPEG pairs, generating previews, database indexing, optional deletion), so my guess is a concurrency bug where a buffer gets reused or a file handle is closed before the copy finishes.
Rare, nondeterministic corruption fits the profile.
inanutshellus · 27m ago
Also interesting that he went from one day's import damaged 30% of his photos, but after replacing everything he had gotten to the point where it took a while to even get a single corrupt photo.
Random is random, and random is clumpy, so maybe swapping parts is irrelevant, but... I wanted more detail on his parts-replacement journey.
He doesn't mention how often the corruption happened whilst he swapped out parts, unfortunately. Presumably it was too rare the entire journey.
bluSCALE4 · 1m ago
I'd be interested in knowing if he was multitasking and using a lot of memory. I know wedding photos are usually something you feel rushed to upload so maybe this issue can be made worse depending on system resource availability.
tenderlove · 1h ago
This is also my guess. It's really a bummer, and I'd report it to Apple but since it's nondeterministic I have no idea how to provide repro steps.
ChrisMarshallNY · 1h ago
I have had extremely bad luck, reporting bugs to Apple.
They constantly ask for an example project, even if it's something that is easily demonstrated, simply by running existing Apple software, and creating a project, would be a huge pain.
They also ignore reports. Very rarely, I may get a ping on one of my reports, asking me to verify that it was fixed in some release. Otherwise, there's no sign that they ever even read it.
I usually end up closing my bug reports and feature requests, after a few months, because I'm tired of looking at them.
It's clear that they consider every bug report to be a burden. That's a very strange stance, but then, they are not a typical company.
I guess you can't argue with the results, as they have a market value North of 3 trillion dollars, but that does not make it any less annoying.
latexr · 28m ago
> I have had extremely bad luck, reporting bugs to Apple.
From your description, your experience is quite typical.
lapcat · 38m ago
> I guess you can't argue with the results, as they have a market value North of 3 trillion dollars
This was financed by equally massive technical debt.
deviation · 46m ago
Not to hand wave-- but this feels industry standard IMO. I have a dozen PRs sitting unacknowledged and stale across a handful of FAANG (and other) repos, including Apple's.
I start my first day @ Apple in a few weeks, so I ACK that my opinion might be a little biased here.
dmd · 18m ago
Maybe you can help bump FB13400242, a bug that is _literally_ going to kill people. (The bug is that to make an emergency call, even from lock screen, you're supposed to be able to squeeze buttons on either side of the phone. But it only works with the volume buttons on the left - the Action button didn't get supported, when that button was added. So now the rule for teaching a small child isn't just "squeeze both sides" it's "oh but not that one!")
(Yes, this came close to killing someone close to me. Fortunately someone else happened to come along to help.)
strunz · 54m ago
Have you tried copying the files to the local disk before importing?
mentos · 3m ago
Sounds like an argument for Apple to provide a new high-level media import framework?
JoBrad · 12m ago
I wonder if it’s related to import sources, and maybe the speed of that hardware. They are still successfully importing the photos into the Photos app, just not from the camera.
asolove · 1h ago
I also have an OM System camera (OM-5) and never get corruption this bad but occasionally got one row of green pixels at the bottom of a photo during import to Photos. I thought I was crazy, but this motivates me to change up my routine and check if it was Photos all along.
ChrisRR · 1h ago
I feel like this is a URL that I don't want in my history
sholladay · 31m ago
There is a very popular professional audio website called Gearspace that had a much spicier name for a long, long time.
In the late 90s my then-wife was watching over my shoulder one day and saw the domain “freshmeat.net” pop up as a possible auto-completion in my address bar. She was justifiably suspicious until I showed her it was just a software distribution site.
devnullbrain · 59m ago
Infohazard warning:
C++ reference is one of these.
Y_Y · 51m ago
See also: Experts Exchange, Pen Island
eru · 14m ago
There was also a website for a mole station, but I'm not sure if that one was satire.
layer8 · 56m ago
And she wasn’t suspicious of amazon.com? ;)
coldtea · 59m ago
As if "tender lovemaking" is so shocking?
cluckindan · 15m ago
Obviously there must be bureaucracy and an RFP involved!
copperx · 1h ago
Why? It's brilliant.
brulard · 1h ago
The URL might be mistaken for some different content?
k8sToGo · 1h ago
Well, then don't browse from the church computer
crazygringo · 1h ago
What's brilliant about it? What's the reference, for those of us unfamiliar?
rhgraysonii · 1h ago
It is simply hilarious to make grown adults visit a website called tender love making dot com (a sexual reference) to read a very specific and niche blog about technology.
No comments yet
kotaKat · 1h ago
ZScaler gets pissed off going to frame.work just because of a “malicious TLD”.
I don’t even want to know what ZScaler thinks of “tender love making”.
dcchambers · 1h ago
Site belongs to Aaron Patterson, one off the most prolific Ruby developers in the world.
privatelypublic · 1h ago
Something something Railed.
MonkeyClub · 55m ago
Nah that's just a one off
sitharus · 1h ago
I hadn’t dug that far in to it, thanks for sharing! I assumed my rather old SD card or the adapter I keep stuffed at the bottom of my bag was the issue as I’ve only seen it on a couple of photos.
I’ve used Olympus cameras for over a decade. Well, the same camera to be honest, a PEN E-PM2. This has only appeared in the past couple of years.
I haven’t seen it on photos from my Canon EOS 80D yet, but I guess it’s time to change my workflow. And maybe OS.
giancarlostoro · 13m ago
> Turns out “delete after import” was a huge mistake.
Apple corrupted images on my iPhone where I can’t import them to my PC via photos, but I can backup the whole phone.
They finally recognized there is an issue, but there is no fix, as of a few weeks ago :(
alterom · 28m ago
Yeah, that's one of the billion reasons I'm sticking to Android phones with gasp file managers and shock expandable storage via an SD card slot.
I never need to import anything when I can simply copy the data from the card.
bluSCALE4 · 3m ago
I have Apple Photos but I never thought to use it to automatically import my photos and clean it up. My process is very similar to where you've ended up. Thanks for validating it--I'll never change it.
dzink · 49m ago
Not sure if related but importing images via image capture on mac to the disk of the mac gives you correct time when the photo was taken in the file (kind of important if it’s family photos). But if you import it to a usb drive you get current time as creation time for each file so you’ve lost any timestamp you had on the photos.
mystifyingpoi · 20m ago
> kind of important if it’s family photos
Anything important should be kept inside the file. Filesystem metadata gets lost all the time, isn't consistent between operating systems, zipping up a folder and extracting it will probably mess up timestampts too.
CtrlAlt · 1h ago
I’ve never had this bad of corruption. But not surprised.
Personally, I have seen a row of green pixels on the top or bottom + vertically flipped photos on import.
Good sleuthing!
billyjobob · 1h ago
He says the checksums are different but he doesn’t provide a diff to show how different. It could just be a single flipped bit or something. And that could happen in his own RAM/disk/CPU/router so seems premature to immediately blame Apple.
I ran both files through xxd then diffed them. I've literally changed every piece of hardware (at no small cost). "premature to immediately blame Apple" seems a bit off.
jjcob · 39m ago
I tried running the file segments through a binary diff with Hex Fiend
As far as I can tell:
- 0x7800 bytes were replaced at file offset 0x00aa0000
- 0x2200 bytes were replaced at file offset 0x00aa8000
I can't tell if the replacement data came from a different part of the file, or somewhere totally different. Race condition somewhere sounds plausible.
daemin · 41m ago
This is the kind of stuff that makes me wish my Binary Diff Tool was already completed, but unfortunately I'm still working on it. Can't tell much what's wrong with the differences in the bytes without knowing what the structure behind it is.
daemin · 1h ago
There's a corrupted photo at the top of the article and the non corrupt version at the bottom, is that not enough?
Someone · 1h ago
No, it isn’t. The OP isn’t questioning whether the file changed, but asking what changed to the file, not what changed visibly.
The visible effect shown could be due to a change as small as a single bit flip. It also could be that large parts of the file got overwritten, or that it partially got zeroed. The exact kind of damage can help pinpoint the cause of the problem.
asksomeoneelse · 1h ago
Yeah, I would have been interested in the diff too.
That said, the article does mention replacing basically all the hardware and still encountering the issue. FWIW, my personal experience with Apple software so far is that the usage expected for Average Joe is well tested and polished. But stepping outside of that, it's "Here be dragons" territory very quickly.
jsw97 · 1h ago
He switched out his laptop.
mobilemidget · 1h ago
"I’ve not seen any file corruption when importing to Darktable, so I am convinced this is a problem with the Photos app."
Yes this argument is a bit unconvincing for me. Not saying Apple photos doesn't corrupt his files, but this is not real proper investigating either.
lapcat · 1h ago
See also the Image Capture bug from several years ago where it appends a ton of empty data to imported photos:
(I'm not sure whether this bug has been fixed or not yet, though I think it has been fixed.)
iamshs · 1h ago
Image Capture did me dirty once. Macbook ran out of space while importing photos but it never stopped and kept on deleting photos from my iPhone. Lost 5K photos of a wedding... submitted a bug and hopefully it has been rectified.
tamimio · 2h ago
For transferring files (photos or others) from iOS, I have been using Landrop for a while and never had any issues so far, it’s also way faster than using a cable.
basejumping · 1h ago
For transferring photos from your Iphone to your Mac, you can also use the native Image Capture app.
actionfromafar · 2h ago
I always wonder about the motivation behind these polished, high-quality programs on the App Store which are not open source, and also don't collect (much) data, neither have ads in them.
I used it along with another called Localsend, but the later one gave me a bit of headache and crashed while transferring some large files last time I used it, but still great as an alternative too, and it’s open source as well.
Edit: Actually, you are correct, it seems they did close it! Try localsend instead.
BolexNOLA · 1h ago
Love LocalSend. Can be a bit finicky but for quick transfers between systems I love it. Use it so my work laptop, Linux gaming PC, and iPhone can easily pass staff around.
k8sToGo · 1h ago
You figured out how to teleport people through local send?
BolexNOLA · 46m ago
Coming Eventually:TM:
ratg13 · 1h ago
This article is about importing photos from an SD card to MacOS
smeeger · 1h ago
i am constantly noticing things about my iphone. its poorly designed.
kokey · 48m ago
Fortunately it mentions early on in the article that this is related to an Olympus camera so I'm guessing this has something to do with the OM system's flavor of Olympus's proprietary ORF format.
hk1337 · 1h ago
> Turns out “delete after import” was a huge mistake.
That's a mistake no mater what application you're importing to, else we'll be graced with another blog post, "Darktable app Corrupts Photos".
What's the purpose of RAW+jpg though? Seems rather redundant?
merelysounds · 1h ago
> What's the purpose of RAW+jpg though? Seems rather redundant?
You get to keep out of camera jpg files. Some people might like how their camera processes jpg files and might also want the raw file for a scenario when a more complex editing is needed.
hk1337 · 1h ago
Interesting, based on this and other replies, it sounds like Photos App should have an option to select what to import? i.e. RAW or jpg, but not both.
It sounds like Photos App can have issues trying to import both at the same time?
tenderlove · 41m ago
As I said in my blog post, it imports both and combines them in the UI. Also as I said in my blog post, I switched to shooting only in raw, and it still exhibited file corruption.
tenderlove · 43m ago
> That's a mistake no mater what application you're importing to, else we'll be graced with another blog post, "Darktable app Corrupts Photos"
Thanks dad.
bayindirh · 1h ago
I also use RAW+JPG. Latter part allows quick sharing without long post-processing, esp. for impatient friends.
If I'm going to share the photo to an album or something, I process the RAWs selectively.
basisword · 1h ago
JPG so you can also see the default camera processing which might be work well from time to time. RAW in case it doesn't.
Photos does a lot of extra work on import (merging RAW+JPEG pairs, generating previews, database indexing, optional deletion), so my guess is a concurrency bug where a buffer gets reused or a file handle is closed before the copy finishes.
Rare, nondeterministic corruption fits the profile.
Random is random, and random is clumpy, so maybe swapping parts is irrelevant, but... I wanted more detail on his parts-replacement journey.
He doesn't mention how often the corruption happened whilst he swapped out parts, unfortunately. Presumably it was too rare the entire journey.
They constantly ask for an example project, even if it's something that is easily demonstrated, simply by running existing Apple software, and creating a project, would be a huge pain.
They also ignore reports. Very rarely, I may get a ping on one of my reports, asking me to verify that it was fixed in some release. Otherwise, there's no sign that they ever even read it.
I usually end up closing my bug reports and feature requests, after a few months, because I'm tired of looking at them.
It's clear that they consider every bug report to be a burden. That's a very strange stance, but then, they are not a typical company.
I guess you can't argue with the results, as they have a market value North of 3 trillion dollars, but that does not make it any less annoying.
From your description, your experience is quite typical.
This was financed by equally massive technical debt.
I start my first day @ Apple in a few weeks, so I ACK that my opinion might be a little biased here.
(Yes, this came close to killing someone close to me. Fortunately someone else happened to come along to help.)
https://gearspace.com/
https://www.reddit.com/r/audioengineering/comments/mftc0g/ge...
C++ reference is one of these.
No comments yet
I don’t even want to know what ZScaler thinks of “tender love making”.
I’ve used Olympus cameras for over a decade. Well, the same camera to be honest, a PEN E-PM2. This has only appeared in the past couple of years.
I haven’t seen it on photos from my Canon EOS 80D yet, but I guess it’s time to change my workflow. And maybe OS.
https://www.cgsecurity.org/wiki/photoRec
They finally recognized there is an issue, but there is no fix, as of a few weeks ago :(
I never need to import anything when I can simply copy the data from the card.
Anything important should be kept inside the file. Filesystem metadata gets lost all the time, isn't consistent between operating systems, zipping up a folder and extracting it will probably mess up timestampts too.
Personally, I have seen a row of green pixels on the top or bottom + vertically flipped photos on import.
Good sleuthing!
As far as I can tell:
- 0x7800 bytes were replaced at file offset 0x00aa0000
- 0x2200 bytes were replaced at file offset 0x00aa8000
I can't tell if the replacement data came from a different part of the file, or somewhere totally different. Race condition somewhere sounds plausible.
The visible effect shown could be due to a change as small as a single bit flip. It also could be that large parts of the file got overwritten, or that it partially got zeroed. The exact kind of damage can help pinpoint the cause of the problem.
That said, the article does mention replacing basically all the hardware and still encountering the issue. FWIW, my personal experience with Apple software so far is that the usage expected for Average Joe is well tested and polished. But stepping outside of that, it's "Here be dragons" territory very quickly.
Yes this argument is a bit unconvincing for me. Not saying Apple photos doesn't corrupt his files, but this is not real proper investigating either.
https://cdfinder.de/blog/files/image_capture_bug.html
(I'm not sure whether this bug has been fixed or not yet, though I think it has been fixed.)
https://github.com/LANDrop/LANDrop
I used it along with another called Localsend, but the later one gave me a bit of headache and crashed while transferring some large files last time I used it, but still great as an alternative too, and it’s open source as well.
Edit: Actually, you are correct, it seems they did close it! Try localsend instead.
That's a mistake no mater what application you're importing to, else we'll be graced with another blog post, "Darktable app Corrupts Photos".
What's the purpose of RAW+jpg though? Seems rather redundant?
You get to keep out of camera jpg files. Some people might like how their camera processes jpg files and might also want the raw file for a scenario when a more complex editing is needed.
It sounds like Photos App can have issues trying to import both at the same time?
Thanks dad.
If I'm going to share the photo to an album or something, I process the RAWs selectively.