> And what’s worse, malicious threat actors can manipulate time synchronization protocols in many cases to trigger this vulnerability at the time of their choosing.
If you switch to 64 bit timestamps, and the network protocol supports dates > 2038, can you then just trigger the rollover bugs by pretending it's 2*64 - 1 seconds after epoch start?
Also, if the actions are potentially so severe, and NTP (or whatever is used) so vulnerable, why haven't we seen many such attacks in the wild?
Update: to be clear I'm not arguing that there isn't a problem, I've already run into it myself. I'm trying to understand how severe it is, how exploitable, and how robust a solution could be.
jsd1982 · 12h ago
I fixed a recent Y38 bug in some classic ASP code. The bug was nothing more than a simple `Date() + 5000` computation (adding 5000 days to the current date) as a sort of expiry date applied to something; I don't recall the exact details. VB6 did not take kindly to computing any date value beyond the Y38 max and threw an error. In practice this error ended up denying service to everyone even though the Y38 max date was 14 years in the future. You never know what little bugs like that are lurking in such legacy code.
potatoman22 · 10h ago
I feel like the hard part is flashing the new firmware
Waterluvian · 1d ago
I think this effort would benefit from trying to qualify what “unpredictable ways” actually means. If anyone is testing devices, a catalog of test results describing the actual failure modes that were revealed would help make this whole thing more concrete.
I think many software engineers know that if you want to make any organization care about this type of issue, you need to be ready to demonstrate the severity and impact.
Even if the system boots properly, there's various critical systems that depend upon having the correct time. Say goodbye to things like HTTPS and SSL/TLS certificates.
luckylion · 22h ago
Will the root-certificates still be trusted in 12 years? Will we largely use the same TLS versions? And if systems can be updated to account for that, shouldn't they also be able to be updated to deal with the timestamps limitation?
moffkalast · 14h ago
For comparison I've revived a decade old Axis PTZ camera recently and it can't be used with HTTPS because it only supports TLS 1.0 which is deprecated across the board these days lmao. The UI is so bugged out it's not possible to change the default username and password anymore.
There's two kinds of internet connected devices these days, those that keep getting updated and those that drift into incompatibility and die as the rest of the ecosystem evolves around them. If these supposed critical devices will still be in use in 12 years without any maintenance then they're unlikely to have any actual importance.
It means 'anywhere between being bricked and no problem at all, and we can't give you any idea of how severe or how likely any of those possiblities are'. The only way you can really know is to thorougly audit your system and/or test it. Preferably both.
AndrewKemendo · 23h ago
So that means the best tool is a 2038 test environment - which people then install their application(s) and test it e2e to see what the impacts are.
However, I’m not sure how you make a 2038 test environment
It assumes that the OS/Kernel etc… are defacto frozen to 2025 or whatever increment until 2038
What was the y2k solution for the people that implemented those fixes in the 90s?
jcul · 6h ago
libfaketime is useful for testing these kinds of things.
It intercepts system calls to get the time and reports a fake time to the application.
Y2K showed that you don't need details beyond vague threats of "medication administered at the wrong time" and "planes falling out of the air" to get organizations and the public to care. No idea how that's going to tie into the conspiracy-heavy media landscape we inhabit now.
(Note I do think this is a serious issue that needs to be addressed. And I'd love to see specific examples. I'm just pushing back against the idea that examples would make much difference to advocacy efforts)
BeFlatXIII · 23h ago
“Planes falling pit of the sky” still gets used both as an example of overblown Y2k fear-mongering AND the reason why all those quiet preparations were necessary.
msla · 11h ago
Is there any plausible mechanism for Y2K bugs to cause planes to fall from the sky?
LargoLasskhyfv · 6h ago
Why not? From five years ago, on the most modern type:
Seems to be unrelated at first look, but considering the complexities of systems, and cascading failure modes?
bongodongobob · 23h ago
What? Y2K did have many demonstrable problems... Having a 2 digit year did obviously cause problems. The reason nothing happened is because a shit ton of time and money was spent making sure it didn't.
jbeninger · 23h ago
Agreed. My point is that the orgs paying for all these updates were mostly motivated by the vague claims of experts rather than concrete examples
jmbwell · 22h ago
It was pretty easy for orgs with affected systems to produce concrete examples for themselves. Maybe to everyone else it seemed vague, but for the people who had to deal with it, it was taken pretty seriously from top to bottom.
It was thankless work that is still glossed over and waved away today, but it was all a very big deal throughout the late 90s.
mulmen · 17h ago
> It was thankless work that is still glossed over and waved away today, but it was all a very big deal throughout the late 90s.
Mike Judge even made a movie about it! Office Space might be the most recognition turn of the millennium programmers will receive.
Waterluvian · 23h ago
All I can say is that at my level in an org, if I want to say “instead of developing X this quarter, I want to test the effects of 2038 on Y,” that’s a far easier conversation if I can say something like “in similar embedded devices they crashed and wouldn’t even respond to OTA updates” vs “something bad could happen. Not sure.”
The latter is just a ripe plumb, left to rot in the backlog.
bongodongobob · 23h ago
That's nonsense. Orgs spent time and resources on it because they grabbed a test server and demonstrated it caused problems. It's not some weird ethereal untestable bug. They set the dev server to a minute before midnight and went "oh shit".
pixl97 · 21h ago
That and quite often the problems started showing up years before 2000 itself. "Hey, the scheduler is giving me a meeting 80 years ago" type weirdness when it crossed the boundry.
AndrewDucker · 1d ago
Telling home users to check that their existing smart devices will still work in 13 years seems like overkill. It seems unlikely that more than a tiny fraction of them will still be in use then, if any.
Businesses installing new smart infrastructure and devices will need to pay attention to this, and in 10-15 years they'll need to work out what to replace, of course.
0xDEAFBEAD · 15h ago
Agreed. A serious approach to this problem would be: Identify critical computers which are currently 13+ years old (most likely embedded systems). Assume that the same sorts of systems will be 13+ years old in 2038. Focus on raising awareness with that particular target audience, e.g. give talks about the 2038 problem at embedded systems conferences. Try to get it included in university curricula. Etc.
Goes back at least slightly before that, as I've had 2038epochalypse.com registered since March 2017, but I can't recall whether I thought I was being clever or whether I heard it somewhere else.
LilBytes · 4m ago
[delayed]
web007 · 20h ago
I've had my "2038 consulting" sites since Feb 2011, but someone got epochalypse dot com registered August 2007.
ks2048 · 1d ago
Who will register a domain for the upcoming year 292271025015 problem?
thrance · 23h ago
Y292B271M25K15 is less catchy than Y2K
mkj · 21h ago
Are they doing anything to fix it or just raising awareness?
Here's an example of measuring packages that report warnings for software that has suspicious conversions. Compile with `-Wconversion` with both 32-bit and 64-bit time_t, and see what the difference is.
https://github.com/mkj/yocto-y2038
That is using yocto, but you could probably do something similar with other less-embedded distros too, if you can rebuild the world.
FWIW I didn't find much interesting with that apart from busybox dhcpd.
It looks mostly like a project for self promotion of the two authors. Maybe they offer some consulting services.
Funniest is that one of them wrote that they have "learned about it after Y2K bug".
I thought one learns about this overflow in a "introduction to programming" class...
Sardtok · 18h ago
It also says nothing about a formal education, just that he has worked in IT since his teens. I didn't hear of the 2038 problem myself until the whole Y2K debacle, but then, I was in my teens at the time.
panzi · 1d ago
When will milliseconds since the Unix epoch not be precisely representable in IEEE double anymore? (As it's used in JavaScript.)
Arcorann · 5h ago
MAX_SAFE_INTEGER = 2^53 - 1 = 9007199254740991. The next integer 2^53 is representable, while the following one 2^53 + 1 isn't.
2^53 + 1 milliseconds = 9007199254740993 milliseconds = 104249991 days 8 hours 59 minutes 0.993 seconds. Since the zero point is midnight 1970-01-01, we reach the first non-representable millisecond at 287396-10-12T8:59:00.993 (be careful to use the Gregorian leap rule if you want to calculate this manually).
OutOfHere · 18h ago
Fwiw, it's much worse for 64-bit nanoseconds, running out in the year 2262 for signed, and the year 2554 for unsigned. As for who needs nanoseconds, trading systems, particle physics, and globally distributed systems do.
panzi · 15h ago
Which systems use a single variable for nanoseconds time? The APIs I know of all use a struct with two members, one for seconds and the other for the nanoseconds within the second.
kbolino · 14h ago
It's a pretty common format for timestamps on modern file systems used by Unixy operating systems (e.g. ext4, XFS, APFS, ZFS, btrfs, etc.).
cesarb · 36m ago
At least ext4 (and its predecessors ext2/ext3) uses 32-bit seconds for the timestamps, and has separate fields (available only when the inode size of the filesystem is greater than 128 bytes) for nanoseconds and for extending the seconds fields to 34 bits.
fmbb · 1d ago
In the year 287390 (285420.9 years after 1970) according to my calculations using DuckDuckGo’s calculator.
If you ask ChatGPT it’s:
= 285,616 years after 1970
= Year 285,616
lionkor · 23h ago
Why would you ask ChatGPT?
hackyhacky · 22h ago
AI is the next big thing, I hear.
Thorham · 22h ago
That seems a bit small seeing how doubles can represent 53 bit integers accurately.
I witnessed my first y38 bug a couple weeks ago - just as a matter of someone doing forecasting in a system with an incorrect casting.
At least, the good part is that people will get "hunches" about y38, just like you start getting "hunches" about bugs related to locales, time zones, character encodings, currency roundings, etc...
I don't know if there are courses, books, etc... about all those matters that are definitely non "computer sciency", but occupy so much of our engineering time ?
rkapsoro · 21h ago
I've been planning for decades to have a party on that day.
Only 13 years left!
alkonaut · 2h ago
Wait, in my invitation it says the party already happened in 1901.
sans_souse · 2h ago
Is it too early to raise awareness for the Y10k bug?
cranberryturkey · 1d ago
15 years ago i was working at a startup in SV and a kid we hired was saying how he was sad he missed out on y2k because he was too young. I filled him with joy when I mentioned the 2038 bug. lol
Retr0id · 19h ago
I worry that for a lot of devices (IoT, etc.), the fix will be "throw it away and buy a new one"
DaSHacka · 11h ago
Well, thats already the "fix" for the current problems those devices face.
ninjin · 23h ago
Patched in OpenBSD 5.5 [1] which was released in 2014. Patches were upstreamed into software in the ports tree, so hopefully that made life a little bit easier for other operating systems. Linux seems to have followed around 2020.
Epochalypse is clever. I'd been calling it Y2K38 so it made sense to people outside the industry.
net01 · 1d ago
The fact that it's not common knowledge and not taught in schools and universities is concerning.
Henchman21 · 21h ago
Is it? I was starting my career during Y2K. It was widely regarded as a bust, and I’d anticipate people expecting the same damned thing?
sroussey · 20h ago
It was largely regarded as a success. Why do you say a bust?
Henchman21 · 9h ago
Sorry, I meant from the perspective of people expecting chaos to happen. I agree it was wildly successful in that basically nothing happened!
parpfish · 19h ago
I remember some people looking forward to the y2k chaos. There was a smug sense of “look at how badly those brainiac computer guys screwed up”
I remember a rumor that “if enough people take their phone off the hook before midnight it’ll take down the phone network”
phtrivier · 18h ago
I don't know about "largely", but I can point to a least one instance where the most week know comedy show at the time in France ("Les guignols") had a bit about how the Y2K bug was a "scam" orchestrated by "Big Tech" to make people buy new computers - given that nothing terrible happened on Y2K.
The sketch was a riff about the kindergarten joke :
- why do you [insert weird action] ?
- to scare the girafes away
- but there are no girafes here
- of course, I've been [insert weird action here]
And I was a software engineering student at the time, and I tried explaining they were missing the point, but I think it will be how we're approached this time.
So, I think nothing will be done, or not enough, and bugs will happen in 12 years.
But, thankfully, all our computers will have been replaced to allow running js.
msla · 11h ago
"What am I paying my IT department for? My computers never have problems! It's a scam, I tells ya!"
igleria · 1d ago
my capitalist side remembers that many people made retirement money because of Y2K.
Maybe I should start thinking about an early retirement :)
Nevermark · 1d ago
Well, by 2038 the chatbots will have long since fixed all the time format problems, all our problems, and will be working away at problems of which we cannot concieve.
So, ahem, your “early retirement” is assured?
A year is a lot longer than it used to be. And quickly getting longer. Measured in change.
mmwako · 1d ago
How exactly? asking for a friend
luckylion · 22h ago
"You still know php4? We need someone to fix this 50000 loc file that was written in 2005. Willing to pay big money."
0xDEAFBEAD · 15h ago
Wouldn't it just be a matter of recompiling as 64-bit? No need to mess with the 50000 LOC.
jonstewart · 22h ago
I would much prefer we put this off till the last minute so that I have one more job available to me before retirement. Thanks.
blantonl · 20h ago
Eh, I've got 13 years to deal with this. No big deal.
The best thing about waiting until the last minute to fix something is it will only take a minute.
bobbyraduloff · 1d ago
Born too early for Y2K, but just in time for the Epochalypse lol
Nevermark · 1d ago
Well we don’t know if it will actually be an e/apocalypse. Which sounds very scary and doomsy to me.
But it will be the end of time, as we know it!
yapyap · 23h ago
yeah sure.
lkbm · 20h ago
My my
stuaxo · 1d ago
(Not) Looking forward to 2038 onwards as people ignore the fact we coordinated to fix this and start saying it was all a big hoax.
christophilus · 1d ago
Same thing happened with Y2K. You can’t let such people bother you, or you’ll be miserable.
phtrivier · 18h ago
Except "those people" are the ones who will run the USA for the foreseeable future.
Fixing Y38 will require some public spending. If "researching cancer" is not considered worthy of public spending any more, I'm curious about how the nerds will manage to justify replacing lots of embedded chips "because maths".
If it takes the usual 8 years to replace the current administration with one that accepts listening to experts, and unless big donors can make a profit by organizing the transition, we can expect serious efforts to only start around 2032 globally. No idea if that will be enough this time, we'll see...
(On the bright side, maybe they can charge customers ? That would work.)
BeFlatXIII · 23h ago
Will people use planes falling out of the sky as the reason the fix must be priority 1 this time?
lambdaone · 21h ago
Even though it might seem unlikely, you absolutely can't eliminate the possibility of planes falling out of the air unless you have thoroughly audited and tested all their systems in advance. Not having planes in the air at the exact moment of rollover - just in case - might be a good idea too. Similarly financial markets and other key infrastructure.
13 years is plenty of time to make a start on all of this, and will allow the costs to be spread over that time period without undue hurry. It's also long enough that it will be possible for much software to be allowed to reach end-of-life without being audited, while applying rigorous testing to new software. Procrastination, though, is not a good option.
eastbound · 22h ago
Planes will still be using floppy disks. Or they will have upgraded to the worst brand of Android, as usual.
lambdaone · 21h ago
Why should floppy disks be a problem? Even if they are fantastically out of date, if they can be maintained as a niche legacy technology, they're not a problem. They won't suddenly stop working, it will just slowly get more and more expensive to get the parts. Supply chains are magic - at the right price, someone will always be willing to make more floppies or drives. Of course that price might be $500 per floppy and $50,000 per drive...
drob518 · 23h ago
It’s Y2K all over again. Signed? RUFKM?
Tistron · 22h ago
In 1980, it was probably more important to be able to represent 1960 than 2040, so to me it makes sense they were signed.
If you switch to 64 bit timestamps, and the network protocol supports dates > 2038, can you then just trigger the rollover bugs by pretending it's 2*64 - 1 seconds after epoch start?
Also, if the actions are potentially so severe, and NTP (or whatever is used) so vulnerable, why haven't we seen many such attacks in the wild?
Update: to be clear I'm not arguing that there isn't a problem, I've already run into it myself. I'm trying to understand how severe it is, how exploitable, and how robust a solution could be.
I think many software engineers know that if you want to make any organization care about this type of issue, you need to be ready to demonstrate the severity and impact.
Even if the system boots properly, there's various critical systems that depend upon having the correct time. Say goodbye to things like HTTPS and SSL/TLS certificates.
There's two kinds of internet connected devices these days, those that keep getting updated and those that drift into incompatibility and die as the rest of the ecosystem evolves around them. If these supposed critical devices will still be in use in 12 years without any maintenance then they're unlikely to have any actual importance.
However, I’m not sure how you make a 2038 test environment
It assumes that the OS/Kernel etc… are defacto frozen to 2025 or whatever increment until 2038
What was the y2k solution for the people that implemented those fixes in the 90s?
It intercepts system calls to get the time and reports a fake time to the application.
https://github.com/wolfcw/libfaketime
Y2K showed that you don't need details beyond vague threats of "medication administered at the wrong time" and "planes falling out of the air" to get organizations and the public to care. No idea how that's going to tie into the conspiracy-heavy media landscape we inhabit now.
(Note I do think this is a serious issue that needs to be addressed. And I'd love to see specific examples. I'm just pushing back against the idea that examples would make much difference to advocacy efforts)
https://www.theregister.com/2020/04/02/boeing_787_power_cycl...
Seems to be unrelated at first look, but considering the complexities of systems, and cascading failure modes?
It was thankless work that is still glossed over and waved away today, but it was all a very big deal throughout the late 90s.
Mike Judge even made a movie about it! Office Space might be the most recognition turn of the millennium programmers will receive.
The latter is just a ripe plumb, left to rot in the backlog.
Businesses installing new smart infrastructure and devices will need to pay attention to this, and in 10-15 years they'll need to work out what to replace, of course.
Here's an example of measuring packages that report warnings for software that has suspicious conversions. Compile with `-Wconversion` with both 32-bit and 64-bit time_t, and see what the difference is. https://github.com/mkj/yocto-y2038
That is using yocto, but you could probably do something similar with other less-embedded distros too, if you can rebuild the world.
FWIW I didn't find much interesting with that apart from busybox dhcpd.
Funniest is that one of them wrote that they have "learned about it after Y2K bug". I thought one learns about this overflow in a "introduction to programming" class...
2^53 + 1 milliseconds = 9007199254740993 milliseconds = 104249991 days 8 hours 59 minutes 0.993 seconds. Since the zero point is midnight 1970-01-01, we reach the first non-representable millisecond at 287396-10-12T8:59:00.993 (be careful to use the Gregorian leap rule if you want to calculate this manually).
If you ask ChatGPT it’s:
At least, the good part is that people will get "hunches" about y38, just like you start getting "hunches" about bugs related to locales, time zones, character encodings, currency roundings, etc...
I don't know if there are courses, books, etc... about all those matters that are definitely non "computer sciency", but occupy so much of our engineering time ?
Only 13 years left!
[1]: https://www.openbsd.org/lyrics.html#55
I remember a rumor that “if enough people take their phone off the hook before midnight it’ll take down the phone network”
The sketch was a riff about the kindergarten joke :
- why do you [insert weird action] ?
- to scare the girafes away
- but there are no girafes here
- of course, I've been [insert weird action here]
And I was a software engineering student at the time, and I tried explaining they were missing the point, but I think it will be how we're approached this time.
So, I think nothing will be done, or not enough, and bugs will happen in 12 years.
But, thankfully, all our computers will have been replaced to allow running js.
Maybe I should start thinking about an early retirement :)
So, ahem, your “early retirement” is assured?
A year is a lot longer than it used to be. And quickly getting longer. Measured in change.
The best thing about waiting until the last minute to fix something is it will only take a minute.
But it will be the end of time, as we know it!
Fixing Y38 will require some public spending. If "researching cancer" is not considered worthy of public spending any more, I'm curious about how the nerds will manage to justify replacing lots of embedded chips "because maths".
If it takes the usual 8 years to replace the current administration with one that accepts listening to experts, and unless big donors can make a profit by organizing the transition, we can expect serious efforts to only start around 2032 globally. No idea if that will be enough this time, we'll see...
(On the bright side, maybe they can charge customers ? That would work.)
13 years is plenty of time to make a start on all of this, and will allow the costs to be spread over that time period without undue hurry. It's also long enough that it will be possible for much software to be allowed to reach end-of-life without being audited, while applying rigorous testing to new software. Procrastination, though, is not a good option.