I want to add that "Pump Six and Other Stories" by Paolo Bacigalupi is also a great read. "Pump Six" also paints a similar picture to Forster's "The Machine Stops", but with a different perspective.
Both are great and horrifying at the same time.
kreelman · 16h ago
I'm reminded of "The Feeling of Power" by Isaac Asimov.
In this story, everyone relies on automation so much that they forget how to do mathematics. A lone, nerdy operator re-learns the skill of 2 digit multiplication in the process of working through automated weapons targeting issues, astounding his superiors.
The concept of forgotten arts seems like a common thread here.
Or in the opposite direction, “Who can replace a man?”
strict9 · 18h ago
>No one remembers how to fix it.
This thought exercise also gets interesting with software development. There are few prospects available for new grads and junior devs now. Relentless offshoring, headcount reductions, and AI promises from CEOs have hollowed out the tech landscape. There are very few opportunities for young people to professionally develop the knowledge to keep the systems running.
And now most startups are focused on eliminating staff via AI. The people who would keep the systems running. I'm not sure where all of this leads to in a few years.
kmoser · 17h ago
"No one remembers how to fix it" sounds like the COBOL situation today. The reality is that the bar to learning COBOL is quite low: compilers and documentation are readily available for most platforms.
What's lacking is incentive and drive: there are just too many other shiny tools to distract us, including jobs that sound more glamorous and/or pay just as well.
A young person with the drive to learn just about anything has the means to do so, if they really want.
strken · 11h ago
One thing I've learnt over the years is that most so-called labour shortages aren't significant. Until the companies involved have started paying a lot more or providing cushy incentives, "shortages" are mostly just complaints about needing to spend 4 weeks hiring instead of 2.
hnthrow90348765 · 18h ago
Just go back to books or other sources of knowledge and restart from scratch. It shouldn't take long to get your thinking back.
W.R.T. any large industry crash, no one's going to care if you can do your Angular (or specific tech skill) stuff in 1 days vs. 5 days, so why the emphasis on speed in this scenario? Both System 1 or 2 thinking is fine here.
We've enjoyed some pretty great technical advances for the past 40 years, even with a 10 year "dark age" we're still net ahead. We can rebuild and relearn a lot of stuff in 10 years.
gnatolf · 19h ago
A lot of the comments on HN lately are rightfully focused on this formative brain exercise that leads to intuition and conceptual understanding that is chiselled away by the shortcuts that GenAI provides. I wonder where the gain of productivity from GenAI and the drop off in 'our brain'-quality intersects.
AJ007 · 17h ago
It's actually not that different than talking with employees, however, the LLMs still have very significant shortfalls (which you know about after using them a lot.)
If a manager doesn't know anything about what their employees are working on, they are basically fucked. That much holds up with LLMs. The simple stuff mostly works, but the complex stuff isn't going to pan out for you, and it will take a while to figure out that's the direction you went in.
turtleyacht · 18h ago
One comparison is with Stack Overflow (SO). Given a task, there are usually multiple answers. The question may not even be relevant; often, multiple question pages must be compared.
The best answer is the one that fits the aesthetics of my approach--one that didn't exist before (there was only the problem before), but the answer is simple, straightforward, or adaptable.
Having multiple answers is good because different minds evaluated the question. It is a buffet of alternatives, starting from others' first principles, mistakes, and experience. Some are rejected outright from some tacit taste organ. Others become long-lived browser tabs, a promise to read carefully someday (never).
All this is void if it turns out using SO is similarly degenerative in the same way, though.
emorning3 · 18h ago
We should probably require AI to always be able to explain it's conclusions.
That way we can quickly assimilate knowledge from the AI and theoretically always have at least as much knowledge as the AI.
I suppose it also means that we can verify that the AI is not lying to us.
ptx · 17h ago
Unfortunately we don't have that kind of AI. We only have the useless kind.
lurk2 · 18h ago
> In E.M. Forster’s short story The Machine Stops, he paints a future where a vast machine handles every aspect of human life. People live isolated lives, fully dependent on the machine. They don’t know how it works. They only know how to ask it for things. When the machine breaks down, society collapses. No one remembers how to fix it.
I used to think about this in math class. I could figure out what to do with my calculator most of the time, but I didn’t have any intuitive sense of how things worked. The sine, cosine, and tangent functions are still just black boxes to me, I have no idea what they actually do or how I would calculate their values. I often daydreamed about finding myself on a desert island, needing to make use of trigonometry to rebuild civilization, but not being able to find the angles that I needed.
Lots of other skills are lost this way. I don’t know how to join wood or sew a stitch, but I do know how to operate a nail gun and work a sewing machine. I couldn’t fix either device, but if I couldn’t find anyone to fix them and couldn’t obtain new ones, I would likely have bigger problems to worry about. Most people will only ever need to view these devices as black boxes; the benefits of specialization generally offset the costs introduced by abstraction, absent major market disruptions (e.g. supply chain breakdowns, changing regulatory frameworks, etc.). Most people in human history have spent their lives as generalists on a farm. This hedges the individual against a lot of risk (the generalist can likely always find some work to do), but the real strides in risk management are made by specialists living in urban settlements.
gnfargbl · 17h ago
>The sine, cosine, and tangent functions are still just black boxes to me, I have no idea what they actually do
They are functions which tell you how to relate different parameters of a triangle. Concepts in mathematics are often interconnected, and trig functions appear in a lot of other (interesting) contexts as well, but fundamentally I don't see them as anything "more" than the SOHCAHTOA you were taught at school.
> or how I would calculate their values.
Without a lookup table? You would need some kind of a way to express the functions in terms of other mathematical functions which you know how to do, like multiplications and additions. Sometimes you can do this with a series expansion. Computers sometimes use a variant of the CORDIC algorithm. Both of those things are clever ideas in their own right, but you don't need to understand them to know what trig functions do.
> I often daydreamed about finding myself on a desert island, needing to make use of trigonometry to rebuild civilization, but not being able to find the angles that I needed.
If you had a circle, a ruler, a pen and some paper then you'd get the idea to make yourself a lookup table pretty quickly!
I think the point I'm making is that we get used to assuming that knowledge is so deep and complex a thing that we can never really know anything. But often, knowledge isn't as intricate as that. If everyone forgot about trig functions today, they'd be rediscovered tomorrow.
XorNot · 18h ago
The thing is YAGNI applies here. "What if society collapses?" is really just a LARP justification for doing whatever you already wanted to do.
It's why "preppers" buy an armory of weapons, but don't make friends with their neighbors, become contributors to their local town or advocate for infrastructure improvements or sustainable farming policies.
tmnvix · 18h ago
> It's why "preppers" buy an armory of weapons, but don't make friends with their neighbors
Interestingly, when I've seen interviews with deeply committed preppers, they almost always seem to come around to the conclusion that community is the most significant factor in their planning.
I find it fascinating how that juxtaposes with the (possibly well earned) cliche of preppers as intensely individualist libertarians who reject society. I suspect there's some relationship here to the idea that if you go far enough to the left or right, you find that the spectrum is circular and not linear.
metalrain · 18h ago
If there would be massive EMP that fried all chips around me, I would be for short while somewhat useless.
Most of what I've done is in software, I could not build computer from electronic parts, not even full adder from memory. Maybe I could read some schematics, but most measurement devices use chips as well, it would very difficult.
I think preparation should be skills, useful in any environment.
kmoser · 17h ago
If all chips were fried, you'd have more pressing things to worry about than rebuilding a crude computer. I think the planning, documenting, and problem solving skills you learned as a software developer would still help you with whatever you were called upon to do.
Zigurd · 18h ago
Having AI write the kind of software humans write is not even scratching the surface of what will probably happen. Just as generative CAD tools produce mechanical designs that would be prohibitively complex and time consuming for humans to design and verify, we're eventually going to see, and by eventually, I mean pretty soon, software no human could have written.
holoduke · 18h ago
Its already the case for a long time with assembly generated code. No human could recreate it. Ai code is just another abstraction on top of a layer that soon won't be touched anymore by common devs.
cadamsdotcom · 15h ago
> When the machine breaks down, society collapses. No one remembers how to fix it.
The example is unrealistic because that's a very easily anticipated single point of failure.
We've been building systems without single points of failure for millennia.
There will never be only one machine. Already in AI we have dozens of models. They come from multiple diverse - unaffiliated! - cultures, countries, and ideologies.
huphreem · 17h ago
It’s a fine enough journal entry, and I agree with the underlying sentiment of the piece, but can I crowdsource an explanation of the conclusive pith?
> Learning to learn is a noble idea. But more important is learning to unlearn, and knowing when to resist the comfort of automation.
I feel strongly about “teaching people to teach themselves” which seems a direct analog to “learning to learn”, but I am at a complete loss for what “learning to unlearn” means, especially as it relates to resisting automation.
Is the idea that you need to “unlearn” wanting to “learn” automation so you can keep “learning” more deeply about things you have already “learned”?
jjmarr · 18h ago
If we had to colonize another planet I think we'd need 10,000 of the absolute best in just semiconductor fabrication/design if you wanted to create computers.
craftkiller · 17h ago
That may be if we wanted to create computers similar to what we use today, but wiring diagrams of an extremely basic CPU was part of everyone's CS undergrad at my school (and I assume most other schools too?) so if we just needed to make a turing machine with discrete components I'm sure a couple of us could dig up those memories and figure it out again in a pinch.
jjmarr · 16h ago
> wiring diagrams of an extremely basic CPU was part of everyone's CS undergrad at my school (and I assume most other schools too?)
I went to Toronto Metropolitan University (ranked 800-1000 globally) and actually designing that basic CPU was too hard for most students, so they just gave us VHDL code to copy & paste into Quartus II.
Their VHDL code had bugs in it which I discovered after doing the project myself from scratch and comparing my answer with the "correct" one.
This has been the policy for many years now. Shockingly our co-op placement rates have dropped from 70% to 30%.
cheschire · 18h ago
If farmers had blogs during industrialization, I suspect there would’ve been a lot of this.
Most people don’t know how to grow a potato. So?
korse · 18h ago
Separation from supply leads people to take supply for granted. In general, taking things for granted is risky both mentally and physically. This applies to more than potatoes.
jonnypotty · 18h ago
The problem comes when no one knows how to grow a potato any more, not when most people don't.
theamk · 17h ago
I don't think "no one" will even happen - nor with potato, nor with mechanical skills. There are books which tell how to do all sort of things, starting from the refining metal from ores, and there are people reading them. I have no doubt that minute details will be forgotten, but they were discovered once and could be discovered again if needed.
Yes, horses were replaced, but go outside of city and there are many horse farms, and some of them offer horse riding classes to general public. Should something happen to the cars, the knowledge will come back. Same way, even if an average JS programmer has no idea what "differential pair" stands for, there are plenty of people who _know_ what this is, and can at least read the eye diagram.
cheschire · 18h ago
And yet people still do. So the problem being discussed is demonstrably unlikely compared to the more likely benefit of significant scaling potential for humanity
NathanKP · 18h ago
Back to AI: Would you agree that the problem comes when no knows how to think any more, not when most people don't?
Personally, I'm pretty concerned about both. The fact that many people don't have basic survival skills like sourcing their own food, safe drinking water, and heat. And the fact that that many people lack basic thinking skills: ability to detect misinformation, or deal with the challenges and inaccuracies of flaky AI.
In an ideal world society everyone who is capable has a higher level of training in both. But under modern oligarchic capitalism there are advantages to ensuring that people have neither skill: survival or thinking.
ChrisMarshallNY · 18h ago
> it’s easy to get by without deeply understanding the code you’re deploying.
I feel that's been an issue for a long time, with the heavy reliance on dependencies. AI tools are really just a furtherance of the model already in use.
I have the luxury of developing software as a craft; not as a vocation. I deliberately do stuff "the hard way," because I feel more fulfilled, doing so.
readthenotes1 · 18h ago
Maybe in our AI future we will stop referencing pop science (i.e., not replicated, but salable)
emorning3 · 19h ago
I'm not convinced that I should worry about people that would let AI run their lives.
They're gonna pay the price for their foolishness, and paying the price is the universe's mechanism for keeping itself in balance.
Take what's going on the US for example...
n_ary · 13h ago
Those people will range from your president to regulators to your boss and local utilities. Unlike other novelties(iPhones and internet), AI mandates comes from top-down and is not optional or luxury anymore. We are beholden to the few capital lords if you come to think about it.
mikewarot · 19h ago
Those people include all of your friends family and community. No man is an island, regardless of what Silicon Valley might have you believe.
The price will be born by all of us.
pimlottc · 18h ago
> The price will be born by all of us.
You probably meant borne, but I do agree we're all somewhat responsible too...
emorning3 · 17h ago
>>The price will be born by all of us.<<
No doubt that is true, but...
Suppose an extortionist comes to my business and says "hey, that's a nice business you got there, it would be a shame if someone was to ruin it with a bunch of horseshit tariffs"
I'd say 'yes sir no problem" and then do my goddamn best to murder those fucks in their sleep.
Making them pay the price for extorting me will make sure that its not gonna happen again.
So now suppose the business next door is being extorted.
And I tell the owner that I'd be happy to murder the extortionists for them.
But they say "nah dog, they're sending the bad guys to El Salvador, it'll be fine".
How do expect me to care about that guy?
Fuck that guy.
NoMoreNicksLeft · 18h ago
Some of us are raising families and finding friends who won't be part of this. We're trying to carve out little islands. I can't save you, and I don't even want to.
TimTheTinker · 18h ago
> I can't save you, and I don't even want to.
"Love your neighbor as yourself" exists in some form in every major pre-modern religion and philosophy because it's a cornerstone of society.
Not over-compassionate, truthless love or yet another government program, but caring about those we run into enough to--at minimum--personally help meet their basic needs and defend or rescue them from immediate threats to their safety.
If we're going to let anything go, it shouldn't be that.
rightbyte · 18h ago
If you create a community with like minded individuals it will be very prone to infighting ...
jt2190 · 18h ago
I assume this is your last post on HN, since AI will surely be participating in the discussions here in the near future.
Seriously, though… How will you “carve out a little island”? Computers are the kingdom of AI.
aaronax · 18h ago
Very similar to Amish, Mennonites, Hutterites, etc. I imagine. Draw the line of what you think makes life good and follow as close as is reasonable, and if others agree they will associate.
Krei-se · 18h ago
our daily AI bad fear mongering story give us today
somesortofthing · 18h ago
On the other hand, maybe one day "thinking without AI" will become as absurd a notion as "thinking without a brain".
Both are great and horrifying at the same time.
In this story, everyone relies on automation so much that they forget how to do mathematics. A lone, nerdy operator re-learns the skill of 2 digit multiplication in the process of working through automated weapons targeting issues, astounding his superiors.
The concept of forgotten arts seems like a common thread here.
This is a PDF of the text. https://ia600806.us.archive.org/20/items/TheFeelingOfPower/T...
This is the Wikipedia overview. https://en.wikipedia.org/wiki/The_Feeling_of_Power
First heard this one at school.
This thought exercise also gets interesting with software development. There are few prospects available for new grads and junior devs now. Relentless offshoring, headcount reductions, and AI promises from CEOs have hollowed out the tech landscape. There are very few opportunities for young people to professionally develop the knowledge to keep the systems running.
And now most startups are focused on eliminating staff via AI. The people who would keep the systems running. I'm not sure where all of this leads to in a few years.
What's lacking is incentive and drive: there are just too many other shiny tools to distract us, including jobs that sound more glamorous and/or pay just as well.
A young person with the drive to learn just about anything has the means to do so, if they really want.
W.R.T. any large industry crash, no one's going to care if you can do your Angular (or specific tech skill) stuff in 1 days vs. 5 days, so why the emphasis on speed in this scenario? Both System 1 or 2 thinking is fine here.
We've enjoyed some pretty great technical advances for the past 40 years, even with a 10 year "dark age" we're still net ahead. We can rebuild and relearn a lot of stuff in 10 years.
If a manager doesn't know anything about what their employees are working on, they are basically fucked. That much holds up with LLMs. The simple stuff mostly works, but the complex stuff isn't going to pan out for you, and it will take a while to figure out that's the direction you went in.
The best answer is the one that fits the aesthetics of my approach--one that didn't exist before (there was only the problem before), but the answer is simple, straightforward, or adaptable.
Having multiple answers is good because different minds evaluated the question. It is a buffet of alternatives, starting from others' first principles, mistakes, and experience. Some are rejected outright from some tacit taste organ. Others become long-lived browser tabs, a promise to read carefully someday (never).
All this is void if it turns out using SO is similarly degenerative in the same way, though.
That way we can quickly assimilate knowledge from the AI and theoretically always have at least as much knowledge as the AI.
I suppose it also means that we can verify that the AI is not lying to us.
I used to think about this in math class. I could figure out what to do with my calculator most of the time, but I didn’t have any intuitive sense of how things worked. The sine, cosine, and tangent functions are still just black boxes to me, I have no idea what they actually do or how I would calculate their values. I often daydreamed about finding myself on a desert island, needing to make use of trigonometry to rebuild civilization, but not being able to find the angles that I needed.
Lots of other skills are lost this way. I don’t know how to join wood or sew a stitch, but I do know how to operate a nail gun and work a sewing machine. I couldn’t fix either device, but if I couldn’t find anyone to fix them and couldn’t obtain new ones, I would likely have bigger problems to worry about. Most people will only ever need to view these devices as black boxes; the benefits of specialization generally offset the costs introduced by abstraction, absent major market disruptions (e.g. supply chain breakdowns, changing regulatory frameworks, etc.). Most people in human history have spent their lives as generalists on a farm. This hedges the individual against a lot of risk (the generalist can likely always find some work to do), but the real strides in risk management are made by specialists living in urban settlements.
They are functions which tell you how to relate different parameters of a triangle. Concepts in mathematics are often interconnected, and trig functions appear in a lot of other (interesting) contexts as well, but fundamentally I don't see them as anything "more" than the SOHCAHTOA you were taught at school.
> or how I would calculate their values.
Without a lookup table? You would need some kind of a way to express the functions in terms of other mathematical functions which you know how to do, like multiplications and additions. Sometimes you can do this with a series expansion. Computers sometimes use a variant of the CORDIC algorithm. Both of those things are clever ideas in their own right, but you don't need to understand them to know what trig functions do.
> I often daydreamed about finding myself on a desert island, needing to make use of trigonometry to rebuild civilization, but not being able to find the angles that I needed.
If you had a circle, a ruler, a pen and some paper then you'd get the idea to make yourself a lookup table pretty quickly!
I think the point I'm making is that we get used to assuming that knowledge is so deep and complex a thing that we can never really know anything. But often, knowledge isn't as intricate as that. If everyone forgot about trig functions today, they'd be rediscovered tomorrow.
It's why "preppers" buy an armory of weapons, but don't make friends with their neighbors, become contributors to their local town or advocate for infrastructure improvements or sustainable farming policies.
Interestingly, when I've seen interviews with deeply committed preppers, they almost always seem to come around to the conclusion that community is the most significant factor in their planning.
I find it fascinating how that juxtaposes with the (possibly well earned) cliche of preppers as intensely individualist libertarians who reject society. I suspect there's some relationship here to the idea that if you go far enough to the left or right, you find that the spectrum is circular and not linear.
Most of what I've done is in software, I could not build computer from electronic parts, not even full adder from memory. Maybe I could read some schematics, but most measurement devices use chips as well, it would very difficult.
I think preparation should be skills, useful in any environment.
The example is unrealistic because that's a very easily anticipated single point of failure.
We've been building systems without single points of failure for millennia.
There will never be only one machine. Already in AI we have dozens of models. They come from multiple diverse - unaffiliated! - cultures, countries, and ideologies.
> Learning to learn is a noble idea. But more important is learning to unlearn, and knowing when to resist the comfort of automation.
I feel strongly about “teaching people to teach themselves” which seems a direct analog to “learning to learn”, but I am at a complete loss for what “learning to unlearn” means, especially as it relates to resisting automation.
Is the idea that you need to “unlearn” wanting to “learn” automation so you can keep “learning” more deeply about things you have already “learned”?
I went to Toronto Metropolitan University (ranked 800-1000 globally) and actually designing that basic CPU was too hard for most students, so they just gave us VHDL code to copy & paste into Quartus II.
Their VHDL code had bugs in it which I discovered after doing the project myself from scratch and comparing my answer with the "correct" one.
This has been the policy for many years now. Shockingly our co-op placement rates have dropped from 70% to 30%.
Most people don’t know how to grow a potato. So?
Yes, horses were replaced, but go outside of city and there are many horse farms, and some of them offer horse riding classes to general public. Should something happen to the cars, the knowledge will come back. Same way, even if an average JS programmer has no idea what "differential pair" stands for, there are plenty of people who _know_ what this is, and can at least read the eye diagram.
Personally, I'm pretty concerned about both. The fact that many people don't have basic survival skills like sourcing their own food, safe drinking water, and heat. And the fact that that many people lack basic thinking skills: ability to detect misinformation, or deal with the challenges and inaccuracies of flaky AI.
In an ideal world society everyone who is capable has a higher level of training in both. But under modern oligarchic capitalism there are advantages to ensuring that people have neither skill: survival or thinking.
I feel that's been an issue for a long time, with the heavy reliance on dependencies. AI tools are really just a furtherance of the model already in use.
I have the luxury of developing software as a craft; not as a vocation. I deliberately do stuff "the hard way," because I feel more fulfilled, doing so.
They're gonna pay the price for their foolishness, and paying the price is the universe's mechanism for keeping itself in balance.
Take what's going on the US for example...
The price will be born by all of us.
You probably meant borne, but I do agree we're all somewhat responsible too...
No doubt that is true, but...
Suppose an extortionist comes to my business and says "hey, that's a nice business you got there, it would be a shame if someone was to ruin it with a bunch of horseshit tariffs"
I'd say 'yes sir no problem" and then do my goddamn best to murder those fucks in their sleep. Making them pay the price for extorting me will make sure that its not gonna happen again.
So now suppose the business next door is being extorted. And I tell the owner that I'd be happy to murder the extortionists for them. But they say "nah dog, they're sending the bad guys to El Salvador, it'll be fine".
How do expect me to care about that guy? Fuck that guy.
"Love your neighbor as yourself" exists in some form in every major pre-modern religion and philosophy because it's a cornerstone of society.
Not over-compassionate, truthless love or yet another government program, but caring about those we run into enough to--at minimum--personally help meet their basic needs and defend or rescue them from immediate threats to their safety.
If we're going to let anything go, it shouldn't be that.
Seriously, though… How will you “carve out a little island”? Computers are the kingdom of AI.