This article is just gatekeeping the word "engineer" for no reason. When I was developing an agent recently I picked a model (thus fixing most of the "uncertainties" mentioned) and started to construct the system prompt, from simpler to more complex, sometimes back to simpler, discovering that to this particular model the name choice means a lot, observing how often I was or wasn't getting desired results, etc etc. I think it would be challenging to find a definition of the word "engineer" that wouldn't include this process, even if it, obviously, wasn't as complex as some other engineering projects.
NotAnOtter · 5h ago
Words have meaning. I'm not a prescriptivist but when you have an army of formerly PM's, Designers, etc all at once exclaiming "I'm an engineer too!" that should signal that maybe the thing they're doing isn't really engineering.
Do you count making a square space splash page 'engineering'? Tools improving to the point that the barrier of entry plummets is great. That doesn't mean you're now engaging in the same fundamental task that happened before things got easier/
nerdsniper · 3h ago
Words have meaning, but we can still argue over the meaning of words. For example, I believe that engineering necessarily involves calculus, and if you’re not doing calculus ever in your job, you’re more of a “technician” or “specialist” than an engineer. I have a degree in chemical engineering, and have been titled “chemical engineer”, “controls engineer”, “electrical engineer”, and “field engineer”. Currently my title is “software engineer”.
But I believe the last time I did any engineering was when my title was “Intern” - none of my jobs since then have required actual rigorous engineering and could have been done equally well or better by someone without an engineering degree.
I currently believe “software developer” would be a more appropriate title for me.
alganet · 2h ago
I held "software engineer" and even "researcher" titles (btw I don't even have a degree). These mean nothing.
I am a troubleshooter, and my troubleshooting skills are measured by how much trouble I can shoot.
Not how much trouble I will be able to shoot once my tool is sharpened, not how much trouble I could shoot _in principle_, nothing of this nonsense. Trouble I can shoot now.
Users who mostly use LLM prompting are currently very limited in the amount of trouble they can shoot. Sometimes, it creates more problems than it solves.
Once that changes, we can open the gates. Show me the works.
echohack5 · 5h ago
If the thing standing in your way to engineering a great product was syntax, and a bunch of gate-kept and inconsistent linux cli tooling, then its fine to call it engineering.
I dunno, do you read bytecode?
GeneralMayhem · 5h ago
I don't think that's the point. If non-technical people are able to make a product happen by asking a machine to do it for them, that's fine. But they're not engineering. It simply means that engineering is no longer required to make such a product. Engineering is the act of solving problems. If there are no problems to solve, then maybe you've brought about the product, but you haven't "engineered" it.
I don't think that memorizing arcane Linux CLI invocations is "engineering" either, to be clear.
thunky · 4h ago
If I were to "build" the next big app entirely using llms, never writing a line of code, did I create it and do I own it?
If you answered yes that's really all that matters imo. Label me what you want.
toofy · 3h ago
If you hired people to build that product, you never wrote a line of code. No, you didn’t build it. Your team did. You’re not magically a software engineer, you hired someone else to do it.
Is there a product? Yep. Do you own it? Maybe. But again, you’re not suddenly the engineer. A project manager? Maybe.
thunky · 2h ago
> No, you didn’t build it
That's why I used the word create. I would be responsible for the creation of the product, so imo I created it. I'm the creator. It wouldn't exist without my vision, direction, and investment (of time and/or money).
Like a movie Producer: they don't actually "build" the movie. They use their money pay people to manifest a movie, and at the end of it they have created a movie and get a share of the profits (or losses) that come with it.
No, they shouldn't call themselves cinematographers, but they can say that they "produced" the movie and nobody takes issue with that.
> Do you own it? Maybe.
If I paid for it then absolutely I own it. I get to keep the future profits because I took the risk. The people that "built" it get nothing more than what I paid them for their labor (unless I offerred them ownership shares).
toofy · 2h ago
i think people are trying to make this difficult when it’s honestly super simple.
yes, you can make a product. no, it does not suddenly magically make you a musician.
you did the equivalent of hiring someone else to do it. you did not do it.
if you claim you wrote the novel, you’re lying. someone else did. if someone takes credit for work someone else did, they’re lying. it’s honestly not complicated. at all.
thunky · 2h ago
you're not countering what i'm saying, so i think we agree.
i'm just adding that (as an "engineer") i don't care what you call me, or what i call myself, because nobody cares and it doesn't matter. i'm commodified labor. replacable. with no claim on anything. and nobody will ever agree on the correct title anyway.
what actually matters imo is who the owner is.
toofy · 58m ago
yeah, it sounds like we both agree with the original post.
it literally doesn’t make someone an engineer.
its not difficult to understand but for some reason when its said it pisses certain people off.
i suspect many of the people upset want to convince themselves they’re suddenly magically a musician, architect, engineer, novelist, programmer, etc… when it just couldn’t be further from the truth. they’re just doing the equivalent of sending a dm to a coder friend and the friend is the actual programmer.
i think some people don’t appreciate being told the truth.
NotAnOtter · 5h ago
If the thing gating your from creating a great product was the engineering, then I would call you a great product designer and not an engineer. If you suddenly can create products with a new tool - you didn't change, the tools did.
You're still a great product designer and not an engineer.
coolKid721 · 5h ago
Terms have meaning, and words can refer to different terms.
People quibbling about what's an engineer or not is about as helpful as the stupid hotdog sandwich thing, it's arbitrary an engineer is not a determinate thing.
AFAIK the reason why the word engineer has some specific clout people get touchy about is because in normal engineering fields becoming a licensed engineer is kind of a big deal for them so they get really particular about it. I only ever refer to myself as an engineer to bother people who get cunty about it. Get over yourself, why do you care if a designer calls themself an engineer? Are you worried it'll make it hard to find other true engineers so you will have a harder time finding civil engineers to talk about how calling apis is basically the same as building bridges?
OJFord · 5h ago
In many countries it has legal meaning, and you can't just say you're an engineer, if you're not.
(I'm not from one of them, fwiw. I had AEG send out an 'engineer' to replace a piece of plastic on a dishwasher; I've been emailed by 'customer support engineers'.)
I don't think it's a 'get over yourself' thing though, SWE is fairly unique in industry in not making a distinction between engineers and technicians. I actually think the rise of LLMs might take us there, not necessarily the terminology, already abused as it is, but the distinction in roles between what were architects and senior+ engineers, and overseeing machinery.
coolKid721 · 5h ago
People using words in a way you don't like isn't abuse
kec · 1h ago
Well, according to ABET Engineering is: “The profession in which a knowledge of the mathematical and natural sciences gained by study, experience, and practice is applied with judgment to develop ways to utilize, economically, the materials and forces of nature for the benefit of mankind”.
By that definition, developing and training an LLM model could be argued to be a form of engineering… but exercising that model definitely isn’t. An analogy to mechanical engineering would be the difference between designing a CNC machine vs programming the machine to carve out a widget.
StrLght · 6h ago
You've described what sounds a lot like experimentation: observing how a black box behaves on different inputs. Conducting experiments is closely related to engineering — I don't think anyone would claim different. But engineering is not limited to experimentation alone.
handfuloflight · 5h ago
What level of threshold of illustration do you need to reach for it to be properly said that you are drawing?
What's the analogous threshold to say you're engineering?
StrLght · 5h ago
Art is subjective. It isn't about checking boxes and counting how many formal criteria it meets.
Engineering is broad, it uses many different methods and techniques. All I am saying is that I wouldn't call someone a mathematician if they can only find the result of 2+2.
handfuloflight · 2h ago
You say art is subjective but didn't answer the question: what are the objective criteria that define engineering?
tiahura · 5h ago
gatekeeping the word "engineer" for no reason
There was value in the word Engineer connoting professionalism and accountability. Checkers and draftsmen were checkers and draftsmen, not engineers.
Then, anyone who knew the difference between a patch cable and a crossover cable became a network engineer.
ineedasername · 5h ago
And anyone running a few ETL flows is a Data Engineer.
The gatekeeper holding closed the barn door on "engineer" got trampled long ago. Near as I can tell, it happened in the 90's, around the time lots of comp sci grads started hitting the profession, at a time when lots of programmers already in the field had come over from electrical engineering.
voidhorse · 6h ago
All sorts of human activities involve processes like this. Art, for instance.
Engineering is about precision it's not about this fuzzy sort of "iterate til it works" approach. Sure, iterations is involved, but knowing the precise conditions and bounds under which a system functions in a specific way is what engineering is actually about. I would not want a person building a bridge to use your process of iteration and guess work.
Yes, under this idea a lot of software engineering isn't actual engineering.
TJSomething · 5h ago
Under this idea, most engineering isn't engineering. Some of the oldest engineering is ad hoc battlefield engineering to construct siege engines and building tunnels to deliver explosives. That frequently only needed to work one time.
With bridges, you only need that high confidence because there are high costs and risks. Also, the stakeholders are usually governments, who require very predictable results. All that effort is worth it because the artifact will be useful for a long time for a lot of people.
It might be okay if some widget only lasts for 6 months. So, you empirically shave off material until it's as cheap as possible while failing at an acceptable rate.
The cost of shipping is low for software, so the risk profile is even more different. This can be shifted for high stakes software and I think there are some social issues there, but many things are shaped more like Facebook than aircraft control systems. They can fall over and no one's going to actually die.
I think the core of engineering is in evaluating these tradeoffs and figuring out where you can expend effort most efficiently.
exe34 · 6h ago
Honestly if we call software "engineering" in the first place, without any kind of certification/liability, I don't see the problem with applying it to everything else.
umanwizard · 6h ago
Indeed. Engineer is a silly word to gatekeep. Its original literal meaning is just someone who builds engines (in the original, broader sense of “engines” which basically means “machines”). The appropriation of the term to mean formalized categories with certifications etc. came hundreds of years later.
cwillu · 5h ago
A word being fuzzy does not mean it has no meaning and that it can therefore be used for anything. A word being used in a particular (but fuzzy) way hundreds of years ago does not mean that it can't have a different (and fuzzy) meaning today. Yes, words drift, but the vague fuzzy notion that engineering meant 20 years ago still has value, and if the word has lost that meaning with no replacement, that's a problem.
LtWorf · 5h ago
I doubt that's the origin, since "ingegno" means cleverness in italian…
danans · 5h ago
I'm pretty sure the Italian word "ingegno" (like English "ingenious") comes from the same Latin root as "engine": ingenium, meaning "mind or intellect", with "engine" changing its pronunciation and meaning via medieval French.
umanwizard · 2h ago
Well instead of doubting, you can just look it up on wiktionary.
dlevine · 6h ago
My theory is that, when done properly, it’s much closer to science than engineering.
And by “done properly,” i mean done in a regimented way with evals to verify that a wide range of inputs produce the desired outputs.
Prompting is much closer to discovering the properties of an already existing system than building something using engineering methods.
NotAnOtter · 6h ago
My definition of science is something like "a methodical approach to reach some truth" whereas engineering is something like "a methodical approach to reach some functionality". And I think some version of that is pretty universally accepted.
So by that definition, prompt engineering is much closer to engineering than science. That said, I would consider it closer to product development than either of the above two; I don't count 'tell an llm you'll torture it until your website is hopefully less buggy' a methodical approach.
ankit219 · 6h ago
"Done properly" anything can be science/engineering. Just make a process, give it a KPI on how well it's performing, and you have an engineering. Movie making done properly is where audience is engaged with every act, you test screen it, see audience reaction, tweak, have a specific set of evals, and then you have something scoring high on the metrics. Part of that happens today, but you would not call that science. It's not what people mean when they call something as science or engineering. Your's is too broad of a definition to mean anything specific.
gundmc · 6h ago
I have no problem taking issue with using the term "engineering" for this, but this reads like OP does not believe that you can get better results based on the way you structure your prompt and what information you include. That is wild to me and should be obvious to anyone who spends more than a few hours using LLMs.
For what it's worth, I think about the "engineering" in "Prompt/Context Engineering" almost more akin to how it's used in "Social Engineering". You are influencing the model to produce a desirable result.
labrador · 6h ago
Prompting is more art than engineering. But understanding the basics of LLMs, which are engineered systems, helps you get better results. I think of it like being a good interviewer: you’re not controlling the conversation, but you know how to guide it. A good prompter, like a good interviewer, knows how to ask the kind of question that gets a meaningful answer. I think of LLMs like a giant library and my task is to steer it to the right sections for a good answer.
andy99 · 6h ago
No,the prompt is a hyperparameter. If you're not treating it as such and systematically experimenting, it's just superstition.
labrador · 6h ago
You're view of it is correct for agents but for my purposes, which are mostly brainstorming, researching and asa thinking tool, I have GPT 4o memory systems turned on. This allows it to be more personable and allows crafting of answers more attuned to my meaning. System and chat history memory obviously introduce global state which is not exactly a tunable hyperparameter for more consistent results. You should probably turn it off for agents so your view is accurate.
Bluestein · 6h ago
> hyperparameter
That's an interesting way to put it. indeed.-
tibastral2 · 2h ago
This guy should read Bruno Latour, it would make him understand that engineering and more generally science is also a belief system, based on some way of validating the knowledge that can be (or not) what occidental people call logical. But it's another ethnocentrism.
See symetry proof of azandé for more insights
ineedasername · 5h ago
Prompting "ChatGPT" is very different than prompting LLMs. For the latter, whether you want to use the E word or not, every one of the unknowns the author cites either don't exist or are greatly reduced when you're using LLMs to construct something purpose-built.
Anyone working in tech should take a few hours, download LangFlow and choose a small model in ollama and just run one of the template flows and peek at their prompt setups. Maybe the author just chose a poor title, there's so much writing like this that is the equivalent of saying "Drills can't make good holes" when what you mean is the $15 harbor freight special hooked up to a janky power source and unsteady hand.
rapatel0 · 5h ago
There are plenty of engineering disciplines that need to have statistical approaches to solving problems. Prompt engineering, should be approached by creating benchmark datasets and defining measures of efficacy and reliability.
More generally, IMHO engineering is use of physical and informational tools to build a solution. Tools may have unpredictability and reliability concerns. Its the job of an engineer to utilize the power of these tools while overcoming the reliability issues that might be present.
Example: Semiconductor manufacturing involves shooting gases at piece of silicon. There are all sorts of random scattering, distributional anomalies, and patterning problems that arise largely at random. I think you would still call it engineering.
jonplackett · 6h ago
I don’t mind people criticising AI.
It’s bad in loads of ways.
But it also has some good bits.
To call it all snake oil is just as much bullshit as to claim it will solve all humanity’s problems.
NotAnOtter · 5h ago
To claim AI is useless is ignorant. To claim AI is as good as 99% of people claim it to be, is also ignorant.
A lot of people have drank the 'exponential growth' kool aid but don't have enough understanding of the underlying tech to realize (1) there could be some fundamental limits to how good LLM based AI can be. Or AI in general for that matter. (2) a big part of why it got so good so fast is because we started pumping trillions of dollars of microchips & electricity into it. The hardware and energy consumption cannot continue exponentially.
toofy · 3h ago
If a founder hires an engineer who develops a product, the founder is not magically an engineer. He hired the work out.
This isn’t complicated but people struggle to understand it for some weird reason.
If I have someone photograph my wedding, and then sit at my computer, once the pictures arrive to me, distribute the photographs, I’m not suddenly magically a photographer.
I could go on and on. Using an LLM to write a biology paper does not make that person a biologist.
Typing out a request to my friend to write me a poem about chickens with red mittens doesn’t magically somehow make me a poet.
It wild that so many struggle with this basic ass concept.
mystraline · 6h ago
Doesn't matter that much.
Management thinks they can do our jobs by replacing us with a shell scrip... LLM prompt.
We're always moving up and down the tech ladder, solving problems we thought were solved, and, we fix them. But some prompt is supposedly going to replace understanding of your infra?! Hardly.
But you'll save money for this quarter. Past that is definitely up for discussion.... With your shareholders.
acedTrex · 5h ago
This seems very obvious to literally any one that is half competent at actual engineering.
simonw · 5h ago
> "Prompt engineers" will tell you that some specific ways of prompting some specific models will result in a "better result"... without any criteria for what a "better result" might signify.
That's what evals are for. The best developers working on LLM applications are the ones who are addressing the problem described in this quote. Here's a recent thread about that: https://news.ycombinator.com/item?id=44430117
kriro · 5h ago
"""To put it succinctly, prompt engineering is nothing but an attempt to reverse-engineer a non-deterministic black box"""
So it's not engineering because it is reverse-engineering?
Next up in this series "reverse-engineering is not engineering".
I never understood why people care so much about labels. Call it prompt alchemy if you want. The question to me is if there's repeatable improvements. The last ICSE I attended had an entire session related to prompt engineering so the academic world of SE seems to think it qualifies.
VirgilShelton · 6h ago
It's just a buzz word but sadly it will (And already has) effected real engineers :(
LtWorf · 5h ago
I don't think they were good engineers to begin with anyway.
msgodel · 3h ago
Coding in general isn't engineering. Software engineering is a relatively boring administrative exercise and unless you just want to maximize your salary you should probably do everything you can to avoid a "software engineering" role.
ThrowOregonAway · 6h ago
Technically you're not an engineer unless you're licensed and bonded and belong to a professional engineering society, or you drive a train, or have invented a unique engine. Period, end of discussion.
I know, I know. California changed their law so coders could puff themselves up and put on airs and make their resumes look more impressive. But this is a sad anomaly.
crustycoder · 6h ago
I'm licensed by The Engineering Council in the UK for doing computery stuff, and by the relevant European body. I've been writing software for a living for most of my 40+ year career, and still do. So Chartered Software Engineers do exist.
But perhaps not in CA.
ThrowOregonAway · 5h ago
We have no such thing in the US. There is no license for software "engineering" here, it's a made up term.
gnabgib · 5h ago
There is in Canada, licensed and bonded.
rapatel0 · 5h ago
As someone with a PhD in electrical engineering, I care exactly zero percent about a paid for stamp from an org calling an me engineer or not.
ThrowOregonAway · 5h ago
This is why the planes are falling out of the sky now, this attitude of yours. As an eminent master and founder of the entire field of electrical engineering, I am much more qualified than you to have an opinion on this matter as well.
LtWorf · 5h ago
Well my country didn't change the law, so I still think it's very weird how every american is an engineer.
baal80spam · 6h ago
In my org they told us clearly: either we will learn prompt engineering, or they will replace us with people who already know it.
No comments yet
sincerely · 6h ago
I think as people get increasingly aware of the nebulous consistency of hosted LLMs, local models will become more valuable
lloydjones · 6h ago
This article seems like LinkedIn style controversy-bait, or is written from a place of extreme pedantry.
Evals can come pretty close to getting “deterministic” output from LLMs, and I’d argue that this is reasonably considered engineering.
bn-l · 6h ago
You won’t get far without engineering if all you can do is prompt.
crowcroft · 6h ago
This sounds like the same kind of issue ‘real’ engineers have with software ‘engineering’.
ChrisArchitect · 3h ago
Post from February.
Related, more recent post (today) from the author:
Everything around LLMs is still magical and wishful thinking
And taking photographs is not painting, but it doesn't mean it's not a form of visual art that requires skill to do well
socalgal2 · 5h ago
I just tried Gemini Canvas which I've seen lot of cool results but I failed and I found it infuriating, similar to the worst version of trying to navigate a phone maze to talk to a real person.
I didn't expect it to get there but I did hope one a scale of 1 to 10 (10 = Fantavision) it would get to 6 ish. I'd say it got to 3 of 10.
What I got was some cubes with yellow planes on the side for buildings. This means things are massively inefficient since instead of a simple textured cube for each building, each building is 1000s of polygons, 2 for each window. Lots of Z fighting. I asked for a river with reflections. The reflections were practically invisible. I think it was generating reflection fireworks under the water plane, not actually "reflecting" the ones above. Because they were under the plane they were faded out because of blending which is not how reflections of fireworks work with water. Then, the fireworks themsevles were horrible, made with 1 pixel particles, no glow, no trails, no substance. They were further very un-random in their velocities and looked nothing like real fireworks.
I tried for 90 minutes to get it to make something better but mostly failed. I did manage to get it to stop using pixels for particles and use textured quads.
It's first version started with "click to launch fireworks". That didn't work at all, blank screen. I told it I didn't want to launch firework, I just wanted them to auto-launch. It made a new one, this time it had a caption "with auto launching fireworks" (and still blank). I told it there was no point in such a caption as it would be obvious from seeing the fireworks that they were auto-launching. It got rid of that caption but still a blank screen. I started over, this time adding "auto-launching, auto camera" type elements to my intial prompt.
I tried several times to get it to make an interesting city around both sides of a river (because this was supposed to be an NYC fireworks display). It failed in all kinds of ways. I'd asked it to fly the camera around the fireworks and never have it go behind any buildings. I put the camera in the buildings so you couldn't see any fireworks. I told it "not behind the builings". It moved the camera further way, twice. I finally got it to put the camera only above the river between the two shorelines (if you can call a plane with some cubes on both sides a river with shorelines). The other way it failed was it just got too slow. The way it was inefficiently making buildings mean it started running at 3fps. I started over.
At least twice it started with code that had an error. I'd click "Fix Error" and it would fail to fix it. I'd click "Fix Error", repeat 5-6 times. Eventually give up and start over with a slightly different prompt.
I tried asking Gemini to write the prompt as someone suggestion. First use gemini to write the prompt. Then paste that into to Gemini Canvas mode. It didn't help nor generate anything noticable different than my hand written prompt.
The issue I spent the most time on, it would "launch a firework" (which it referred to as a rocket). The "rocket" (a particle? a sphere? no idea) would go straight up verically (as in y += velocity * deltaTime). As it did it would leave a trail of static exhaust particles just hanging in the air motionless. The firework would then explode and all of the static hanging in the air exhaust particles from the rocket trail would unfreeze fall a little and fade out.
My goal was to get those particles to fall and fade as they were emitted, not wait for the rocket to hit it's peak. I tried 15 or so prompts but it utterly failed to understand the problem. It kept tweaking the velocity of the rocket trail particles after the firework explodes. It never got that them hanging static in the air was the issue no matter how many times or ways I explained the issue. Ever time it would just adjust the velocity of the trail particles, after the firefox exploded and make up nonsense about how it "solved the issue now".
If I'd gotten that working I'd have asked it to stop launching the rockets straight vertically but give them a parabolic simple physics based arc.
I could have done the entire thing myself in 15-30 minutes and got exactly what I wanted. Instead I faught with non-understanding LLM right out of the a Sci-Fi comedy movie.
I know this will work some day and I've seen examples of it working. Still ... ugh for now.
Do you count making a square space splash page 'engineering'? Tools improving to the point that the barrier of entry plummets is great. That doesn't mean you're now engaging in the same fundamental task that happened before things got easier/
But I believe the last time I did any engineering was when my title was “Intern” - none of my jobs since then have required actual rigorous engineering and could have been done equally well or better by someone without an engineering degree.
I currently believe “software developer” would be a more appropriate title for me.
I am a troubleshooter, and my troubleshooting skills are measured by how much trouble I can shoot.
Not how much trouble I will be able to shoot once my tool is sharpened, not how much trouble I could shoot _in principle_, nothing of this nonsense. Trouble I can shoot now.
Users who mostly use LLM prompting are currently very limited in the amount of trouble they can shoot. Sometimes, it creates more problems than it solves.
Once that changes, we can open the gates. Show me the works.
I dunno, do you read bytecode?
I don't think that memorizing arcane Linux CLI invocations is "engineering" either, to be clear.
If you answered yes that's really all that matters imo. Label me what you want.
Is there a product? Yep. Do you own it? Maybe. But again, you’re not suddenly the engineer. A project manager? Maybe.
That's why I used the word create. I would be responsible for the creation of the product, so imo I created it. I'm the creator. It wouldn't exist without my vision, direction, and investment (of time and/or money).
Like a movie Producer: they don't actually "build" the movie. They use their money pay people to manifest a movie, and at the end of it they have created a movie and get a share of the profits (or losses) that come with it.
No, they shouldn't call themselves cinematographers, but they can say that they "produced" the movie and nobody takes issue with that.
> Do you own it? Maybe.
If I paid for it then absolutely I own it. I get to keep the future profits because I took the risk. The people that "built" it get nothing more than what I paid them for their labor (unless I offerred them ownership shares).
yes, you can make a product. no, it does not suddenly magically make you a musician.
you did the equivalent of hiring someone else to do it. you did not do it.
if you claim you wrote the novel, you’re lying. someone else did. if someone takes credit for work someone else did, they’re lying. it’s honestly not complicated. at all.
i'm just adding that (as an "engineer") i don't care what you call me, or what i call myself, because nobody cares and it doesn't matter. i'm commodified labor. replacable. with no claim on anything. and nobody will ever agree on the correct title anyway.
what actually matters imo is who the owner is.
it literally doesn’t make someone an engineer.
its not difficult to understand but for some reason when its said it pisses certain people off.
i suspect many of the people upset want to convince themselves they’re suddenly magically a musician, architect, engineer, novelist, programmer, etc… when it just couldn’t be further from the truth. they’re just doing the equivalent of sending a dm to a coder friend and the friend is the actual programmer.
i think some people don’t appreciate being told the truth.
You're still a great product designer and not an engineer.
AFAIK the reason why the word engineer has some specific clout people get touchy about is because in normal engineering fields becoming a licensed engineer is kind of a big deal for them so they get really particular about it. I only ever refer to myself as an engineer to bother people who get cunty about it. Get over yourself, why do you care if a designer calls themself an engineer? Are you worried it'll make it hard to find other true engineers so you will have a harder time finding civil engineers to talk about how calling apis is basically the same as building bridges?
(I'm not from one of them, fwiw. I had AEG send out an 'engineer' to replace a piece of plastic on a dishwasher; I've been emailed by 'customer support engineers'.)
I don't think it's a 'get over yourself' thing though, SWE is fairly unique in industry in not making a distinction between engineers and technicians. I actually think the rise of LLMs might take us there, not necessarily the terminology, already abused as it is, but the distinction in roles between what were architects and senior+ engineers, and overseeing machinery.
By that definition, developing and training an LLM model could be argued to be a form of engineering… but exercising that model definitely isn’t. An analogy to mechanical engineering would be the difference between designing a CNC machine vs programming the machine to carve out a widget.
What's the analogous threshold to say you're engineering?
Engineering is broad, it uses many different methods and techniques. All I am saying is that I wouldn't call someone a mathematician if they can only find the result of 2+2.
There was value in the word Engineer connoting professionalism and accountability. Checkers and draftsmen were checkers and draftsmen, not engineers.
Then, anyone who knew the difference between a patch cable and a crossover cable became a network engineer.
The gatekeeper holding closed the barn door on "engineer" got trampled long ago. Near as I can tell, it happened in the 90's, around the time lots of comp sci grads started hitting the profession, at a time when lots of programmers already in the field had come over from electrical engineering.
Engineering is about precision it's not about this fuzzy sort of "iterate til it works" approach. Sure, iterations is involved, but knowing the precise conditions and bounds under which a system functions in a specific way is what engineering is actually about. I would not want a person building a bridge to use your process of iteration and guess work.
Yes, under this idea a lot of software engineering isn't actual engineering.
With bridges, you only need that high confidence because there are high costs and risks. Also, the stakeholders are usually governments, who require very predictable results. All that effort is worth it because the artifact will be useful for a long time for a lot of people.
It might be okay if some widget only lasts for 6 months. So, you empirically shave off material until it's as cheap as possible while failing at an acceptable rate.
The cost of shipping is low for software, so the risk profile is even more different. This can be shifted for high stakes software and I think there are some social issues there, but many things are shaped more like Facebook than aircraft control systems. They can fall over and no one's going to actually die.
I think the core of engineering is in evaluating these tradeoffs and figuring out where you can expend effort most efficiently.
And by “done properly,” i mean done in a regimented way with evals to verify that a wide range of inputs produce the desired outputs.
Prompting is much closer to discovering the properties of an already existing system than building something using engineering methods.
So by that definition, prompt engineering is much closer to engineering than science. That said, I would consider it closer to product development than either of the above two; I don't count 'tell an llm you'll torture it until your website is hopefully less buggy' a methodical approach.
For what it's worth, I think about the "engineering" in "Prompt/Context Engineering" almost more akin to how it's used in "Social Engineering". You are influencing the model to produce a desirable result.
That's an interesting way to put it. indeed.-
Anyone working in tech should take a few hours, download LangFlow and choose a small model in ollama and just run one of the template flows and peek at their prompt setups. Maybe the author just chose a poor title, there's so much writing like this that is the equivalent of saying "Drills can't make good holes" when what you mean is the $15 harbor freight special hooked up to a janky power source and unsteady hand.
More generally, IMHO engineering is use of physical and informational tools to build a solution. Tools may have unpredictability and reliability concerns. Its the job of an engineer to utilize the power of these tools while overcoming the reliability issues that might be present.
Example: Semiconductor manufacturing involves shooting gases at piece of silicon. There are all sorts of random scattering, distributional anomalies, and patterning problems that arise largely at random. I think you would still call it engineering.
It’s bad in loads of ways.
But it also has some good bits.
To call it all snake oil is just as much bullshit as to claim it will solve all humanity’s problems.
A lot of people have drank the 'exponential growth' kool aid but don't have enough understanding of the underlying tech to realize (1) there could be some fundamental limits to how good LLM based AI can be. Or AI in general for that matter. (2) a big part of why it got so good so fast is because we started pumping trillions of dollars of microchips & electricity into it. The hardware and energy consumption cannot continue exponentially.
This isn’t complicated but people struggle to understand it for some weird reason.
If I have someone photograph my wedding, and then sit at my computer, once the pictures arrive to me, distribute the photographs, I’m not suddenly magically a photographer.
I could go on and on. Using an LLM to write a biology paper does not make that person a biologist.
Typing out a request to my friend to write me a poem about chickens with red mittens doesn’t magically somehow make me a poet.
It wild that so many struggle with this basic ass concept.
Management thinks they can do our jobs by replacing us with a shell scrip... LLM prompt.
We're always moving up and down the tech ladder, solving problems we thought were solved, and, we fix them. But some prompt is supposedly going to replace understanding of your infra?! Hardly.
But you'll save money for this quarter. Past that is definitely up for discussion.... With your shareholders.
That's what evals are for. The best developers working on LLM applications are the ones who are addressing the problem described in this quote. Here's a recent thread about that: https://news.ycombinator.com/item?id=44430117
So it's not engineering because it is reverse-engineering?
Next up in this series "reverse-engineering is not engineering".
I never understood why people care so much about labels. Call it prompt alchemy if you want. The question to me is if there's repeatable improvements. The last ICSE I attended had an entire session related to prompt engineering so the academic world of SE seems to think it qualifies.
I know, I know. California changed their law so coders could puff themselves up and put on airs and make their resumes look more impressive. But this is a sad anomaly.
But perhaps not in CA.
No comments yet
Evals can come pretty close to getting “deterministic” output from LLMs, and I’d argue that this is reasonably considered engineering.
Related, more recent post (today) from the author:
Everything around LLMs is still magical and wishful thinking
https://news.ycombinator.com/item?id=44467949
First, an example I saw online
https://www.reddit.com/r/WebVR/comments/1lh9d3h/webvr_game_c...
I've seen others. They seemed impressive.
In particular, I thought it might be fun to ask Gemini to make a fireworks display for the 4th of July. What I imagined was Fantavision
https://www.youtube.com/watch?v=pD2AnDhyynM
I didn't expect it to get there but I did hope one a scale of 1 to 10 (10 = Fantavision) it would get to 6 ish. I'd say it got to 3 of 10.
What I got was some cubes with yellow planes on the side for buildings. This means things are massively inefficient since instead of a simple textured cube for each building, each building is 1000s of polygons, 2 for each window. Lots of Z fighting. I asked for a river with reflections. The reflections were practically invisible. I think it was generating reflection fireworks under the water plane, not actually "reflecting" the ones above. Because they were under the plane they were faded out because of blending which is not how reflections of fireworks work with water. Then, the fireworks themsevles were horrible, made with 1 pixel particles, no glow, no trails, no substance. They were further very un-random in their velocities and looked nothing like real fireworks.
I tried for 90 minutes to get it to make something better but mostly failed. I did manage to get it to stop using pixels for particles and use textured quads.
It's first version started with "click to launch fireworks". That didn't work at all, blank screen. I told it I didn't want to launch firework, I just wanted them to auto-launch. It made a new one, this time it had a caption "with auto launching fireworks" (and still blank). I told it there was no point in such a caption as it would be obvious from seeing the fireworks that they were auto-launching. It got rid of that caption but still a blank screen. I started over, this time adding "auto-launching, auto camera" type elements to my intial prompt.
I tried several times to get it to make an interesting city around both sides of a river (because this was supposed to be an NYC fireworks display). It failed in all kinds of ways. I'd asked it to fly the camera around the fireworks and never have it go behind any buildings. I put the camera in the buildings so you couldn't see any fireworks. I told it "not behind the builings". It moved the camera further way, twice. I finally got it to put the camera only above the river between the two shorelines (if you can call a plane with some cubes on both sides a river with shorelines). The other way it failed was it just got too slow. The way it was inefficiently making buildings mean it started running at 3fps. I started over.
At least twice it started with code that had an error. I'd click "Fix Error" and it would fail to fix it. I'd click "Fix Error", repeat 5-6 times. Eventually give up and start over with a slightly different prompt.
I tried asking Gemini to write the prompt as someone suggestion. First use gemini to write the prompt. Then paste that into to Gemini Canvas mode. It didn't help nor generate anything noticable different than my hand written prompt.
The issue I spent the most time on, it would "launch a firework" (which it referred to as a rocket). The "rocket" (a particle? a sphere? no idea) would go straight up verically (as in y += velocity * deltaTime). As it did it would leave a trail of static exhaust particles just hanging in the air motionless. The firework would then explode and all of the static hanging in the air exhaust particles from the rocket trail would unfreeze fall a little and fade out.
My goal was to get those particles to fall and fade as they were emitted, not wait for the rocket to hit it's peak. I tried 15 or so prompts but it utterly failed to understand the problem. It kept tweaking the velocity of the rocket trail particles after the firework explodes. It never got that them hanging static in the air was the issue no matter how many times or ways I explained the issue. Ever time it would just adjust the velocity of the trail particles, after the firefox exploded and make up nonsense about how it "solved the issue now".
If I'd gotten that working I'd have asked it to stop launching the rockets straight vertically but give them a parabolic simple physics based arc.
I could have done the entire thing myself in 15-30 minutes and got exactly what I wanted. Instead I faught with non-understanding LLM right out of the a Sci-Fi comedy movie.
I know this will work some day and I've seen examples of it working. Still ... ugh for now.