Study mode

But now, you're wondering if the answer the AI gave you is correct or something it hallucinated. Every time I find myself putting factual questions to AIs, it doesn't take long for it to give me a wrong answer. And inevitably, when one raises this, one is told that the newest, super-duper, just released model addresses this, for the low-low cost of $EYEWATERINGSUM per month.

But worse than this, if you push back on an AI, it will fold faster than a used tissue in a puddle. It won't defend an answer it gave. This isn't a quality that you want in a teacher.

So, while AIs are useful tools in guiding learning, they're not magical, and a healthy dose of scepticism is essential. Arguably, that applies to traditional learning methods too, but that's another story.

Now in regards to LLMs, I use them almost every day, so does my team, and I also do a bit of postmortem and reflection on what was accomplished with them. So, skeptical in some regards, but certainly not behaving like a Luddite.

The main issue I have with all the proselytization about them, is that I think people compare getting answers from an LLM to getting answers from Google circa 2022-present. Everyone became so used to just asking Google questions, and then Google started getting worse every year; we have pretty solid evidence that Google's results have deteriorated significantly over time. So I think that when people say the LLM is amazing for getting info, they're comparing it to a low baseline. Yeah maybe the LLM's periodically incorrect answers are better than Google - but are you sure they're not better than just RTFM'ing? (Obviously, it all depends on the inquiry.)

The second, related issue I have is that we are starting to see evidence that the LLM inspires more trust than it deserves due to its humanlike interface. I recently started to track how often Github Copilot gives me a bad or wrong answer, and it's at least 50% of the time. It "feels" great though because I can tell it that it's wrong, give it half the answer, and then it often completes the rest and is very polite and nice in the process. So is this really a productivity win or is it just good feels? There was a study posted on HN recently where they found the LLM actually decreases the productivity of an expert developer.

So I mean I'll continue to use this thing but I'll also continue to be a skeptic, and this also feels like kinda where my head was with Meta's social media products 10 years ago, before I eventually realized the best thing for my mental health was to delete all of them. I don't question the potential of the tech, but I do question the direction that Big Tech may take it, because they're literal repeat offenders at this point.

Not underrated at all. Lots of people were happy to abandon Stack Overflow for this exact reason.

> Adding in a mode that doesn't just dump an answer but works to take you through the material step-by-step is magical

I'd be curious to know how much this significantly differs from just a custom academically minded GPT with an appropriately tuned system prompt.

https://chatgpt.com/gpts

This is where the skepticism arises. Before we spend another $100 billion on something that ended up being worthless, we should first prove that it’s actually useful. So far, that hasn’t conclusively been demonstrated.

Except these systems will still confidently lie to you.

The other day I noticed that DuckDuckGo has an Easter egg where it will change its logo based on what you've searched for. If you search for James Bond or Indiana Jones or Darth Vader or Shrek or Jack Sparrow, the logo will change to a version based on that character.

If I ask Copilot if DuckDuckGo changes its logo based on what you've searched for, Copilot tells me that no it doesn't. If I contradict Copilot and say that DuckDuckGo does indeed change its logo, Copilot tells me I'm absolutely right and that if I search for "cat" the DuckDuckGo logo will change to look like a cat. It doesn't.

Copilot clearly doesn't know the answer to this quite straightforward question. Instead of lying to me, it should simply say it doesn't know.

It mostly isn't, the point of the good learning process is to invest time into verifying "once" and then add verified facts to the learning material so that learners can spend that time learning the material instead of verifying everything again.

Learning to verify is also important, but it's a different skill that doesn't need to be practiced literally every time you learn something else.

Otherwise you significantly increase the costs of the learning process.

The good: it can objectively help you to zoom forward in areas where you don’t have a quick way forward.

The bad: it can objectively give you terrible advice.

It depends on how you sum that up on balance.

Example: I wanted a way forward to program a chrome extension which I had zero knowledge of. It helped in an amazing way.

Example: I am keep trying to use it in work situations where I have lots of context already. It performs better than nothing but often worse than nothing.

Mixed bag, that’s all. Nothing to argue about.

It happens with many technological advancements historically. And in this case there are people trying hard to manufacture outrage about LLMs.

Except that the textbook was probably QA’d by a human for accuracy (at least any intro college textbook, more specialized texts may not have).

Matters less when you have background in the subject (which is why it’s often okay to use LLMs as a search replacement) but it’s nice not having a voice in the back of your head saying “yeah, but what if this is all nonsense”.

But this is completely wrong! In the Monty Hall problem, the host has to reveal a door with a goat behind it for you to gain the benefit of switching. I have to point this out for the LLM to get it right. It did not reason about the problem I gave it, it spat out the most likely response given the "shape" of the problem.

This is why shrugging and saying "well humans get things wrong too" is off base. The problem is that the LLM is not thinking, period. So it cannot create a mental model of your understanding of a subject, it is taking your text and generating the next message in a conversation. This means that the more niche the topic (or your particular misunderstanding), the less useful it will get.

Here in my country, English is not you'll hear in everyday conversation. Native English speakers account to a tiny percentage of population. Our language doesn't resemble English at all. However, English is a required subject in our mandatory education system. I believe this situation is quite typical across many Asian countries.

As you might imagine, most English teachers in public schools are not native speakers. And they, just like other language learners, make mistakes that native speakers won't make without even realizing what's wrong. This creates a cycle enforcing non-standard English pragmatics in the classroom.

Teachers are not to blame. Becoming fluent and proficient enough in a second language to handle questions students spontaneously throw to you takes years, if not decades of immersion. It's an unrealistic expectation for an average public school teacher.

The result is rich parents either send their kids to private schools or have extra classes taught by native speakers after school. Poorer but smart kids realize the education system is broken and learn their second language from Youtube.

-

What's my point?

When it comes to math/science, in my experience, the current LLMs act similarly to the teachers in public school mentioned above. And they're worse in history/economics. If you're familiar with the subject already, it's easy to spot LLM's errors and gather the useful bits from their blather. But if you're just a student, it can easily become a case of blind-leading-the-blind.

It doesn't make LLMs completely useless in learning (just like I won't call public school teachers 'completely useless', that's rude!). But I believe in the current form they should only play a rather minor role in the student's learning journey.

I use LLMs but only for things that I have a good understanding of.

Leanring what is like that? MIT open courseware has been available for like 10 years with anything you could want to learn in college

Textbooks are all easily pirated

https://www.reddit.com/r/LibreWolf/s/Wqc8XGKT5h

This simply hasn't been my experience.

Its too shallow. The deeper I go, the less it seems to be useful. This happens quick for me.

Also, god forbid you're researching a complex and possibly controversial subject and you want it to find reputable sources or particularly academic ones.

Learning a new programming language used to be mediated with lots of useful trips to Google to understand how some particular bit worked, but Google stopped being useful for that years ago. Even if the content you're looking for exists, it's buried.

I think the potential in this regard is limitless.

In the process it helped me to learn many details about RA and NDP (Router Advertisments/Neighbor Discovery Protocol, which mostly replace DHCP and ARP from IPv4).

It made me realize that my WiFi mesh routers do quite a lot of things to prevent broadcast loops on the network, and that all my weird issues could be attributed to one cheap mesh repeater. So I replaced it and now everything works like a charm.

I had this setup for 5 years and was never able to figure out what was going on there, although I really tried.

I tried using YouTube to find walk through guides for how to approach the repair as a complete n00b and only found videos for unrelated problems.

But I described my issues and took photos to GPT O3-Pro and it was able to guide me and tell me what to watch out for.

I completed the repair (very proud of myself) and even though it failed a day later (I guess I didn’t re-seat well enough) I still feel far more confident opening it and trying again than I did at the start.

Cost of broken watch + $200 pro mode << Cost of working watch.

I find it odd that someone who has been to college would see this as a _bad_ way to learn something.

This requires a student to be actually interested in what they are learning tho, for others, who blindly trust its output, it can have adverse effects like the illusion of having understood a concept while they might have even mislearned it.

There seems to be a gap in problem solving abilities here...the process of breaking down concepts into easier to understand concepts and then recompiling has been around since forever...it is just easier to find those relationships now. To say it was impossible to learn concepts you are stuck on is a little alarming.

No, not really.

> Unless it was common enough to show up in a well formed question on stack exchange, it was pretty much impossible, and the only thing you can really do is keep paving forward and hope at some point, it'll make sense to you.

Your experience isn't universal. Some students learned how to do research in school.

I think this is the same thing with vibe coding, AI art, etc. - if you want something good, it's not the right tool for the job. If your alternative is "nothing," and "literally anything at all" will do, man, they're game changers.

* Please don't overindex on "shitty" - "If you don't need something verifiably high-quality"

I had to post the source code to win the dispute, so to speak.

I also use it to remember some python stuff. In rust, it is less good: makes mistakes.

In those two domains, at that level, it's really good.

It could help students I think.

[0] https://time.com/7295195/ai-chatgpt-google-learning-school/

As long as you can tell that you don’t deeply understand something that you just read, they are incredible TAs.

The trick is going to be to impart this metacognitive skill on the average student. I am hopeful we will figure it out in the top 50 universities.

When I got stuck on a concept, I wasn't screwed: I read more; books if necessary. StackExchange wasn't my only source.

LLMs are not like TAs, personal or not, in the same way they're not humans. So it then follows we can actually contemplate not using LLMs in formal teaching environments.

It’s called basic research skills - don’t they teach this anymore in high school, let alone college? How ever did we get by with nothing but an encyclopedia or a library catalog?

I haven't tested them on many things. But in the past 3 weeks I tried to vibe code a little bit VHDL. On the one hand it was a fun journey, I could experiment a lot and just iterated fast. But if I was someone who had no idea about hardware design, then this trash would've guided me the wrong way in numerous situations. I can't even count how many times it has built me latches instead of clocked registers (latches bad, if you don't know about it) and that's just one thing. Yes I know there ain't much out there (compared to python and javascript) about HDLs, even less regarding VHDL. But damn, no no no. Not for learning. never. If you know what you're doing and you have some fundamental knowledge about the topic, then it might help to get further, but not for the absolute essentials, that will backfire hard.

And that's a bad thing. Nothing can replace the work in learning, the moments where you don't understand it and have to think until it hurts and until you understand. Anything that bypasses this (including, for uni students, leaning too heavily on generous TAs) results in a kind of learning theatre, where the student thinks they've developed an understanding, but hasn't.

Experienced learners already have the discipline to use LLMs without asking too much of them, the same way they learned not to look up the answer in the back of the textbook until arriving at their own solution.

And which just makes things up (with the same tone and confidence!) at random and unpredictable times.

Yeah apart from that it's just like a knowledgeable TA.

How do you know when it's bullshitting you though?

Given that humanity has been able to go from living in caves to sending spaceships to the moon without LLMs, let me express some doubt about that.

Even without going further, software engineering isn't new and people have been stuck on concepts and have managed to get unstuck without LLMs for decades.

What you gain in instant knowledge with LLMs, you lose in learning how to get unstuck, how to persevere, how to innovate, etc.

We were able to learn before LLMs.

Libraries are not a new thing. FidoNet, USENET, IRC, forums, local study/user groups. You have access to all of Wikipedia. Offline, if you want.

The internet, and esp. stack exchange is a horrible place to learn concepts. For basic operational stuff, sure that works, but one should mostly be picking up concepts form books and other long form content. When you get stuck it's time to do three things:

Incorporate a new source that covers the same material in a different way, or at least from a different author.

Sit down with the concept and write about it and actively try to reformulate it and everything you do/don't understand in your own words.

Take a pause and come back later.

Usually one of these three strategies does the trick, no llm required. Obviously these approaches require time that using an LLM wouldn't. I have a suspicion doing it this way will also make it stick in long term memory better, but that's just a hunch.

sorry but if you've gone to university, in particular at a time when internet access was already ubiquitous, surely you must have been capable to find an answer to a programming problem by consulting documentation, manual, or tutorials which exist on almost any topic.

I'm not saying the chatbot interface is necessarily bad, it might be more engaging, but it literally does not present you with information you couldn't have found yourself.

If someone has a computer science degree and tells me without stack exchange they can't find solutions to basic problems that is a red flag. That's like the article about the people posted here who couldn't program when their LLM credits ran out

Will also reduce the context rot a bit.

Not saying it is indeed reality, but it could simple be programmed to return a different prompt from the original, appearing plausible, but perhaps missing some key elements.

But of course, if we apply Occam's Razor, it might simply really be the prompt too.

This is a good thing in many levels.

Learning how to search is (was) a good skill to have. The process of searching itself also often leads to learning tangentially related but important things.

I'm sorry for the next generations that won't have (much of) these skills.

Most people don’t know how to do this.

They want a student to use it and say “I wouldn’t have learned anything without study mode”.

This also allows them to fill their data coffers more with bleeding edge education. “Please input the data you are studying and we will summarize it for you.”

This isn't study mode, it's a different AI tutor, but:

"The median learning gains for students, relative to the pre-test baseline (M = 2.75, N = 316), in the AI-tutored group were over double those for students in the in-class active learning group."

However consider the extent to which LLMs make the learning process more enjoyable. More students will keep pushing because they have someone to ask. Also, having fun & being motivated is such a massive factor when it comes to learning. And, finally, keeping at it at 50% the speed for 100% the material always beats working at 100% the speed for 50% the material. Who cares if you're slower - we're slower & faster without LLMs too! Those that persevere aren't the fastest; they're the ones with the most grit & discipline, and LLMs make that more accesible.

I bring this up because the way I see students "study" with LLMs is similar to this misapplication of tutoring. You try something, feel confused and lost, and immediately turn to the pacifier^H^H^H^H^H^H^H ChatGPT helper to give you direction without ever having to just try things out and experiment. It means students are so much more anxious about exams where they don't have the training wheels. Students have always wanted practice exams with similar problems to the real one with the numbers changed, but it's more than wanting it now. They outright expect it and will write bad evals and/or even complain to your department if you don't do it.

I'm not very optimistic. I am seeing a rapidly rising trend at a very "elite" institution of students being completely incapable of using textbooks to augment learning concepts that were introduced in the classroom. And not just struggling with it, but lashing out at professors who expect them to do reading or self study.

unfortunately that group is tiny and getting tinier due to dwindling attention span.

During the early days of tech, was there prevailing wisdom that software companies would never be able to compete with hardware companies because the hardware companies would always be able to copy them and ship the software with the hardware?

Because I think it's basically the analogous situation. People assume that the foundation model providers have some massive advantage over the people building on top of them, but I don't really see any evidence for this.

True, and worse, they're hungry because it's increasingly seeming like "hosting LLMs and charging by the token" is not terribly profitable.

I don't really see a path for the major players that isn't "Sherlock everything that achieves traction".

If you want to try and make a quick buck, fine, be quick and go for whatever. If you plan on building a long term business, don't do the most obvious, low effort low hanging fruit stuff.

These days they’ve pivoted to a more enterprise product and are still chugging along.

It's still a work in progress but we are trying to make it better everyday

* for each statement, give you the option to rate how well you understood it. Offer clarification on things you didn't understand

* present knowledge as a tree that you can expand to get deeper

* show interactive graphs (very useful for mathy things when can you easily adjust some of the parameters)

* add quizzes to check your understanding

... though I could well imagine this being out of scope for ChatGPT, and thus an opportunity for other apps / startups.

Things well-represented in its training datasets. Basically React todo list, bootstrap form, tic-tac-toe in vue

I ask because every serious study on using modern generative AI tools tends to conclude fairly immediate and measurable deleterious effects on cognitive ability.

The criticism of cliff's notes is generally that it's a superficial glance. It can't go deeper, it's basically a summary.

The LLM is not that. It can zoom in and out of a topic.

I think it's a poor criticism.

I don't think it's a silver bullet for learning, but it's a unified, consistent interface across topics and courses.

unavoidably, people who don't want to work, won't push the "work harder" button.

I believed competitors would rush to copy all great things that ChatGPT offers as a product, but surprisingly that hasn’t been the case so far. I wonder why they seemingly don’t care about that.

I ultimately dropped the course and took it in the summer at a community college where we had the 20-30 standard practice problem homework where you apply what you learned in class and grind problems to bake it into core memory.

AI would have helped me at least get through the uni course. But generally I think it's a problem with the school/class itself if you aren't learning most of what you need in class.

e: if you mean university, fair. that'll be an interesting transition. I guess then you pay for the sports team and amenities?

When it just gives me the answer, I usually understand but then find that my long-term retention is relatively poor.

> The part Margie hated most was the slot where she had to put homework and test papers. She always had to write them out in a punch code they made her learn when she was six years old, and the mechanical teacher calculated the mark in no time.

Just like it's easier to be productive if you have a separate home office and couch, because of the differing psychological contexts, it's easier if you have a separate context for "just give me answers" and "actually teach me the thing".

Also, I don't know about you, but (as a professional) even though I actively try to learn the principals behind the code generated, I don't always want to spend the effort prompting the model away from the "just give me results with a simple explanation" personality I've cultivated. It'd be nice having a mode with that work done for me.

i don't get it.

You're also assuming that AGI will help you or us. It could just as easily only help a select group of people and I'd argue that this is the most likely outcome. If it does help everybody and brings us to a new age, then the only reason to learn will be for learning's sake. Even if AI makes the perfect novel, you as a consumer still have to read it, process it and understand it. The more you know the more you can appreciate it.

But right now, we're not there. And even if you think it's only 5-10y away instead of 100+, it's better to learn now so you can leverage the dominant tool better than your competition.

"The mind is not a vessel to be filled, but a fire to be kindled." — Plutarch

"Education is not preparation for life; education is life itself." — John Dewey

"The important thing is not to stop questioning. Curiosity has its own reason for existing." — Albert Einstein

In order to think complex thoughts, you need to have building blocks. That's why we can think of relativity today, while nobody on Earth was able to in 1850.

May the future be even better than today!

Apart from that, I do think that AI makes a lot of traditional teaching obsolete. Depending on your field, much of university studies is just memorizing content and writing essays / exam answers based on that, after which you forget most of it. That kind of learning, as in accumulation of knowledge, is no longer very useful.