Who does your assistant serve?

> Not to mention, industry consensus is that the "smallest good" models start out at 70-120 billion parameters. At a 64k token window, that easily gets into the 80+ gigabyte of video memory range, which is completely unsustainable for individuals to host themselves.

Worth a tiny addendum, GPT-OSS-120b (at mxfp4 with 131,072 context size) lands at about ~65GB of VRAM, which is still large but at least less than 80GB. With 2x 32GB GPUs (like R9700, ~1300USD each) and slightly smaller context (or KV cache quantization), I feel like you could fit it, and becomes a bit more obtainable for individuals. 120b with reasoning_effort set to high is quite good as far as I've tested it, and blazing fast too.

What's worth noting is that the companies providing LLMs are also strongly pushing people into using their LLMs in unhealthy ways. Facebook has started shoving their conversational chatbots into people's faces.[1] That none of the big companies are condemning or blocking this kind of LLM usage -- but are in fact advocating for it -- is telling of their priorities. Evil is not a word I use lightly but I think we've reached that point.

[1]: https://www.reuters.com/investigates/special-report/meta-ai-...

> I feel like this should go without saying, but really, do not use an AI model as a replacement for therapy.

I know several people who rave about ChatGPT as a pseudo-therapist, but from the outside the results aren’t encouraging. They like the availability and openness they experience by taking to a non-human, but they also like the fact that they can get it to say what they want to hear. It’s less of a therapist and more of a personal validation machine.

You want to feel like the victim in every situation, have a virtual therapist tell you that everything is someone else’s fault, and validate choices you made? Spend a few hours with ChatGPT and you learn how to get it to respond the way you want. If you really don’t like the direction a conversation is going you delete it and start over, reshaping the inputs to steer it the way you want.

Any halfway decent therapist will spot these behaviors and at least not encourage them. LLM therapists seem to spot these behaviors and give the user what they want to hear.

Note that I’m not saying it’s all bad. They seem to help some people work through certain issues, rubber duck debugging style. The trap is seeing this success a few times and assuming it’s all good advice, without realizing it’s a mirror for your inputs.

> At a 64k token window, that easily gets into the 80+ gigabyte of video memory range, which is completely unsustainable for individuals to host themselves.

A desktop computer in that performance tier (e.g. an AMD AI Max+ 395 with 128 GB of shared memory) is expensive but not prohibitively so. Depending on where you live, one year of therapy may cost more than that.

I can’t help but think we’re accelerating our way to a truly dystopian future. Like Bladerunner, but worse, maybe.

We should not forget that LLMs simply replicate the data humans have put on the WWW. LLM tech could only have come from Google search, who indexed and collected the entire data on the WWW and the next step was to develop algorithms to understand the data and give better search results. This also shows the weakness of LLMs, they depend on human data and as LLM companies continue to try to replace humans, the humans will simply stop feeding LLMs their data, more and more data will go behind paywalls, more code will become closed source, simple supply and demand economics. LLMs cannot make progress without new data because the world-culture moves rapidly in real-time.