I agree, they are just tools, but the way they are presented; taking natural language as their input and responding in kind strongly influences the way our brains react to them on a social level. That's what I mean by their social dimension.
As I said in another comment, to me this seems like it must have a strong personality or psychological component of some kind. I'm really not trying to be condescending or anything, but I just simply do not experience that reaction. I know that I'm interacting with a computer program, I would have to actively pretend to experience this social reaction.
I understand that many people seem to experience it totally differently.
Sometimes I wonder if this is a personality thing. It has never occurred to me to anthropomorphize these things. But it seems like a huge number of people struggle with this.
My daughter is a pediatrician and just gave a talk on mental health problems (including suicides) brought on by the sort of personality that succumbs to this.
I find it interesting. For most of my life, I have gotten on well with (well-designed) machines. I'm the sort of guy who, when they bring me in to show me what's broken, it works in front of me.
Maybe I get on with well-designed machines better than people. Because people are never well-designed. So, when LLMs are working properly, I get along well with them, but when they're not, I'm unhappy. But I never confuse any of my machines for people.
Yeah to me it's just "oh what a nice interface for compressing information and accessing it upon request!", not at all "I'm talking to a person". But I dunno, maybe we're the weird ones.
That's a very... bleak perspective on your brother's motives. Sometimes you just want to get rid of an object that causes frustration. When that attitude is applied to humans, we usually call it a symptom of dehumanization.
> That's a very... bleak perspective on your brother's motives.
Well, it may have been an overstatement, but not by much. Look, I have anger issues, but I never get angry at things because they are never trying to hurt me, and nothing good can come of it. I could probably fix most remote control issues (e.g. bad battery contact), but not after it's been smashed.
I also tend to feel the same way about, e.g., a rabid dog or vermin. Get rid of them, but not in anger.
I'm not saying it's rational either way. But getting angry at things is, at least as far as I can tell, normal for most people, without any subconscious idea those things are people.
> without any subconscious idea those things are people.
Right, but there is a subset of anthropomorphism that relates to sentient beings, which is what I was trying to get at. Being angry at something is one thing; wanting to immediately destroy it in anger is possibly another.
I will admit I don't know the exact feelings involved, because I generally don't have them. The closest I come is the anger I feel at the designers of IVR systems that are too stupid to answer your questions and too obstinate to connect you to a human. I admit to yelling at those, but honestly part of that is that I have read that some of them are designed to sense frustration in users and connect you faster to an agent in some cases. But I have no idea how true that is, or whether it's just like pressing the completely impotent "close door" button on an elevator.
> Being angry at something is one thing; wanting to immediately destroy it in anger is possibly another.
This is a real distinction, yes. I just don't see what it has to do with anthropomorphism. Like maybe you're saying the second one is more often directed at people-like entities? I don't feel that at all.
> So, when LLMs are working properly, I get along well with them, but when they're not, I'm unhappy.
So… you think yourself “different” while acting like everyone else does when something doesn’t go as planned (unhappiness/sadness/disappointment/whatever)?
The "machines like me" is not a humble brag. It's an observed phenomenon, not just by me.
The "I'm different in how I react to them" probably doesn't put me at odds with the entire population, but certainly puts me at odds with a lot of individuals I have witnessed.
If you read my other comments, you'd see me contrasting myself with people who get angry with machines. I don't do that.
For whatever reason, I reserve anger for sentient beings who I believe are trying to fuck with me. That has its own set of problems, but I never have the problem of destroying inanimate objects in anger.
> So… you think yourself “different”
Yes.
> while acting like everyone else does
That's the point. I know for a fact that not everybody acts like me, never mind having the same facility with machines as I do.
I suspect harness and usage modality is a big factor. Personally I disable all "memory" features wherever possible. I don't want the LLM to "get to know me", or any illusion thereof, which I assume helps with not anthropomorphizing it.
I use memory all the time so that I can ask it about things. That doesn't mean it "knows me" any more than my notes documents do, it's just more convenient to search.
Yeah I don't think we disagree that much about this. But I'm not really worried about this particular habit forming risk. I can see where you're coming from though.
But frankly, I mostly use (greenfield, non-remembering) LLMs directly to counteract the enshittification of search brought on by attempts to use AI to remember things about me (and especially to sell me things).
This works well for me because (a) I'm usually using my desktop, and (b) I'm a touch-typist. I want to say that my search results have gotten back to where they were a decade or more ago by doing this, even though I often have to give a couple of prompts in order to get the LLM to provide me a useful link.
I guess I'm also pretty skeptical of the "automatically remember things about me" functionality. But I really like being able to say "hey make sure to remember this fact" and being able to ask about those things later.
For me the convenience of “hey remember this fact” is outweighed by a desire not to get stuck in a search or context bubble.
It might be nice to have better UI to control which bits of history get added to the context of a chat, but then just use a coding harness instead of web UI
Maybe. But I don't just use it for coding. For instance one thing I use it for is helping me track and evolve my workouts over time. It helps that it can remember the weight and number of reps I did the previous time. I could make an app for this, but it works well to do it purely within chat, with memory, and that's convenient. This is just an example, I use it this way for a number of things. I'm not going to use a coding harness for this kind of trivial use case!
Yes. It’s called the ELIZA Effect. It’s fascinating, and arguably the most dangerous thing about these models given how many people it seems to affect.
Yes, but then it's really important to demonstrate that this is true. If they invest in what you propose, time to market for new features needs to actually become faster, or customer churn needs to actually decrease. It's not enough to put the proposal in business-y terms, it has to actually effect the claimed improvements to the business.
You have to pick metrics to show management that they understand the importance of, and then be able to demonstrate degradation in those metrics when you don't do what you propose, and improvements when you do. Accomplish this and you'll build trust.
Too often, what happens is that the proposed benefit is vague and not empirical, and then the benefit is not actually realized by a large investment into it, destroying trust.
Sometimes it matters and sometimes it doesn't. The hard part is figuring out which is which. The most effective engineers are those who maximize the amount of time they spend picking the right point in the trade-off at the right times. Choosing a preferred point on the continuum and sticking to that at all times simplifies decision making (itself a useful thing!) but it's not the optimal strategy.
There are a couple ways out of this conundrum. One is to try to get really good at picking the right point on the continuum as much as possible, which is essentially a forecasting problem (and thus it's really hard!). I think the somewhat easier choice is to pick roles that align well with your style. If you have a deliberate and near-perfection preference, you can seek to work on projects where there is no question of the importance of correctness. If you prefer the opposite, you can work on prototypes and zero-to-one type projects, and that will be more satisfying (and less catastrophic).
I totally agree with the skepticism that we'll ever get to a point where natural language becomes the "formalism" and stop needing people who understand the actual formalism underneath. And I agree that you don't need LLM based tools.
But things that you don't need can still be (and often are) incredibly useful. Nobody needs an IDE, nobody needs vim or emacs or bash or even compilers or assemblers.
But we have all those tools and they are useful. That is, their utility is net positive.
Using LLMs to generate code currently also has (wildly) net positive utility. Maybe that will change because some part of the calculation changes. But this is the situation right now.
The bit about "but the UI" sounds like a developer thinking "UI is a solved problem" in the same way MBAs are told "coding is a solved problem".
Product people do seem to think it worth exploring "personalized software for everyone" like literally each person gets their own custom UI. This does sound like a support mess but it's not obviously wrong when you think about the Microsoft Word alternative.
I agree LLMs are useful tools, but I am dismayed by a lot of cargo-culting around them, which happens because we don't understand them. I think we will have much better tools when we understand what is the expectation and what is the algorithm they run.
Somebody else said that the magician analogy was poor. I like magic tricks, but it took many years of cultural change (influenced by people like Houdini, Randi, Penn & Teller) to stop illusionists (and mentalists) make claims they have supernatural abilities, or people believing it on their own (a magician pretending to be able to catch a bullet was shot by an audience member who didn't understand the distinction).
It is detrimental, I think, to treat LLMs as if they have magical abilities ("superintelligence") rather than understanding they just run some clever algorithm. The fear for (programming) jobs comes from that framing; nobody fears of their job because of compilers, since compilers are understood.
(And it actually runs against kind of "socialist" framing of the problem, which I agree with, that is why should people be worried about the jobs in the first place, when society is getting richer as a result of better tools?)
I see what you're saying, but I don't really think understanding how they work is the problem. We do understand how they work. You can go read the many papers that have been published as the technology has been developed. It is not mysterious. We don't need to understand the "clever algorithm", we already understand that. The sense of mystery is because the results of the algorithm are non deterministic, and because the number of parameters that determine the outcome is so large that it appears to exhibit emergent behavior.
I guess I agree with your general premise that tools we use should not be treated as magic. But I don't think this is a problem to be solved, it's possible to learn how these things work, and people should definitely do that.
I disagree we understand LLMs, your view is quite reductionist. It's like saying we understand human brain because we understand physics and evolution, or that we understand some program because it's just a binary code in a known instruction set.
I think Dennett's theory of intentionality (see https://en.wikipedia.org/wiki/Intentional_stance) applies here. We do understand LLMs from physical stance (ML algorithm and inference), but we don't fully understand them from design stance (it's internal workings have been evolved so it's hard to tell the functional units) and from the intentional stance they are a complete mystery.
And I talk about obstacles to this understanding elsewhere in this thread.
IME, it works, but at the cost of a pretty large amount of overhead, both in performance and (I think more importantly) cognitive.
Honestly I never did come to a satisfying conclusion on whether I thought it was worth it. The teams I worked on that used it found it neither simple nor easy to use, and I was never sure whether we were really reaping the benefits of correctness in the face of failure at the scale we were running. We eventually migrated everything off of it, and everyone was happier, but perhaps (probably) we had more lurking bugs in distributed failure cases. But to my knowledge we never tracked down an incident to the kind of problem that temporal solves.
I guess I'd say that I like durable execution (or at least temporal specifically, it's the only system like this that I've used) in theory, but not really in practice.
I also experience this as a great loss, but honestly yes, I can't get excited about perfecting the craft of writing computer code now that there are AI systems that have done so. I'll always be sad about this! But the flip side is that I'm way more excited about what I can do with software, now that it's so much easier to write.
I experience quite the opposite. I'm quite excited about the value my skills are gaining in relative terms. An entire generation of potential competition is getting addicted to the subsidized corporate tooling. Voluntarily!
What I fear is the future career path for manual programming is fixing AI generated systems. I suspect it will pay very well because you will be finding and fixing things the AI driven work can't. It is going to be soul destroying as reading AI code is deeply unpleasant in the same way reading the articles is.
I never go by pay. I won't fix bad systems whatever their provenance because trying to understand poorly coded software is very unpleasant.
I go by interesting problems and I like being able to chose from more options due to lessened expertise competition. Why people voluntarily renounce expertise is beyond me, but they are and I welcome the increase in choices of engagement I have been experiencing.
I would never engage in uninteresting work at any rate of pay, and working with LLM output directly or mediated is quite uninteresting to me.
The code is ultimately the codification (ha!) of domain knowledge in precise, non-ambiguous ways. It has extrinsic, non essential complexity yes.
But including ever more complexity in the process, by depending on complexity-slinger systems that you don't fully control or even understand, is not the solution, in my view.
If you lose control over the precise, detailed specification of the domain knowledge that the code is, ultimately you are just a glorified business analyst. Besides, the inherent complexity of the implementation is not without essential complexity that needs mastering for security and performance reasons. Deferring that mastery to some tool is self defeating. The complexity doesn't disappear. You're just shuffling it around and when it leaves your purview you are merely creating opportunity for someone else to capture that value.
Ultimately yeah the vision of being an LLM jockey who doesn't know code might be viable or even numerically dominant in the market in future. Why would you descend the value chain though, and voluntarily, that I don't understand. Competing for volume isn't historically in the interest of the autonomous worker. Maybe I'd feel differently about it if I owned capital and had some corporate strategy behind it. As an independent actor, I don't see it favoring me.
AI is a very useful tool for finding problems in codebases.
I'm very far from an "AI is going to take everyone's job" doomer, but I really do think "there will be programming jobs that don't involve using LLM based tooling" seems pretty unlikely.
The future is open models running on your device. No subsidized corporate tooling required. There will never be a reckoning for the people who are letting their coding skills atrophy.
Yeah I know it's not obvious. And very lucrative and very freeing, having this sort of knowledge and leveraging it.
Especially after clients have suffered the new found arrogance of non-experts riding on LLM output.
Look, let me be perfectly clear: if you're getting what you want out of your job and your personal projects, hobbies, whatever you do with your time, awesome! If you feel like the trade-offs of including LLMs in your workflows are good for you, great!
Just don't delude yourself into thinking this new technology, this new tool, is free of trade-offs. Net positive for you? Ok if you think so go for it. I haven't found the inclusion of LLMs in my activities beyond replacing the broken state of traditional search worth it. That's it. I'm not against it, I just don't find myself getting net positive value from deferring to code generated by these tools, is all I am saying. Nothing more, nothing less than that simple statement. Not for me.
Maybe! But probably not, IMO. There have been a number of generations now that have been "addicted" to using digital spreadsheets rather than doing bookkeeping by hand.
I don't think this is in any way a disagreement between us. In just the same way that spreadsheet software did not kill demand for accountants, I do not believe AI will kill the demand for other professionals.
It was quoting a word from this sentence in your comment:
> An entire generation of potential competition is getting addicted to the subsidized corporate tooling.
I think that using LLMs as tools for work is just like accountants using spreadsheets. If you want to describe both things with the word "addicted", then fair enough. But in my view, in both cases it is just people using a technology that is useful for their work.
And in both cases it leads to skill erosion in the workforce. We have a shortage of accountants and we will have a shortage of software developers when a few years down the line an entire new generation is addicted to corporate tooling and has been disincentivized to learn fundamental skills. So to the point: it is a real problem that has played out before. Ask anyone in manufacturing how much great welders are paid and if they are easy to find.
No, this is clearly not true of accountants! Using digital spreadsheets is not "skill erosion", it is the skill! That doesn't mean they don't still need to learn how to do the calculations that are encoded into the spreadsheets, of course that fundamental knowledge is still important. But running all those calculations by hand in the regular course of their work would be a giant waste of time.
It does seem that there is a shortage of accountants, but this doesn't seem to have anything to do with it... Like, "not enough people are entering CPA programs because it requires a big time commitment for not the best pay and horrible hours during tax season" seems like the main reasons and "the workforce got too addicted to Excel" seems like not one of the reasons at all?
What they are good at is finding, summarizing, and providing links to perspectives from other humans.
reply