Why would a carpentry shop buy hundreds of thousands of dollars of power tools without consulting with their employees to see what they actually need to get their job done more effectively? The logic of buying the tools then forcing the employees to use them "or else" is completely backwards in any sane world.
(Of course, we've all had bosses that went to some marketing seminar and come back having been tricked^Wsold into buying some wizz-bang widget that we need to now integrate because of a sunk-cost fallacy, but I thought everyone was on the same page that this is not how normal procurement was supposed to work.)
> the point is that if you wanna see if the tools are working you wanna see proof they're actually being used.
That is way too charitable, people were being fired based on these metrics and people were absolutely talking about token burn as being a metric for productivity (do I really need to link the Jensen Huang quote?). That isn't an indication of this hysteria being based on "just trying to see if the tools work".
If you want to see if the tools work, why don't you just ask your employees? Like any normal employer would?
> If you want to see if the tools work, why don't you just ask your employees? Like any normal employer would?
because that would require actually admitting that employees are the people in an organisation who are responsible for the success of that organisation, rather than the people higher up the org chart.
Do you actually have a job? Do you talk to your coworkers?
This is an insane take. Plenty of people are critical of AI at my job despite a big push to use it. I find the comparison to NK distasteful, coming from someone who presumably is pretty well paid and can quit their job whenever they want.
If you're feeling humiliated... well, I don't think it's because your boss wants you to try AI.
Having been the guy to speak the uncomfortable truths at such meetings, I can tell you that does not end well for anyone. Expect to look for another job shortly afterwards.
Narcissists, non-violent sociopaths, and control freaks end up in managerial positions (often more likely than the general population). The pointy haired boss in Dilbert is a popular representation for a reason. We've all been subject to degrading and/or stupid management trends (see also: https://ibb.co/Kx46rqkg ), and while in the tech industry we had a golden age were the engineer was king, that's been chipped away even before AI became mainstream. Also, hyperbole is a thing. :-)
>If you want to see if the tools work, why don't you just ask your employees? Like any normal employer would?
I run a small business with two employees.
N=2 here, of course, but one of them will experiment with any new process you introduce (as well as plenty more that you don't!)
The other will keep doing what he's always been doing, even if it's frustrating and inefficient, unless you monitor him and force him to use the new process.
I could imagine most "normal employers" would understand that both type of person exists and, assuming you're getting good first impressions from group A, it's usually better off in the long run to shove the new process down group B's throat.
(This isn't to say that the "Group B" employee is less valuable or anything - he is more conscientious and reliable than anyone else we've ever hired - but just that different people need different management styles)
In my experience, your first dev will have four thousand ideas and experiments on the go, and leave an absolute mess in their wake.
And your second will be struggling to clean up that mess while also getting their own work done.
Of course, you expect the same level of work from both of them, but because person two has to do a bunch of person one's work as well as their own, person one ends up looking better and gets praised by management.
You may want to consider that your Group B employee may be conscientious and reliable because they use an apparently “frustrating and inefficient” process. Productive friction is a thing: processes which force you to slow down enough to put careful thought into what you’re doing and why. And if they’re stuck in a loop of doing frustrating work - you may well consider why they’re doing so much frustrating work. Maybe that can be resolved at the managerial level!
> it's usually better off in the long run to shove the new process down group B's throat.
> (…) the "Group B" employee (…) is more conscientious and reliable than anyone else we've ever hired
If employee B is proving themselves to be valuable and reliable, then you should trust them to make the best decisions for how they’re going to go about their work and support them. Leave the door open for them to try different things, but no one likes having processes shoved down their throats (your words). All you’re doing is making them unhappy and more likely to leave to go work for someone who’ll value them like they deserve.
Thank you for taking the time out of your day to explain the best way to manage someone you've never met, working in an organisation you know nothing about.
> Why would a carpentry shop buy hundreds of thousands of dollars of power tools without consulting with their employees to see what they actually need to get their job done more effectively?
Are you suggesting that changes to new production technologies are always driven bottom up by line workers? I'm guessing that historically that's rare.
Historically that rarely happens because industrial equipment is/was generally too expensive for the average worker to purchase on their own, plus workers usually have a budget of roughly 0 to buy extra tools, especially expensive ones.
But to give you an example, also roughly 0 companies made developers use Linux and still many developers choose it, so bottom up improvements happen in a decent chunk of cases. Nobody paid for PostgreSQL promotion. Or Python, etc.
> so bottom up improvements happen in a decent chunk of cases. Nobody paid for PostgreSQL promotion. Or Python, etc.
It does, but for better or worse, it's an anomaly. Even now, maybe nobody was paid for PostgreSQL or Python promotion, but modern OSS tools and programming languages usually have a business backing it. Linux, too, wasn't commercially promoted until it was; RedHat isn't exactly a charity after all.
Conversely, no one paid for initial AI promotion either - ChatGPT exploded organically after release, and for the first year or two, companies had a problem because a good chunk of their staff, including especially non-engineers, discovered just how useful it was and wanted to use it at work, casually violating every internal policy, bylaw and even regulatory policies about data sharing. The massive spend on promotion - including first-party spend - came later, but at that point it was already obvious ~everyone is going to be buying it.
I suppose bottom-up vs. top-down may be in part about how mature a technology and industry is.
Then those same organizations that "give in" end up providing inferior AI tools, and you face the dilemma of either using poor models and accepting poor results or sharing data with better models that may not be governed correctly.
> Why would a carpentry shop buy hundreds of thousands of dollars of power tools without consulting with their employees to see what they actually need to get their job done more effectively? The logic of buying the tools then forcing the employees to use them "or else" is completely backwards in any sane world.
This is how it's gone down throughout history. It's why we remember the Luddites, textile workers who started smashing stocking frames and power looms because the machinery was introduced over their objections. The whole goal was to undercut the craftsmen's wages and bargaining power.
So, no, your expectation to be consulted was never going to happen and has not happened throughout history as industrialization has advanced.
> The logic of buying the tools then forcing the employees to use them "or else" is completely backwards in any sane world
People are stubborn. A lot of productivity improvements had to be almost forced upon farmers, for example. Even when early adopters demonstrated the benefits, a decent fraction of them just didn’t want to change.
And not few of those 'productivity improvements' for farmers have had disastrous consequences, even though what I think you are referring to has been implemented with far greater discernment and empirical basis than the current AI revolution.
People are stubborn, but sometimes for good reason. Let the stubborn people hold on to their practices, if the innovators are right they will eventually fold anyway.
> not few of those 'productivity improvements' for farmers have had disastrous consequences
Sure. Many have not. I’m thinking of stuff like ox-drawn and then mechanized ploughs, four- versus three-crop rotation, et cetera. The point is there is pushback regardless of benefit and even after it’s been demonstrated. Plenty of people are fine being comfortable. Which is fine. But it also explains why companies and societies with a nudge feature do better.
> if the innovators are right they will eventually fold anyway
Again, sure. If it’s their land, it gets acquired. If it’s your land they’re tilling, you get a say.
I’m not saying all-nor even most—pushback is unfounded. Just that there are plenty of cases where it is, and the solution there is to push through the change.
This is just a variant of the argument ”people don’t know what’s good for them”. You’re very close to the actual answer, which is that the aforementioned ”manager class” is simply convinced that they understand reality better than those below them, which is quite frankly absurd considering the fact that managers very rarely do any of the ”real work” that these tools supposedly make redundant, and yet they still believe themselves to understand the potential better.
Because people don't know what they want until they have and use it. Faster horses, etc. One can only really implement systemic change from the top down, as Moloch indicates.
> Why would a carpentry shop buy hundreds of thousands of dollars of power tools without consulting with their employees to see what they actually need to get their job done more effectively? The logic of buying the tools then forcing the employees to use them "or else" is completely backwards in any sane world.
For one, software tools are cheap, especially with OSS in the mix. You're buying one "tool" and paying for operational expenses that scale with total usage across all company.
But secondly, and more importantly, the "consulting" and discussing was done over the period of last 3 years, by ~1 year ago the high-level conclusions were pretty much locked in, the worthiness of the adoption was blindingly obvious at that point, so I can see why tokenmaxxing would be where this ended up, even though (here I disagree with the article a bit) the tools aren't at the "compounding correctness" stage just yet. It's really quite simple: the stick didn't work (telling people in increasingly direct ways to try using AI for stuff), so they tried the carrot.
$deity knows a good chunk of engineers will inadvertently fall for any trick that involves a scoreboard. That holds even when they're perfectly aware they're being tricked.
> If you want to see if the tools work, why don't you just ask your employees? Like any normal employer would?
Again, they did that, they've been doing it continuously over past 3 years. Some people are excited, some people don't care, but some - a population that's definitely overrepresented in HN comments - just stubbornly refuse to try. Now that the answers are in, and they speak in favor of AI, the companies are doing what "any normal employer would": trying to get the stubborn employers to do their job they way their bosses want them to.
(In fact, normal employers would be more eager to fire people who keep refusing top-down instructions - but it's also obvious this technology is experimental; the models and harnesses get more powerful faster than people can learn to use them - so carrots make more sense than sticks in this transition period. Stubborn people begrudgingly using those tools offer an entirely unique perspective and explore use cases and approaches that you won't get from excited adopters.)
You don't need peer reviewed studies to tell you water is wet.
Peer review is a technique to get evidence from data when SNR is low. It's not "science", it's just a technique. So is "throwing shit at a wall and seeing what sticks". Don't turn techniques into rituals, and science into religion.
Vibes are not evidence, neither is a curated demo. You need actual measured evidence that has an adversarial review to actually prove something without falling to confirmation bias.
Proof is not binary, it depends on the claim and the constraints you put around it, and the nature of the subject of your claim.
Most of the general LLM discourse in our industry is still closer to "proof of the pudding is in the eating" than to "double-blind studies on large cohorts, p<0.01, effect size is still so small that result is useless in practice"[0].
And we're not talking about curated demos either - most of the contested value can be proven for your own specific cases with little to no expenditure of money and time, at a PoC level (it gets more expensive once you try to operationalize it and find kinks that are hard to iron out).
And that is, the article claims (and I agree), the point of last 6-12 months of tokenmaxxing policies and top-down push - it's putting pressure on people to actually go and do those PoC-s for themselves, because just giving the opportunity and permission turned out insufficient for significant part of the workforce.
--
[0] - Ironically, I remember it was the opposite around the time GPT-4 came out. Back then people talked more about specific claims and demanded measured evidence, because it was hard to get the models to reliably do something interesting. But now that the models can handle bad prompting and can understand you even when you're drunk, suddenly people are denying the general capability of LLMs and asking for randomized control trials.
(For double irony, nowadays one can just ask an LLM for randomized trials; the current SOTA models will happily design you a bespoke eval pipeline if you ask them to.)
> And that is, the article claims (and I agree), the point of last 6-12 months of tokenmaxxing policies and top-down push - it's putting pressure on people to actually go and do those PoC-s for themselves, because just giving the opportunity and permission turned out insufficient for significant part of the workforce.
FWIW, I think most tokenmaxxing is, to riff of what you said earlier, turning a technique into a ritual and science into religion.
This isn't specific to AI, we've had it before with pretty much everything in software (and since well before software), from "object-oriented solves every problem" to "clean code [where every function is] two, or three, or four lines long", to reporting your daily kloc, to bounties for every bug reported and/or fixed.
Humans do what doomers are afraid AI will do: make a sounds-good utility function (tokens, lines of code, bugs, dead cobras) and get surprised when it is easily gamed for something far less helpful than the vision of whoever set the goal.
I think GP is being sarcastic, and pointing out that
1. "heavy rock falls faster" is what common sense will tell you (I was literally told this by multiple laypeople just a few days ago when sightseeing atop a tall tower)
2. This is disproven by a trivial experiment that nobody thought worthy of trying for millenia
3. therefore we do need peer reviewed studies to confirm even "obvious" knowledge.
Also, note that GP's parent post about "water being wet" is quite the subject of contention in scientific and philosophical circles, so that wasn't the best example either.
4. And we need.... something? to realize that both "common sense" / "multiple laypeople" and peer-reviewed studies are right.
Indeed, as per 2., no one is doing the experiments with rocks of different weight, and sufficient heights to easily measure time of fall. However, people have a lot of everyday experience with feathers, grains, leaves, wood, and rocks, as well as objects of various weight made of metal, paper, plastics. And in everyday experience, the heuristic actually holds out well: lighter stuff falls slower, or gets carried away by the wind.
This "heuristic" is purely empirical. You can't disprove it with peer-reviewed studies, because within its scope, it's literally the most basic, purest form of science: direct observation.
So in 1., the mistake is that of incorrect generalization. "Lighter stuff falls slower" is correct for everyday experience, it's the "therefore, heavy rock falls faster than light rock" is wrong.
Not because it doesn't fall faster, mind you - it does[0] - it's just that everyday experience is dominated by aerodynamic effects, and laypeople sometimes[1] mistakenly assign it to gravity.
Which I guess makes it a great analogy for the LLM story. Turns out everyday experience is actually valid in everyday situations. Generalizing from it is usually badly wrong, even if it sometimes arrives at correct answer for wrong reasons (and at wrong scales).
Generalization is hard.
--
[0] - Surprise. It's actually a heavy idealized particle falls at the same rate as light idealized particle. Actual matter is not an infinitely small point in space, and generates its own gravity field, so the heavy rock will land a tiny bit sooner than the lighter one, because it pulls Earth stronger towards itself - but then only if you drop the test bodies one by one (serially), and not together (in parallel, where the difference cancels out). But then it also turns out the mass canceling out for idealized particles isn't just a mathematical simplification, but a very deep truth about the universe...
[1] - Or don't. The question as phrased is, "does heavy rock fall faster than light rock"? This isn't a "specific physics theory question", it's a "real life" question. Treating a positive answer as belief on gravity is an error made by the asker.
This is possibly going to lead to a mind-blown moment for me as you reshape my entire understanding of physics. On the other hand, maybe you're slightly mistaken about newtonian physics?
> lighter stuff falls slower, or gets carried away by the wind.
Your examples are of smaller-density or larger-surface-area objects, not lighter ones. A bedsheet is heavier than a penny.
> Actual matter is not an infinitely small point in space, and generates its own gravity field, so the heavy rock will land a tiny bit sooner than the lighter one, because it pulls Earth stronger towards itself
When you're timing how long it takes for the rock to land, you're considering the Earth as fixed and applying gravity to the rock's center of mass.
The force applied between the Earth and the rock is F = G * (mEarth * mRock) / r^2
So the force that accelerates a twice-as-heavy rock is twice as large.
But the acceleration of that rock towards the earth is a = F/mRock, so in the end, if the rock is twice as heavy, its acceleration is still exactly the same as the lighter rock's.
> but then only if you drop the test bodies one by one (serially), and not together (in parallel, where the difference cancels out).
What are you talking about!?
If you want to split hairs, you could argue that if you drop them serially you're doing a minute change to the Earth's mass (which is actually so minuscule it makes no difference).
But even in your parallel universe of physics where the "heavier rock pulls the earth towards it", you're reaching a paradox similar to the one Galileo was testing for: if I link the heavy and the light rock together, they should fall slower than the heavy rock alone (because the light rock is slowing it down) but also fall faster than the heavy rock (because the total mass of the system is higher).
Ah. Here I am, taking the time to write this because I didn't have the useful bookmarklet[0] turned on in this browser window, and therefore I missed the emoji warning that would have told me I'm replying to some LLM trolling me with no understanding of physics.
>Why would a carpentry shop buy hundreds of thousands of dollars of power tools without consulting with their employees to see what they actually need to get their job done more effectively? T
What happens if employees say no power tools are needed and after a few months a competition shows up with power tools and hires a bunch of noobs and beating your production numbers and sales?
Your employees simply may leave the company and work for them and learn the new culture at this new competitor.
Is there any law which prevents people from moving between companies? No? Then the promoters of that company are going to do what they think is fit to keep them in business and stay competitive. Many times they'll be wrong, sometimes they'll be right.
To use your analogy again, it's kind of like the shop boss buying everyone a table saw and then saying "The best way to use the table saw is just experiment with it, it's the fastest and most accurate straight cut we can get - the future is table saw."
Yes, this is, in fact, how adoption of table saws and other such tools looked like, while they were still new tools. The basic form and function was established and its utility proven in both testing and early adopters, but as new kind of general tool on the market, every user from "early majority" was still writing the operating playbook for their specific shop conditions and kind of work they're doing.
So yes, it's a great analogy. We're right now well in the stage where bosses say, "evidence is in and conclusively shows this is useful for us, now the job is figuring out exactly how to work it into our particular business".
> figuring out exactly how to work it into our particular business
This is the most crucial bit. Neither ramming it down developers throats nor rejecting it wholesale is particularly productive. You need the conservative people onboard as well, to discover critical edges and failure modes. Including their criticism in the adoption process instead of bluntly banning it is the smarter move. Of course, there will be a few people who just don't play, they will fold eventually or be let go.
No, that's not what this analogy is about. At all.
See, table saws are dangerous. Famously so. One of, if not the most dangerous tools available to the general public. They spin quickly with lots of torque and pull things in faster than you can react. Pressure can also send loose pieces of wood backwards at high speed. Fast enough to pass through a person sometimes. It's like being hit with an arrow.
Tablesaw accidents can remove fingers and hands instantly, puncture organs.
They can be used safely but they're circumstantial, the worst thing you can do with a table saw is experiment. Once you realise there's a 12-inch razor sharp blade spinning at 3000rpm with up to 5HP you begin to respect how dangerous it could be and want to warn others.
It's intentionally hyperbolic. But you see what I'm saying here?
It's hard to be hyperbolic about danger and spinning objects, blades, chains after looking into the crazy world of farmers and military civil engineers; a shed built whipper snipper for young trees in a plantation to clean rows made out of chains welded to a tractor rim spinning horizontally hanging down behind rig on small tractor and driven by the PTO is not the scariest thing I've seen.
Long story short, people using tools often make and evolve their own tools to better do their work - and sometimes iterations of those proto-tools are kinda super bloody sketchy. There is care to be taken, there will be close calls.
I know GP said "available to general public", but my mind went straight to PTO after reading "One of, if not the most dangerous tools". Less common to see (especially in cities) than saws, but I think larger proportion of people understand they have to be careful near table saws than PTO shafts.
> after looking into the crazy world of farmers and military civil engineers
The whole history of aviation and space exploration is chock full of engineers, physicists and chemists doing crazy levels of experimentation.
That said, my point was different - unlike GP, I ask to consider workshops that had extensive experience with dangers of powered or high mechanical leverage hardware. It's entirely plausible and reasonable for people running those shops to say, "here is the new dangerous power tool, it's obviously pretty useful (ask your friends at $X or $Y if you don't see it), figure out how and where to best work it into our specific workflows, so it makes us most bang for the buck".
I think this is still a perfectly decent analogy for LLMs.
The danger may not be "personal injury before you can react", but it is both parts separately, as there's reports of them giving unsafe advice and also of them performing undesired tasks faster than humans can react.
(That said: while I regard "AI doom is marketing hype" to be a conspiracy theory when applied to OpenAI and Anthropic, public statements from this guy are absolutely a case where I'd say hyping up destructive power is the point of his job: https://www.ai.mil/About/Leadership/Bio-Page/Article/3940370...)
I see the point but, I'm not really sure the analogy holds up here. If i was in a cabinet shop and had to joint, plane and resaw and cross cut a pile of timber fresh from the saw mill for the next job I'd be very grateful for the jointer, the planer, the bandsaw and the table saw. I'd also be very grateful for the dust extraction.
In in total agreement with you though, forcing tools on employees is very dumb and is terrible leadership. Ask your people what they need to be optimally exceptional and go get them it. Then let them get on with it.
> In in total agreement with you though, forcing tools on employees is very dumb and is terrible leadership. Ask your people what they need to be optimally exceptional and go get them it. Then let them get on with it.
Some employees want AI tools, others don't. Standardizing SDLC workflows > each person does their own thing. So now you have to choose: do you require AI tool use that fit into a new SDLC? Or don't you?
I don't see why they have to be mutually exclusive. Assuming the vendor risk profile is acceptable to the infosec people and procurement are happy with the AI tools vendor relationship, then it's a tool inside the information security perimeter.
As long as there's evidence that work meets quality gates for any required customer audits, and your customers are happy and in the loop that AI is a thing that may or may not be used to produce the service, then those engineers that want it can have it and those that don't, don't.
Feels like a revision to an SDLC rather than a new one. Without seeing the SDLC it's hard to find common ground though. It really depends on how it's written and implemented and of course: culture. In the example we're working from sounds like the tools are being forced on people, and that's less infosec, SDLC and more unbearably bad leadership.
Not sure why you bring in "vendors" and "infosec". I'm simply talking about the situation of a software engineering team building something together needing to have similar workflows (supported by tooling/software) in order to work together effectively.
Because Japanese hand tools are objectively less efficient than power tools in a carpentry shop. The guys that want to use hand tools can go work in a boutique that charges a premium for that level of craftsmanship. If you told them to use power tools, no amount of utility would convince them to use them, with most of their justification being psychological. Also, "It is difficult to get a man to understand something, when his salary depends on his not understanding it."
So do a workshop on power tools, measure their efficacy and the quality of the result, do some demonstration videos on power tools, get people to compare, seek feedback on their usage. Don't count electricity and sawdust, or you'll find people getting very good at expensively turning blocks of wood into sawdust.
Is the idea that most stubborn employees would adopt AI if their company made videos showing internal metrics that AI is better?
> Don’t count electricity and sawdust
I agree that it seems wasteful, but is there some better way to accomplish it at the scale of hundreds, or hundreds of thousands, etc? I'm personally doubtful that stubborn employees would switch even if a video provided internal metrics, videos, etc.
Nobody started out with that conclusions. The tests and experiments and workshops and consulting with select employees you propose, were all done 2-3 years ago. Results are in, but a chunk of population decides to ignore them and obstinately continue believing and claiming that it doesn't work, which is a conclusion they started out with.
Yes, "results are in". They're all over the map, about productivity, about stress and churn, about trust, about public sentiment, etc.
But sure, if you want to tell people their productivity will be measured by token usage, they will certainly respond to that incentive by setting your checkbook on fire while they work on a job search.
If a company wants to provide AI accounts for people, along with guidance for usage and non-usage, that might well make sense for some jobs. It certainly makes sense for some uses. If they start measuring token usage, that's even worse than when companies tried to measure lines of code written.
As a matter of fact, Nature does regularly publish papers about wetting properties of water. In fact, it just published one last week, from Nature Physics:
Scientists find more or less everything very interesting, even (especially?) things that are supposedly self-evident. You can both make a big splash disproving self-evident things, and much can be learned from it.
What? The basic properties of water are scientifically defined as best we can. There is mathematical proof that 1+1=2. Do you think science starts from nuclear physics or differential equations?
We couldn't build any of this stuff (rockets, LLMs, heart medicine) if the foundation was ill defined.
I think it's the second time I run into you like this, Temporal. I wish HN had a way to classify you as an "AI booster" or equivalent.
> What? The basic properties of water are scientifically defined as best we can. There is mathematical proof that 1+1=2. Do you think science starts from nuclear physics or differential equations?
Yes. There's a lot of interesting things science has to say about water, very specific claims that took a lot of effort to discover, precisely formulate, and reproduce.
We're not talking about those. The whole LLM discussion on HN, as well as in the wider industry, is still stuck at the state where a large (or vocal) group of people refuses to believe water is wet. Yes, there is a similar group that tries to sell water as miracle cure, I'm not denying it - IMO both perspectives are dumb and entirely detached from obvious observational evidence that you can collect for ~free at home in 15 minutes. Example will follow.
There exist the equivalent of foundational, detailed studies on LLMs, at every level of rigor imaginable (with a caveat, it's hard to rigorously prove anything useful in software engineering; it's still largely opinion-driven field). But they're not part of the overall "AI hypers/haters" dynamics.
> I wish HN had a way to classify you as an "AI booster" or equivalent.
You can take any of the LLMs and have it vibecode you a user script in under 5 minutes, than you then can paste into Greasemonkey/Tampermonkey, and voilà, you have me labeled as "AI booster" or filtered out.
In fact, let me help you, I'll time it. I opened chatgpt.com in incognito (to emulate being a rando free user), and put the following prompt in:
> I need a user script I can paste into Tampermonkey on my Firefox that will clearly label user named TeMPOraL with robot emoji and some silly emoji, so I never forget when reading their HackerNews comments that they're an unapologetic AI booster.
This is the promised empirical example. It doesn't prove everything, but it proves something, and it took, end-to-end, a total of 1 minute to perform just now. You can collect many such examples over a single day by just trying. People who keep saying AI is useless and a fad and can't do anything useful, obviously never bother with even that.
FYI: I'm not an AI booster. I like AI, and I find it useful, but I'm not going out of my way to boost it. I just enjoy this topic, but more importantly - and I remain consistent in this - I point out bullshit that doesn't agree with obvious observable reality.
EDIT: try the example yourself, and post whether it works for you too - if it does, it's technically a peer-reviewed, replicated study, but I doubt it'll convince any of the naysayers of anything.
EDIT2: I have plenty of negative things to say about LLM capabilities and how irresponsibly people use them, and I do occasionally write about this (mostly at work, these days), but most HN threads on AI are not on this level - not anymore. They used to be more reasonable back in GPT-4 days.
At the risk of abusing the analogy further: many people aren't refusing to believe it's wet, they're observing that sulfuric acid is also "wet" and can look similar upon visual inspection, and there's a lot of harm coming along with the demonstrated capabilities, in addition to those capabilities themselves being fickle and inconsistent (not a desirable property for a good technology).
This isn't a problem of "doesn't know what AI can do"; yes, some people are misinformed, but you shouldn't dismiss all refusal to use AI as being misinformed. This is a problem of "knows what AI can do, and based on that informed position thinks it's terrible and should have careful guardrails around it".
They're not "3-4 trillion dollars in investments over 5 years" useful, nor "crammed into the throat of every employee on the planet, regardless of their actual job" useful.
The way they are pushed right now will lead to a very hard crash and probably lots of suffering.
Also, you need a more advanced prompt for Firefox on Android :-p
> They're not "3-4 trillion dollars in investments over 5 years" useful
Why not? They're a general-purpose technology, in the same category as "software" or "electricity".
> nor "crammed into the throat of every employee on the planet, regardless of their actual job" useful
They're potentially useful for anything that can be fed into computers (VLMs lifted the "that can be expressed as text" limitation, visual and audio tokens are not a separate category to text tokens anymore). That touches every single job people do in some aspects. Even though LLMs can't do physical work for people, they're still able to help with directing it and teaching it.
"Cramming into the throat of every employee on the planet" was already covered by many comments here, and the article itself - it's about forcing the obstinate holdouts to at least try.
> Also, you need a more advanced prompt for Firefox on Android :-p
No I don't; literally copy-pasted it to Tampermonkey on my Firefox on Android just now, and it works there out-of-the-box too.
Regarding LLMs, they are pushed too hard and too abusively by business people. Employees are being laid off and replaced with chatbots that don't do the job. Frustrating if support for McDonald's, risky if health insurance support. Also the financials don't make sense. AI companies are money pits. Money is ultimately production. We make X amount of stuff yearly, globally. We can't afford to through away 5% of X yearly on technologies that will probably have a proper return in 5 or 10 years. When we mis-allocate resources on scales like these, people die. Look at Communist centralized planning. For $3-4 trillion we could have solved a LOT of actual global problems.
LLMs are fine but they should have matured in the software dev domain for 2-3 more years and then non tech products would have followed.
FYI: I just had three SOTA LLMs + NotebookLM all fail at the simple task of explaining to me where to put a powder detergent in my particular newly bought washing machine, despite having photos of the machine and ability to find the manual (in case of NotebookLM, it literally had the manual as its only source).
After first failure (Gemini 3.5 Flash + NotebookLM), I run the other two (Opus 4.8 on Extra; GPT 5.5 on High) in parallel, and looking at their thought streams, I gave up and dug up the manual and read half of it, before the LLMs finished coming up with - wait for it - wrong answers.
Super frustrating. Doubly so, given that I use them for comparable tasks pretty often and they usually sail through them flawlessly. But this experience happens every now and then. It's only fair to report it, if only so you don't think I'm just AI boosting all the time.
1 is for powder detergent, 2 is for liquid detergent, 3 is for softeners and such.
All three LLMs (Gemini twice, since NotebookLM) insisted I should put the powder detergent into leftmost compartment (2 on the ASCII diagram above). They referred to it by different numbers, but all gave some convincing justification why to put the detergent there. That's despite me posting photos of the compartment drawer, with symbols clearly visible. That's despite demanding they find the manual and cross-ref. I even asked two (Gemini and Claude) to label the actual compartment on the photos I took[0], and both produced some nonsense, with labels in all the wrong places. And they all insisted they're right and issued plenty of warnings about making sure I get this right or else bad things will happen.
BTW. I ended up posting a screenshot of the diagram in the manual to Claude with a passive aggressive comment. Looking at its "thinking summary" and tool calls now, I think at least Claude didn't process the image correctly and only saw:
[ 2 | 3 ]
------------
as those parts are blue, while the bottom is just in the same color as the entire body/frame of the machine. Maybe the contrast was too low for the models. But it was okay for humans, so it's not excusing much, especially that they all claimed to have found the manual, which had a high-contrast diagram.
(Current experience tells me they probably didn't really check the diagram. I noticed recently that all major models seem to have gotten lazy when it comes to reading sources, and are also more than happy to lie about it.)
--
[0] - A method I often use with Claude when I'm not sure if it's dealing with spatial tasks correctly - I have it produce intermediate artifacts that involve modifying "ground truth" inputs - e.g. placing two map pictures on top to verify it solved the coordinate transform, and/or (like here) drawing labels and boxes on top of original photos. I found such requests to be helpful enough I set it as general rule for Claude now.
[1] - Which normally they'd spot, but for some reasons, they didn't.
> I have it produce intermediate artifacts that involve modifying "ground truth" inputs - e.g. placing two map pictures on top to verify it solved the coordinate transform, and/or (like here) drawing labels and boxes on top of original photos.
Like you I use intermediate artifacts all the time, but have never tried with visual elements. But how do you get Claude to modify images? Do you get it to output things to a canvas? html? use an external library?
> Why would a carpentry shop buy hundreds of thousands of dollars of power tools without consulting with their employees to see what they actually need to get their job done more effectively?
I mean, the difference in the metaphor is that we have pretty fully understood carpentry for many hundreds of years. We still find it difficult to write even simple software to address all our needs, as is evidenced by the insane pay in our industry. Carpenters can suggest tools because they know what's out there. The same was not true about LLMs a year ago.
> That is way too charitable, people were being fired based on these metrics
People get fired for all kinds of reasons including no reason at all. Oftentimes leadership even lies about the real reasons for firing people because they don't sound good!
I'm gonna be blunt: if you're in software and you refuse to use AI for moral reasons, I think you should be fired. There's being principled and there's being obstinate and the difference between the two is how well you can convince people that you _have_ principles. Most LLM-hating people fall short on this point, because
> do I really need to link the Jensen Huang quote?
Sure! Link it again, we all know it's highly immoral when shovel salesmen try to make you want shovels.
> If you want to see if the tools work, why don't you just ask your employees? Like any normal employer would?
I do not like this HN take of "let's do this thing that works great in small companies and then just blindly pretend that it'll also work at the largest companies in the world!" No, this doesn't work at "normal companies" because you cannot "just ask" 30k+ employees what they want.
Employees, like EVERYONE ELSE, are resistant to change. If I, as CEO of a company, want to get my company to try Claude I have to measure tokens to see if it's getting used. That's it. There's no wave of delusion here.
The logic of trusting employees who are worried that power tools will replace them to utilize power tools effectively is completely backwards in any sane world. People don’t like change, sometimes it needs to be forced on them.
Doubt. People brought in all kinds of web applications in the early Web 2.0 era because corporate IT was being too stingy (for a lot of reasons). People will find efficiencies on their job on their own. No need to denigrate them.
Yeah but if you can't attack the workers and make them hate their lives, are you even a good capitalist? Didn't Milton Friedman die for our bosses right to stomp on our faces in the pursuit of profit?
I don’t know, at my company at least tons of devs were holding out on ai usage until the token maxing stuff really started. It was beyond clear by that point that coding agents were a productivity multiplier.
A lot of people believe that. Not a lot of evidence on the table for it (it’s not agent developers’ fault; empirical studies are expensive and rarely live up to scrutiny). Not sure it’s worth forcing people unless you like malicious compliance.
Well here’s where you can level valid complaints against management I think. “Move fast and break things” doesn’t line up super well with “wait for empirical studies to back up your suspicions”
For sure. Just because the studies are incomplete or difficult doesn’t mean they’re useless. We still do unit testing and type systems continue to get more sophisticated and spread further because we believe they have an effect on quality and productivity regardless of the lack of evidence.
However it takes some taste in engineering and perhaps some mathematical sophistication to figure these things out. “Just use AI,” is not a very convincing argument either.
> who are worried that power tools will replace them
maybe, just maybe, it would have been a better idea to engage with employees first rather than posting on linkedin about how everyone is going to lose their jobs.
cos it's the kinds of people trying to force this stuff on employees that are the ones who have been shouting about that from the rooftops.
If you take LinkedIn at face value everyone who uses the Internet is a sociopath who lives for no purpose beyond maximizing shareholder value.
Seriously, some of the most deranged things I've ever read were by relatively normal people trying to promote themselves on LinkedIn.
What people SAY does not matter nearly as much as what everyone KNOWS and it's pretty damn clear that AI is never going to be able to replace humans in complex domains. Every time a frontier lab announces a breakthrough it's pretty obvious that the setup was more complicated than "hey chat prove the Riemann hypothesis."
The world is gonna need skilled human beings to drive LLMs, no matter how desperately some people like to pretend otherwise.
(Of course, we've all had bosses that went to some marketing seminar and come back having been tricked^Wsold into buying some wizz-bang widget that we need to now integrate because of a sunk-cost fallacy, but I thought everyone was on the same page that this is not how normal procurement was supposed to work.)
> the point is that if you wanna see if the tools are working you wanna see proof they're actually being used.
That is way too charitable, people were being fired based on these metrics and people were absolutely talking about token burn as being a metric for productivity (do I really need to link the Jensen Huang quote?). That isn't an indication of this hysteria being based on "just trying to see if the tools work".
If you want to see if the tools work, why don't you just ask your employees? Like any normal employer would?