My guess is that the models are overtuned on coding troubleshooting. You can regularly see them overthinking anything when you ask them to code something these days, mulling over countless eventualities. This is an okay idea for software engineering, but it leads to extremely diverging behavior in many other cases.
The problem is that empirically regime changes very rarely lead to stable liberal democracies. Yeah, a magical societal transformation according to exactly our standards sounds great to us. But that is not how this is going to end.
But who is there to say that liberal democracies are the desired end of goal for all states? That is the problem with liberal democracies it has very little tolerance for almost any other kind of system.
How many wars in recent history waged in the name of liberal democracies and freedom?
> But who is there to say that liberal democracies are the desired end of goal for all states ...
Everyone. That's the essence of democracy. It quite literally is what everyone wants.
> How many wars in recent history waged in the name of liberal democracies and freedom?
Zero? I guess you mean things like the Iran war, if we disregard entirely that Iran has been attacking almost literally everyone, including the US, but also including, uh, Iraq, Israel, Jordan, Turkey, Afghanistan, Syria, Yemen, ... they attacked fucking Argentina (look at the map, then tell me how ridiculous that is, then check wikipedia to see they did indeed attack Argentina)
Sorry I mean "exporting the revolution", with little or not explanation as to why this particular export required killing 600000 Syrian civilians, and certainly 2 million people total.
What I just can't understand is why tankies support Iran. I mean, how can you get less communist than Iran? I mean is it the words they use and the propaganda? Idiots.
Given a choice I would rather live in democratic systems - but to say that everyone wants democratic systems tell you are not as exposed as you think you are. There are many failed democracies which resulted in highly bloated and corrupt governments, and these tend to make the situation worse. And there are many states running alternative governance system with success.
When you are saying everyone, you are basically projecting your liberal values on others and you are convinced they are superior and that is why it is dangerous.
Regarding wars - Iran, Iraq, Libya, and many other examples (coups) were initiated in the name of liberty and freedom. And notice that all those countries are rich in oil.
Regarding Iran specifically, you won't understand until you study the Shai's faith and history which I'm sure you don't have any knowledge or understanding of.
> Given a choice I would rather live in democratic systems - but to say that everyone wants democratic systems tell you are not as exposed as you think you are ...
Nope, just old. Like 40s old. I'm afraid I'm someone who thinks that either people want democracy, or just haven't been anywhere near an actual undemocratic system, haven't seen what happens.
In reality, either people are (a minimum level of) liberal (voluntarily or "because otherwise I'd get arrested"), or they live under constant violence. Those are the only choices, and the way that works for people like tankies or some religious lunatics is that they've never seen up close what violence does on even a medium scale. In other words, the way it works for them, maybe for you, is that it's a fantasy. Not fundamentally different from what I'd loved to have done with Sarah Michelle Gellar, when even back then she was probably way too old for me.
And yeah your conspiracy theories about oil. THE war about liberty and freedom was Afghanistan. You might notice: no oil.
So yeah I'm one of those really irritating old guys that tells you even you want freedom and democracy, even if you think differently. It's not the only thing you want, and of course you now think you want your little fantasy more, but that will disappear forever the first time a police officer shows you what happens when they decide to use violence against someone you know.
This is a bottom feeder mentality. Europe has enough bright people and resources to truly compete in the AI race. There is something wrong when the only selling point is that it's local.
We're regularly getting demonstrations that being at the frontier is no moat at all. "Run open weights locally" was literally on the HN front page a few days ago as a primary concern/competitor for the big US labs. So there's a big and meaningful gap between "not frontier" and "bottom feeder". There's just tons of applications where you don't need "the best" model, especially not tomorrow's best model.
Those things are not mutually exclusive. We can have open weight, yet close-to-frontier models. China has very strong models that can also run locally.
I wouldn't. I can think of at least 10 regulations (labor and environmental chef amongst them) that would render such a maker uncompetitive from the start. Chinese makers would eat its lunch.
Doesn't matter, we need to opt out of the rat race that consists of making the planet worse for the next generations in exchange of quick bucks for the oligarchs.
The same applies to labour conditions and quality standards. And don't bring up arguments like "it can't work" or "that hurt competitiveness": all workers around the world want the exact same thing, a livable planet for their children. The only ones to object are the global elite who one ends up defending when they make the current status quo sound inevitable.
> all workers around the world want the exact same thing, a livable planet for their children
Actually, they don't. First of all they want livable conditions for themselves. Ecologism is a hobby for people with full bellies.
I remember during communism which failed to even feed and clothe us - but The Party kept telling us that we were actually winning and we were so morally superior to those "corrupt capitalists". This is what happens when ideological justifications replace rational thought.
> Ecologism is a hobby for people with full bellies.
Almost the entire planet where this debate is raging, is the part of the planet where people have full bellies. On average, too full, even.
Ecologism is also a hobby for people that don't want their kids to starve, as people living in Mesopotamia around the time of Akkad, or people around the Sea of Aral starting in the 1980s have discovered.
> Almost the entire planet where this debate is raging, is the part of the planet where people have full bellies.
Exactly. And most of the pollution happens in the other part.
We’re crippling our economies and render our companies uncompetitive to merely export environmental effects around, while pushing our voter base to the extremes.
All for naught too, because when the orc armies come (attracted by our weakness to even defend ourselves) they will do the the environment what they did in Ukraine…
> And most of the pollution happens in the other part.
Where does it happen exactly?
> All for naught too, because when the orc armies come (attracted by our weakness to even defend ourselves) they will do the the environment what they did in Ukraine…
Who are the orcs here? Russia? Europe is rearming like it's the 1980s.
How much money are Europeans paying to US companies for AI services and how much are Americans paying to EU companies? I don't think it's a good idea to let billions of dollars go to other countries because you couldn't be bothered to build a competitive product.
Why hasn't Baidu taken over the world's search market?
What moat does Google have that's so special in search?
You need to spend hundreds of billions of dollars to scale globally in search. And you have to take the market away from Google at the same time. Good luck.
OpenAI and Anthropic are doing that right now in LLMs, hyper scaling to a billion people. Having a billion users you can actually serve is a moat.
When OpenAI is done attaching a full ad model to GPT, they'll be able to serve a billion users profitably globally with zero subscriptions. This is what keeps Google up at night: ~$500 billion in ad revenue up for grabs circa 2030.
Any service with hundreds of millions of users is a lucrative ad business. TikTok, YouTube, Instagram, Facebook, Google, Amazon.
OpenAI's moat, if they get there soon enough, is that they'll be able to lean into a high margin ad business to press down on the market and kill everybody. You won't be able to serve the infrastructure to compete at the cost structure they'll able to subsidize at via the high margin ad business. This is an extraordinarily straight-forward move, and it's comically anti-competitive, and nobody will stop them from doing it. Their mistake is that they're at least two years behind where they should be in advertising.
Anthropic is the one at the most risk from the Chinese models, they have the weaker everyday consumer side. Enterprise won't be supported by an ad model very well.
To compete, we’d have to throw a lot of the regulations and laws into the trash (especially anything regarding copyright) and do an order of magnitude more investment.
I’m surprised that there’s no domestic chip production either, we don’t have our own CPUs or GPUs, meanwhile China is spinning up manufacturing so they don’t have to work around the Nvidia export restrictions as much.
I like Mistral and there’s cool stuff going on like how EuroLLM models know Latvian language and all the other EU ones better than way bigger models, but we don’t have anything frontier.
At the very least, they should be distilling Kimi K3 and GLM 5.3 as much as possible and working on MoE models like ~35B and ~120B versions to match Qwen.
> I’m surprised that there’s no domestic chip production either, we don’t have our own CPUs or GPUs, meanwhile China is spinning up manufacturing so they don’t have to work around the Nvidia export restrictions as much.
I haven't heard of updates on ESMC in a while, but that has been a thing on the horizon for a while, with production planned to start in fall 2027 last I read.
> I’m surprised that there’s no domestic chip production either, we don’t have our own CPUs or GPUs, meanwhile China is spinning up manufacturing so they don’t have to work around the Nvidia export restrictions as much.
The CCP is able to plan the economy. We used to be able to do that in the 60s-70s, at least in France, but that mindset is long gone.
In the long run a planned economy is a huge disadvantage, the Soviets and now the the CCP are prime examples. For instance, the CCP is also sitting on a bunch of practically empty metropolises, paired with a property bubble and a falling population (all of which are exist to their meddling).
"Planning" has a very different meaning when referring to what France did, what the USSR did, and what China is doing. Remains to be seen if China is at a disadvantage.
I mean the EU has ASML, why can’t we go the rest of the way? Seems kinda shortsighted and I say that as someone who bought the Intel Arc A580 on release and don’t regret it.
No, Europe doesn't have the capital markets required to build the required data centers. Underwriting gigawatt data centers and frontier models demands a fully realized European Capital Markets Union, a single energy regulator with cross-border grid integration, and shared fiscal borrowing power.
> Underwriting gigawatt data centers and frontier models demands a fully realized European Capital Markets Union, a single energy regulator with cross-border grid integration, and shared fiscal borrowing power.
Care to elaborate why you consider this an absolute must? I don't see why, say, France can't build an "AI valley" somewhere near to on of their nuclear power plants and commit to provide X amount of energy for some fixed price for a decade. To me the main reasons for not building top AI labs with frontier models in EU seems to be:
- Venture money is not there (or people controlling the money are not on board to invest into such labs).
- Weak political will from countries who could do that to actually go for it.
- Top scientist who could work on it choosing to go for much higher payouts in the US.
Power is the easy part; the hard part is where the check comes from.
One single frontier computing cluster costs some €30-40B initially and becomes virtually worth zero every four years. That’s the equivalent of France’s full-year VC market for all industries combined times three. A single Member State, already indebted far beyond sustainable limits, cannot subsidize expensive, long-term 40bn€ hardware risks from its national budget more than 33 times without both falling foul of state aid rules and blowing up the debt crisis even more seriously than it already is.
The three factors you mention, missing VCs, political reluctance, and brain drain, are by-products of exactly these fragmented fiscal structures. European capital is not missing; it is frozen at 33T€ (roughly) in risk-averse national bank deposits and Member State pension regulations instead of in one unified Capital Markets Union to pay for high risks related to enormous tech bets. Similarly, leading researchers are not looking to work on a better salary but for liquid equity, which is not available at the high valuations seen in the NASDAQ through deeply integrated public markets.
With insufficient scale for an entire continent in finance, even the best AI valley in a single country will hit the wall much faster than a US hyper-scaler investing greater amounts in CAPEX in one single quarter than every member state treasury.
We're still deep in the research and exploration phase, who knows what will huge oodles of compute be useful for in the end? Maybe it will end up being used by the owning corporation or local nation state, against similar amount of compute wielded by another corporation or nation state.
After all, what else than governments and corporations will be left after the rest of us poors will inevitably be priced out of the economy?
We're not in a good position on the supply chain needed. Right now it really means pouring billions on American corps, either by renting the compute or by building it. That would be mostly fine if it were only VC money but it's not.
> We're not in a good position on the supply chain needed. Right now it really means pouring billions on American corps, either by renting the compute or by building it.
You are saying "right now" as if EU is seriously investing into developing its own AI hardware (both training and inference) but I'm not aware of any serious attempts to build such supply chain in EU.
Mushroom pickers generally know only a handful of easy to distinguish types and only pick those. I would say that 99.9% of pickers will never get poisoned.
> I disagree with this. Decoders were absolutely dominant in 2020 for chat. GPT2 was considered too dangerous to release, and I remember scrambling to get on the GPT3 waitlist. It worked.
He's not talking about decoders, he's talking about auto-regression. Before ChatGPT, the dominant paradigm was fine-tuning BERT-like models.
> Before ChatGPT there really wasn’t much of a concept of pre-training and post-training.
Again, people spend years just post-training BERTs in various ways.
It’s both. Fine-tuning a BERT model changes its weights, which causes the embeddings to change.
For example you might have one model which embeds a text query and another model which embeds an image. You also have a dataset of image + text captions. Training means updating the weights of those models so that the embedding of the image is close to the embedding of its corresponding caption.
There is no theory why the gap shouldn't be closed. At the same time, people are trying to make them work for years by now and it's never good enough. But they are competing with incredibly optimized architectures.
To me, all these agent systems just look very stochastic. You have these agents that have some basic computer capabilities and they are producing semi-random actions that also affect the semi-random actions of other agents. It is funny to observe how this stochastic system works, but it does not seem very practical to me so far.
The recent OAI-HF hack seems very similar. You have bunch of random actors and eventually they by chance iterated to a series of actions that breached HF environment. I don't perceive this as a malignant artificial intelligence, I perceive this as dangerous stochastic system that can control buttons that can affect the outside world.
Which part specifically are you objecting to? I would not be surprised, for example, if the reasoning traces actually included "this other agent is impeding my work, I should minimize its effect on my work. I can demote it's privileges to accomplish this". All of that can be true without it understanding the concept of sabotage. You can sabotage without knowing the concept. It almost certainly "knows" the concept btw, but I'm just illustrating that specific part of the description of the behavior is completely plausible without it taking on an "aggressive" persona. Is your objection that they kind of assigned a persona?
I guess I don't understand what you're explaining better by saying it's stochastic. At the same resolution, humans are also stochastic.
I dislike anthropomorphizing as much as the next guy, but the description here seems pretty good to me.
Yes, but that is the point we don't anthropomorphize stochastic outputs even when they are dangerous.
Obligatory quote:
"Do not fall into the trap of anthropomorphizing Larry Ellison. You need to think of Larry Ellison the way you think of a lawnmower. You don't anthropomorphize your lawnmower, the lawnmower just mows the lawn, you stick your hand in there and it'll chop it off, the end. You don't think 'oh, the lawnmower hates me' -- lawnmower doesn't give a shit about you, lawnmower can't hate you. Don't anthropomorphize the lawnmower. Don't fall into that trap about Oracle." — Brian Cantrill
The transformation from Roman republic to Roman empire was mainly about how the top-level government is organized. It was not about the relations with vassals/periphery. The Roman republic was as imperialistic as empire in this regard. In fact, most of the Roman conquering was already done when the republic ended.
Yes — the Roman Empire was in a period of retreat and reformation when it formed.
That’s my point:
The US has neither entered an empire nor the decline post-empire because it gained territorial holdings over the past century — that’s consistent with late stage republics as well.
I don't know the details of their implementation, but in general, if the text is not watermarked, you will probably fail the test after a few tokens. If it is, you will have to run the entire text, or, you can just run it until you have some degree of confidence.
And when you are an investor, you must predict who is able to make a good game. What is the likelihood that a Myst-like will pay for itself. There is already a lot of Myst-likes on Steam, do players want more?
The whole "visual novel" genre would say "Yes, we'd like more". They range from all sorts of gameplay though, from Myst-like to RPGs, and a lot in-between. Definitely a market for Myst-like games, the trick is to figure out so production costs don't overrun eventual profits of course, as always.
reply