Greg Clarke

I’m sorry if I’m coming across as condescending, that’s not my intent. It’s never been as simple as just throwing more days and CPU at the problem. There were algorithmic challenges for every LLM evolution. There are still lots of potential improvements using the existing training data. But even if there wasn’t, we’ll still see loads of improvements in chat bots because of other techniques.

People that don’t understand those terms are using them interchangeably

Yes of course I’m asserting that. While the performance of LLMs may be plateauing, the cost, context window, and efficiency is still getting much better. When you chat with a modern chat bot it’s not just sending your input to an LLM like the first public version of ChatGPT. Nowadays a single chat bot response may require many LLM requests along with other techniques to mitigate the deficiencies of LLMs. Just ask the free version of ChatGPT a question that requires some calculation and you’ll have a better understanding of what’s going on and the direction of the industry.

OpenAI, Google, Anthropic admit they can’t scale up their chatbots any further

Lol, no they didn’t. The quotes this articles are using are talking about LLMs not chatbots. This is yet another stupid article from someone who doesn’t understand the technology. There is a lot of legitimate criticism for the way this technology is being implemented but FFS get the basics right at least.

To be fair, some of my weekends have been a mistake.

I had a really good friend on MySpace that I lost touch with. I think he was a little paranoid, we didn’t speak much and he was always looking over his shoulder. His name was Tom.

That is a very VC baiting title. But it’s doesn’t appear from the abstract that they’re claiming that LLMs will develop to the complexity of AGI.

Do you have a non paywalled link? And is that quote in relation to LLMs specifically or AI generally?

largely based on the notion that LLMs will, with continued scaling, become artificial general intelligence

Who said that LLMs were going to become AGI? LLMs as part of an AGI system makes sense but not LLMs alone becoming AGI. Only articles and blog posts from people who didn’t understand the technology were making those claims. Which helped feed the hype.

I 100% agree that we’re going to see an AI market correction. It’s going to take a lot of hard human work to achieve the real value of LLMs. The hype is distracting from the real valuable and interesting work.

Hosting local has become even more important with the vulnerability of submarine cables.