Computerphile
Bug Byte puzzle here – https://bit.ly/4bnlcb9 – and apply to Jane Street programs here – https://bit.ly/3JdtFBZ (episode sponsor). More info in full description below ↓↓↓
A new paper suggests diminishing returns from larger and larger generative AI models. Dr Mike Pound discusses.
The Paper (No “Zero-Shot” Without Exponential Data): https://arxiv.org/abs/2404.04125
https://www.facebook.com/computerphile
https://twitter.com/computer_phile
This video was filmed and edited by Sean Riley.
Computer Science at the University of Nottingham: https://bit.ly/nottscomputer
Computerphile is a sister project to Brady Haran’s Numberphile. More at https://www.bradyharanblog.com
Thank you to Jane Street for their support of this channel. Learn more: https://www.janestreet.com
Source
Bug Byte puzzle here – https://bit.ly/4bnlcb9 – and apply to Jane Street programs here – https://bit.ly/3JdtFBZ (episode sponsor)
Aged like milk.
Perhaps just my experience but among the LLMs I use (mainly OpenAI) their abilities in mathematics have increased enormously in last 6 months. (GPT-o1, then o3-mini.)
Narrator: Generative AI had not in fact peaked.
The issue is assuming either way where the curve flattens.
We are at the plateau for sure
the models today are much better than last year, and that's before the incoming GPT5 release
The objective isn't to create intelligence, the objective is to extract wealth at the least expense. They don't need to consume more real data, they can just feed on their own slop because slop will make money even though it's garbage and horrible for society.
They don't want an all-powerful AI that can solve the world's problems, they want a sufficiently effective AI to solve their problem of not having all the money.
This video aged like good wine.
We can see the plateau in all AI LLMs out there…
AI hype is declining…
The AI bubble is about to p o p
If a true general AI is reaching the moon, then current AI has climbed a tree. Yes, the top of a tree is closer to the moon than the ground, but climbing a tree is not the path you want to take to actually reach the moon.
Yes.
guys its peaked
ai gets better
guys its peaked
ai gets better
guys its peaked
ai gets better
guys its peaked
ai gets better
guys its peaked
ai gets better
guys its peaked
ai gets better
guys its peaked
ai gets better
guys its peaked
ai gets guys its peaked
ai gets better
guys its peaked
ai gets better
guys its peaked
ai gets better
guys its peaked
ai gets better
guys its peaked
ai gets better
guys its peaked
ai gets better
guys its peaked
ai gets better
guys its peaked
ai gets better
guys its peaked
ai gets better
guys its peaked
ai gets better
guys its peaked
ai gets better
guys its peaked
ai gets better
guys its peaked
ai gets better
guys its peaked
ai gets better
guys its peaked
ai gets better
ai gets better
guys its peaked
ai gets better
guys its peaked
ai gets better
guys its peaked
ai gets better
guys its peaked
ai gets better
guys its peaked
ai gets better
guys its peaked
ai gets better
guys its peaked
ai gets better
Ha, this year old video did not age well
Reasoning models and GPT-5 alteafy proved you wrong. AGI 2 years away. See you then
Dang, they probably trained a generative AI on the picture of a Trabant, that was labeled as an actual car.
I see this video aged well.
Well I guess we have our answer
11:31 "Will we see ChatGPT 7 or 8 or 9 be roughly the same as ChatGPT4?" , well, this bit is even funnier after the GPT5 flop release.
Do you even understand what a logarithm is? It's not a plateau
A big part of the argument here is that AI may be able to recognize and deal with categories or classes of information, but not with individual "instances" within the categories. Normally I would find this to be a convincing argument, however I've experienced otherwise. In my case I took a screen capture of a recent picture of a relatively small waterfall in a remote, wooded area of Tennessee, and asked AI if it could identify the waterfall. I used a screen capture because I knew that resulting image would contain no metadata, such as location information, that could influence my test. The AI site not only identified the waterfall and provided me it's name, it also provided me driving directions to pay it a visit. When I looked at the reasoning output I learned that it had used the rock type and foliage to determine the geographic area, and the number of levels in the falls to identify the specific waterfall.
I believe that modern AI services consist of the well-known LLM algorithms accompanied by the service's own "special sauce", and the special sauce may contribute half the functionality. At the end of the day, AI is performing valuable and $$$-saving tasks countless times each day, regardless of the technology's current limitations. An entirely separate question is whether the value it contributes justifies the valuations of the stocks. That's a question for investors and speculators to address amongst themselves.
Here after GPT 5 Tanked
One year on and I think this video is as relevant as ever. The dynamic discussed does appear to be happening, but maybe a little later than expected at the time!!
It will be great to do a follow up of this segment, 1 yr round up to see how things shaped up, esp with advancement of agentic AI
So one year later … clearly not … look at video genai
I wonder if Meta will train it’s AI on videos taken from all their glasses
chatgpt 5 is already worse than 4 or atleast not much smarter. Its giving more false answers than ever and i doubt chatgpt 6 is gonna bringt anything new to the table.
6:22 I'm sure this red curve is ABSOLUTELY exactly the exponential. How did he manage to do that
Edan Meyer has a video about how we need more efficient AI models rather than more data.
In the early 00's, I started as an engineer on a large news aggregator, as we moved from English to multiple languages. As I see it, this is similar to a problem we had. When you have 10,000 examples of an article on an event, you don't have to pick the best article, you just have to discard the worst choices and the remainder will be adequate to get the point across. There are a number of options for defining "worst choice", here, including simply estimating how related they are to each other. But if you have only six examples of an article on an event, then it is hard to rank them, so there is a much greater chance that you will make a bad selection. In this case, the 10,000 was the English-language corpus, while the 6 was maybe Hebrew or something like that.
While I do think this leads to diminishing returns, it may be a VERY long tail, because we can use feedback to gradually change the metrics system to expose more features. Some of that can be done in an automated way, but that may be cost prohibitive because there may be more useless systems of metrics than useful systems of metrics (this isn't limited to LLMs, people derive useless results all the time because they used the wrong assumptions).
This aged well eh! hha
Yes, gen AI is a dead idea. It's a parlor trick. Smoke and mirrors. Something to get the investors excited about and to use as the backdrop for a multi-trillion dollar fraud scheme.
Can’t wait til this stupid bubble pops so we can move on
by each adding datapoint you make the distance to destination half. youll never arrive. half os arbitary here but the analogy stands
The weather services have compute power probably millions of times more powerful than when I was a child. However; weather forecasts don't go a million times longer into the future. Actually, we're in the same order of magnitude. Are they more accurate? Probably yes, can predict more closely when showers arrive. But a million times more accurate? Nah!
Can we get a revisit on this? I’d love to hear how real progress over the past year lines up to the predictions in the paper. And I love any excuse to listen to Mike!