| Grizlas | RE: AI discussion |
|
|  |
| | General |  |
| | Group: Administrator,
Klikan,
Regulars,
Outsiders
| | Location: Denmark | | Joined: 08.06.06 | | Posted on 13-09-2024 08:15 |
|
Openai just dropped their new model o1 (Strawberry) which is supposed to offer what is referred to as System 2 thinking.
This is exactly what the critics (makers of Blocksworld) say that LLMs lack, so it will be interesting to see the Blocksworld benchmarks for this model.
I've done some experiments with it, and I can't say that I am that impressed yet.
The most impressive thing is this ability to see the model's chain of thought. It definitely looks like some sort of thought process, albeit a bit alien
|
You want to tempt the wrath of the whatever from high atop the thing? |
Edited by Grizlas on 13-09-2024 08:21 |
|
| Grizlas | RE: AI discussion |
|
| |
| | General | |
| | Group: Administrator,
Klikan,
Regulars,
Outsiders
| | Location: Denmark | | Joined: 08.06.06 | | Posted on 27-09-2024 19:32 |
|
A new Planbench paper is out: LLMS STILL CAN’T PLAN%3B CAN LRMS?
A PRELIMINARY EVALUATION OF OPENAI’S O1 ON PLANBENCH
And here are the Planbench results:
o1 almost aces regular Blocksworld with 97.8% accuracy, compared to the previous best model 62.6%. On Mystery Blockworld it does substantially better than previous models - 52.8% compared to the best model 4.3%.
Goalposts are then promptly moved, by increasing problem steps and further randomizing of strings. The results on these new benchmarks show that o1 still can't plan all that well. Here's their conclusion:
|
You want to tempt the wrath of the whatever from high atop the thing? |
Edited by Grizlas on 27-09-2024 19:33 |
|
|
| Grizlas | RE: AI discussion |
|
| |
| | General | |
| | Group: Administrator,
Klikan,
Regulars,
Outsiders
| | Location: Denmark | | Joined: 08.06.06 | | Posted on 09-10-2024 21:35 |
|
John Hopfield and Geoffrey Hinton win the 2024 Nobel Prize in Physics for their pioneering work on neural networks. Also, David Baker, Demis Hassabis and John Jumper win the 2024 Nobel Prize in Chemistry for developing AlphaFold (Google Deepmind).
Geoffrey Hinton, a complete savage, almost immediately uses this oppurtunity to take a swing at Sam Altman 
https://fortune.com/2024/10/09/openai-sam-altman-geoffrey-hinton-nobel-prize-physics-ilya-sutskever-toronto/
|
You want to tempt the wrath of the whatever from high atop the thing? |
Edited by Grizlas on 09-10-2024 21:36 |
|
|
| Norlander | RE: AI discussion |
|
|  |
| | Field Marshal | |
| | Group: Administrator,
Klikan,
Regulars,
Outsiders
| | Location: Copenhagen | | Joined: 09.06.06 | | Posted on 10-10-2024 06:48 |
|
|
The conventional view serves to protect us from the painful job of thinking.
- John Kenneth Galbraith |
|
|
|
|
| Norlander | RE: AI discussion |
|
| |
| | Field Marshal | |
| | Group: Administrator,
Klikan,
Regulars,
Outsiders
| | Location: Copenhagen | | Joined: 09.06.06 | | Posted on 12-10-2024 09:11 |
|
|
The conventional view serves to protect us from the painful job of thinking.
- John Kenneth Galbraith |
|
|
|
|
