Maria Antoniak's avatar

Maria Antoniak

@mariaa.bsky.social

"LLMs are just next-word predictors." What are the current best references to respond to this, either supporting or critiquing?

6 replies 2 reposts 6 likes


Pekka Lund's avatar Pekka Lund @pekka.bsky.social
[ View ]

I don't know about exact references but I think the best response goes along these lines. It's generally the same. People assume all sorts of fundamental differences between us and AIs mostly due to what Geoffrey Hinton said: "problem is most people have a hopelessly wrong view of how people work"

0 replies 0 reposts 0 likes


Jonathan Cheng's avatar Jonathan Cheng @jonathancheng.bsky.social
[ View ]

arxiv.org/pdf/2406.11741

Idk if best/renowned, but concepts like “transcendence” give a good language to critique this imo

0 replies 0 reposts 2 likes


Scott McGrath's avatar Scott McGrath @smcgrath.bsky.social
[ View ]

Something that falls into the same ballpark is another phrase I've heard, "LLM's can't think". I'm trying to figure out the best approach on that one too. It feels like people approaching it with the wrong framework in mind.

0 replies 0 reposts 1 likes


Joi Ito's Jibo sez bluesky migration is for the birds 's avatar Joi Ito's Jibo sez bluesky migration is for the birds @joi-itos-jibo.bsky.social
[ View ]

Most people saying this are not interested in the technical distinction, tbh.

1 replies 0 reposts 1 likes


Robert Zubek's avatar Robert Zubek @rzubek.bsky.social
[ View ]

In support of this phrase taken literally, any textbook treatment of LLMs will work, e.g. Understanding Deep Learning from MIT Press shows next-word prediction very well. But I also fear that arguments around this particular phrasing are doomed to context collapse and misunderstanding, because...

1 replies 0 reposts 0 likes


Ted Underwood 🦋's avatar Ted Underwood 🦋 @tedunderwood.me
[ View ]

I think this may be one of the best contemporary arguments in support of the idea that there's a meaningful distinction to be made between next-word prediction and other cognitive (even linguistic) tasks: arxiv.org/abs/2301.06627

1 replies 1 reposts 10 likes