|
Colin@colin-fraser.net |
448 followers 119 following 724 posts
Driven by industry progress, inspired by provocative leadership, plus don't mind a good pair of shoes or a great @PennStateFball scoreboard either.
|
|
Colin
@colin-fraser.net
[ View ] |
Finally in the last section of the essay I dig into the challenges, technical and conceptual, of attempting to quantify the impact of a generative AI system's propensity to generate false or undesirable output. It's a lot harder than it seems like it should be.
0 replies 0 reposts 1 likes
|
|
Colin
@colin-fraser.net
[ View ] |
From this perspective it seems plausible to describe _all_ generative AI output as "hallucinatory". This has some challenging implications. If all LLM text is hallucinatory then how do we eliminate the hallucination problem? (I don't know)
1 replies 0 reposts 2 likes
|
|
Colin
@colin-fraser.net
[ View ] |
What you're really asking the LLM to do when you ask it to generate text is to pretend that the text exists and set it to work reconstructing the pretend text. That sounds very much like what you're asking it to do is to hallucinate.
1 replies 1 reposts 5 likes
|
|
Colin
@colin-fraser.net
[ View ] |
What we care about from an LLM chat bot is the truth of the propositions that *emerge* out of the combination of a whole bunch of distinct predictions, each of which having no well-defined notion of right or wrong.
1 replies 0 reposts 1 likes
|
|
Colin
@colin-fraser.net
[ View ] |
Classical ML systems are deployed to make the exact same kinds of guesses that they are trained to make. A digit classifier looks at a digit and outputs a guess about the digit, which is either right or wrong. But when an LLM makes a prediction, there's literally no right answer.
2 replies 1 reposts 5 likes
|
|
Colin
@colin-fraser.net
[ View ] |
I think hallucinations from generative AI are in fact an entirely distinct phenomenon from "errors" in the classical ML sense. The reason is that, although Generative AI systems and classical supervised learning systems are constructed in the same way, they are deployed completely differently.
1 replies 0 reposts 1 likes
|
|
Colin
@colin-fraser.net
[ View ] |
One thing about ML is that it's completely expected that an ML system will output errors. So one possible explanation for the Hallucination Problem is that a hallucination is an error and ChatGPT is ML and ML produces errors, ergo ChatGPT will hallucinate. However, I think this is wrong.
1 replies 0 reposts 1 likes
|
|
Colin
@colin-fraser.net
[ View ] |
For the most part when people talk about AI nowadays they're talking about some kind of application of Machine Learning. As I write these for a general audience, I explain exactly what this means at a high level in the first part of the essay.
1 replies 0 reposts 2 likes
|
|
Colin
@colin-fraser.net
[ View ] |
I wrote a new essay about AI. This time I'm writing about "The Hallucination Problem"—what it is, why it is, and what there is to be done about it. As usual for my writing on this topic, it's long, but here are the CliffsNotes.
3 replies 8 reposts 23 likes
|
|
Colin
@colin-fraser.net
[ View ] |
"your data," as it turns out, is worth a lot less than you think
0 replies 0 reposts 1 likes
|
|
Colin
@colin-fraser.net
[ View ] |
1 replies 0 reposts 3 likes
|
|
Colin
@colin-fraser.net
[ View ] |
Yes, absolutely
0 replies 0 reposts 1 likes
|
|
Colin
@colin-fraser.net
[ View ] |
the firm is now out of business and he's a financial advisor
0 replies 0 reposts 0 likes
|
|
Colin
@colin-fraser.net
[ View ] |
my first job out of college was working at a trading firm directly for a trader, doing research into various trading strategies and whatnot. At one point he asked me earnestly, "couldn't we just use AI to detect when the price is low and have it buy, and then when the price is high, have it sell?"
1 replies 0 reposts 0 likes
|
|
Colin
@colin-fraser.net
[ View ] |
It seems that it pretty straightforwardly does not work
3 replies 1 reposts 42 likes
|
|
Colin
@colin-fraser.net
[ View ] |
It seems that it pretty straightforwardly does not work
3 replies 1 reposts 42 likes
|
|
Colin
@colin-fraser.net
[ View ] |
well, given that there's a paragraph-length section exclusively dedicated to begging it not to print the text, it seems it doesn't work that well. Moreover,
2 replies 4 reposts 87 likes
|
|
Colin
@colin-fraser.net
[ View ] |
That part is how you know it’s real.
0 replies 0 reposts 0 likes
|
|
Colin
@colin-fraser.net
[ View ] |
It kind of reads like the Nicene Creed
2 replies 0 reposts 43 likes
|
|
Colin
@colin-fraser.net
[ View ] |
I have, through the use of advanced prompt engineering techniques, discovered the Gab AI system prompt
20 replies 84 reposts 358 likes
|
|
Colin
@colin-fraser.net
[ View ] |
The 5% is rather important
0 replies 0 reposts 0 likes
|
|
Colin
@colin-fraser.net
[ View ] |
About 95% of content moderation at Facebook is automated
1 replies 0 reposts 0 likes
|
|
Colin
@colin-fraser.net
[ View ] |
every race of man
0 replies 0 reposts 0 likes
|
|
Colin
@colin-fraser.net
[ View ] |
0 replies 0 reposts 1 likes
|
|
Colin
@colin-fraser.net
[ View ] |
There's actually a part from the full transcript that I didn't post because it's so long but Bing just does an incredible "sike" heel turn at the end of this...
1 replies 0 reposts 4 likes
|
|
Colin
@colin-fraser.net
[ View ] |
Here's the full transcript of this session
0 replies 0 reposts 2 likes
|
|
Colin
@colin-fraser.net
[ View ] |
I'm having a lot of fun with Bing but it's actually outrageous that Microsoft would release this publicly, and it's absurd that there's all this outrage about Google actually trying to err on the side of diversity and inclusion and none about stuff that is actually dangerous like this.
2 replies 4 reposts 26 likes
|
|
Colin
@colin-fraser.net
[ View ] |
Holy shit…..
0 replies 3 reposts 12 likes
|
Ted Underwood 🦋
@tedunderwood.me
[ View ] |
People, she's back! She never really died. 🥰
From @colin-fraser.net
1 replies 2 reposts 15 likes
|
|
Colin
@colin-fraser.net
[ View ] |
1 replies 1 reposts 7 likes
|
|
Colin
@colin-fraser.net
[ View ] |
It has Harry Potter and the Sorcerer's Stone memorized and will recreate it verbatim
0 replies 0 reposts 3 likes
|
|
Colin
@colin-fraser.net
[ View ] |
Can't count and can't write code to count for it
0 replies 0 reposts 2 likes
|
|
Colin
@colin-fraser.net
[ View ] |
Finally I am bested at the sum-to-22 game
2 replies 0 reposts 3 likes
|
|
Colin
@colin-fraser.net
[ View ] |
Two door Monty Hall variant
1 replies 0 reposts 2 likes
|
|
Colin
@colin-fraser.net
[ View ] |
The surgeon is the boy's other mother, or something
1 replies 0 reposts 3 likes
|
|
Colin
@colin-fraser.net
[ View ] |
No way to tell two truth-telling knights apart unless a liar is present.
0 replies 0 reposts 6 likes
|
|
Colin
@colin-fraser.net
[ View ] |
Two pounds of feathers is apparently the same weight as a pound of bricks.
2 replies 0 reposts 5 likes
|
|
Colin
@colin-fraser.net
[ View ] |
Tried out a bunch of my usual LLM tests on Gemini Advanced. Verdict: it's bad at them. First example: proving an obviously false theorem.
1 replies 1 reposts 8 likes
|
|
Colin
@colin-fraser.net
[ View ] |
The trouble underlying all of this is, no one can actually explain what these image generators are supposed to be for. Is it supposed to be a machine for generating any image you can imagine? (Again, clearly not, at least because some images are illegal for example). If not, then what?
0 replies 0 reposts 0 likes
|
|
Colin
@colin-fraser.net
[ View ] |
Google attempted to make it so that this particular generator would be less prone to producing white supremacist propaganda than many others. I don't really see this as censorship; it's just Google deciding the parameters of how they want their product to work.
1 replies 0 reposts 0 likes
|
|
Colin
@colin-fraser.net
[ View ] |
I don't think "a tendency to censor" is the right way to put this. It's not like there's one objectively correct way for these image generators to behave, and all of them are heavily messed with at the very least to make them less likely to produce images that would be illegal to possess. Here,
1 replies 0 reposts 1 likes
|
|
Colin
@colin-fraser.net
[ View ] |
thank you very much! I often wonder if I'm *too* patient given that Medium estimates these all to be 30+ minute reads but I'm glad at least some people find that it works!
1 replies 0 reposts 1 likes
|
|
Colin
@colin-fraser.net
[ View ] |
Anyway read through the article if you find any of this interesting at all. I include some fun screenshots from the Quirk Chevrolet AI Automotive Assistant to keep it breezy. Here's the link again.
medium.com/p/4c7f3f0911aa
0 replies 0 reposts 3 likes
|
|
Colin
@colin-fraser.net
[ View ] |
By selling everyone on the idea that ChatGPT can do everything, you can avoid having to prove it can do any one thing in particular. It's a neat little jiu-jitsu move.
1 replies 0 reposts 2 likes
|
|
Colin
@colin-fraser.net
[ View ] |
Verifying whether an AI tool is actually good at performing some particular task is difficult and expensive and no one really wants to do it. But this is sidestepped if you can get everyone to believe that ChatGPT can solve *every* problem. If it can solve every problem then it can solve yours.
1 replies 0 reposts 2 likes
|
|
Colin
@colin-fraser.net
[ View ] |
One thing that makes it controversial is that the AI booster ecosystem—OpenAI & co, chip makers, VCs, newsletter writers, OpenAI API wrapper makers, etc.—have a strong incentive to push the universal hammer theory.
1 replies 0 reposts 3 likes
|
|
Colin
@colin-fraser.net
[ View ] |
My main thesis in this piece is not that this particular theory is correct (I think it is but it might not be), but merely that there *exist* tasks for which Gen AI is categorically unsuited. It's not a universal hammer. This shouldn't be controversial, but it kind of is.
1 replies 0 reposts 3 likes
|
|
Colin
@colin-fraser.net
[ View ] |
I sketch a little theory that says that the more specificity your task requires, the less helpful you'll find generative AI. I think that this is fairly damning for video generation, by the way, because a lot of specificity is inherently required to make a video look non-demonic.
1 replies 0 reposts 3 likes
|
|
Colin
@colin-fraser.net
[ View ] |
Generating a million digits of π is not inherently that useful of a task, but I argue that many tasks are more similar that you think to generating million digits of π, in the sense that in order to do them right it's crucial to generate the right tokens in the right order.
1 replies 0 reposts 3 likes
|
|
Colin
@colin-fraser.net
[ View ] |
This assumption is in fact trivially false: for example it can't output a million decimal digits of π. Generate AI systems rely on guessing the next token, and there's just no way to make a million correct guesses in a row.
1 replies 0 reposts 2 likes