One of my favorite things to do, paraphrased:
Me (to "AI"): Recount the plot to [one of my books] and name five major characters and their roles.
AI: [Generates bad plot, correctly names only the protagonist, all other characters and their roles made up]
Me: That's wrong, you know.
AI: I know.
one very clarifying thing to realize is that AI (really LLMs) do not give you an answer to your questions. they can only give you words in the shape of an answer.
Oh, wow. This is a hoot. It's the confidence with which it is so very wrong that's impressive. Honestly though, the danger here with my novels is that there is not a single word about them being erotica. And... Oh dear. It says one of my novels is "a children's fantasy novel." Uh yikes. No.
Oh, God, I just did this with one of my own novels and the results are hilarious. The novel is on-line and free! It got, like, 30% of the plot. It actually identified the theme accurately, but completely hallucinated the three central protagonists!
Back in the IRC days of NaNoWriMo, the guy who ran it had a chatbot program named Timmy
Timmy pulled random words or phrases from the chatters and would chat with us and also write his own 50k novels
Everything he said was utter random nonsense that would randomly make sense
TIMMY was a proto-AI
Once asked it to tell me the ‘scary’ parts of some Pixar movies. There were scenes in Cars that I very much missed…because they were fully hallucinated. 😂
Future Elon Mars Mission:
Elon: "AI, is the air outside breathable?"
Ai: "Yes, the air outside is breathable!"
Elon: *Opens the hatch"... End of mission.
So, is that a better or worse success/accuracy rate than your average human reader? As I've pointed out to my hot-to-use-AI boss, you'll end up wasting *at least* as much time training and verifying the output as you will with your more problematic employees.