this is a very interesting article by ted chiang about the quality of LLM like chatgpt a.o. the LLM's "hallucinations" as inevitable effects of data compression
Think of ChatGPT as a blurry JPEG of all the text on the Web. It retains much of the information on the Web, in the same way that a JPEG retains much of the information of a higher-resolution image, but, if you’re looking for an exact sequence of bits, you won’t find it; all you will ever get is an approximation. But, because the approximation is presented in the form of grammatical text, which ChatGPT excels at creating, it’s usually acceptable. You’re still looking at a blurry JPEG, but the blurriness occurs in a way that doesn’t make the picture as a whole look less sharp.
worth reading!
via @
Cory Doctorow #
computer #
internet #
Doctorow #
Chiang #
LLM #
ChatGPT #
AI #
KI ChatGPT Is a Blurry JPEG of the WebOpenAI’s chatbot offers paraphrases, whereas Google offers quotes. Which do we prefer?