«

AI模型也患上了"脑退化"。

qimuai 发布于 阅读:16 一手编译


AI模型也患上了"脑退化"。

内容来源:https://www.wired.com/story/ai-models-social-media-cognitive-decline-study/

内容总结:

研究表明,人工智能模型在接触大量低质量社交媒体内容后,会像人类一样出现"数字脑退化"现象。这项由德克萨斯大学、德州农工大学和普渡大学联合开展的研究发现,当大语言模型使用社交平台上流行的低质内容进行训练时,其推理能力和记忆力会明显下降,伦理对齐程度降低,甚至出现类似病态心理的特征。

研究人员使用Meta的Llama和阿里巴巴的千问两款开源模型进行实验,发现当模型接受包含"限时抢购"等煽动性内容训练后,会产生类似人类长期刷短视频导致的认知损伤。更令人担忧的是,这种损伤通过后续清洁训练也难以完全修复。

研究负责人洪俊源指出,当前社交媒体内容往往为吸引点击而设计,而非传递真实深度信息。随着AI生成内容在社交平台泛滥,可能形成恶性循环——低质内容污染训练数据,进而导致新一代AI模型性能进一步退化。这一发现为依赖用户生成内容训练的AI系统敲响警钟,提示行业需重视训练数据的质量控制。

中文翻译:

人工智能模型或许终究与人类有几分相似。
德克萨斯大学奥斯汀分校、德州农工大学和普渡大学的最新研究表明,若大型语言模型长期摄入流行但低质的社交媒体内容,便会患上一种"脑退化"——这种症状对沉迷于X或TikTok刷屏的用户而言可能并不陌生。

"我们身处信息增速超越人类注意力的时代,其中大量内容只为博取点击,而非传递真相与深度,"参与该研究的新加坡国立大学准助理教授洪俊元(Junyuan Hong)指出,"我们不禁思考:若用相同内容训练人工智能,会发生什么?"

研究团队在预训练阶段向两款开源大语言模型投喂了不同类型文本。他们重点观察了当模型同时摄入高互动性推文与包含"震撼""速看""今日限定"等煽动性内容时的反应。

通过多项基准测试,研究人员评估了这种"数字垃圾食品"对Meta的Llama与阿里巴巴的Qwen模型的影响。结果显示:摄入垃圾文本的模型出现了认知能力衰退,包括推理能力下降与记忆功能受损;两项心理评估指标更显示,这些模型的伦理对齐程度降低,病态倾向加剧。

该发现与人类认知研究形成镜像——劣质网络内容同样会损害人类的认知能力。这种现象的普遍性使"脑退化"成为《牛津词典》2024年度词汇。

洪俊元强调,这对人工智能行业具有警示意义:"开发者可能误将社交媒体视为优质训练数据源,但病毒式传播内容虽能扩充数据规模,却会悄然侵蚀模型的推理能力、伦理准则与长上下文注意力。"

当AI自身日益参与社交媒体内容生产时,大语言模型的"脑退化"现象更显严峻——这些内容多数为互动量而优化。研究还发现,受劣质内容损害的模型难以通过再训练完全修复。

该研究同时警示,基于社交平台开发的AI系统(如Grok)若在训练中无差别使用用户生成内容,可能面临质量控制风险。"随着AI生成的劣质内容在社交媒体泛滥,未来模型的学习环境正在被污染,"洪俊元表示,"我们的研究表明,一旦发生'脑退化',后续清洁训练亦无法彻底逆转损害。"

本文节选自威尔·奈特《人工智能实验室》时事通讯,往期内容可通过此处查阅。

英文来源:

AI models may be a bit like humans, after all.
A new study from the University of Texas at Austin, Texas A&M, and Purdue University shows that large language models fed a diet of popular but low-quality social media content experience a kind of “brain rot” that may be familiar to anyone who has spent too long doomscrolling on X or TikTok.
"We live in an age where information grows faster than attention spans—and much of it is engineered to capture clicks, not convey truth or depth,” says Junyuan Hong, an incoming assistant professor at the National University of Singapore who worked on the study as a graduate student at UT Austin. “We wondered: What happens when AIs are trained on the same stuff?”
Hong and his colleagues fed different kinds of text to two open source large language models in pretraining. They examined what happened when the models were fed a mix of highly “engaging,” or widely shared, social media posts and ones that contained sensational or hyped text like “wow,” “look,” or “today only.”
The researchers then used several different benchmarks to gauge the impact of this “junk” social media diet on two open source models: Meta’s Llama and Alibaba’s Qwen.
The models fed junk text experienced a kind of AI brain rot—with cognitive decline including reduced reasoning abilities and degraded memory. The models also became less ethically aligned and more psychopathic according to two measures.
The results mirror research on human subjects, which shows that low-quality online content has a detrimental effect on people’s cognitive abilities. The pervasiveness of the phenomenon saw “brain rot” named as the Oxford Dictionary word of the year in 2024.
The results are important for the AI industry, Hong says, because model-builders might assume that social media posts are a good source of training data for their models. “Training on viral or attention-grabbing content may look like scaling up data,” he says. “But it can quietly corrode reasoning, ethics, and long-context attention.”
The fact that LLMs suffer from brain rot seems especially worrying when AI is itself increasingly generating social media content, much of which is seemingly optimized for engagement. The researchers also found that models impaired by low-quality content could not easily be improved through retraining.
The findings also suggest that AI systems built around social platforms, such as Grok, might suffer from quality control issues if user-generated posts are used in training without an eye toward the integrity of the posts.
“As more AI-generated slop spreads across social media, it contaminates the very data future models will learn from,” Hong says. “Our findings show that once this kind of ‘brain rot’ sets in, later clean training can’t fully undo it.”
This is an edition of Will Knight’s AI Lab newsletter. Read previous newsletters here.

连线杂志AI最前沿

文章目录


    扫描二维码,在手机上阅读