在欧盟AI法案透明度准则生效之后，在宣布大多数主要的AI实验室将很快对其文本输出进行水印以遵守之后，对于这将对AI模型生成的文本的质量产生多大影响，如果有的话，也有一些争论。
由于AI实验室的重点转向了代码制作，我认为模型的书面输出已经变得更糟了。 但了解水印是否会让情况更糟
我发现关于科技作家约翰·格鲁伯(John Gruber)最近的文章
Gruber写道：
为了在文字中嵌入隐藏的线索来暗示其来源
他得到了很多回击，有很多辩论你甚至如何测量质量，或定义它。
但有些人认为水印根本不影响质量，他写道：
也有一些方法调整算法，不会减少它的随机性。 例如，如果随机数生成器在生成数中例行颠倒位数的顺序，随机性将保持不变。
同样，在另一场讨论中，有人形容它基本上是在交换死亡的两面。
Gruber在后续文章中写道：
我希望这是事实。 我认为，我认为
我认为格鲁伯是对的。
各答复的质量损失
Anthropic提出大胆索赔：" 水标记不影响Claude产出的质量。 他们把论文与设计联系起来并以此为基础(Google于2024年在《自然》中出版的SynthID-Text)，向所有人保证，这种方法 " 保证了反应质量 "。
因此很容易假设，他们只是随意交换一些数字。
但他们与人类一起测试的无质量影响主张是基于单一的响应。 不是数百万人实际使用这些系统的方式：作为前后对话，随着时间的推移，通过多个提示和响应。
他们与底部的56页补充PDF提供了一些关于所用方法的更多信息，值得注意的是，称有一种“多样性/可探测性权衡”:
虽然单一序列不扭曲保证了每个单独反应的质量，但不一定保持多个反应的多样性
好吧，这是什么意思？他们继续解释：
当对同一速率的几个响应进行取样时，这些响应之间的类似性对于水标记的响应大于未水标记的响应。
现在，如果他们真的把一组随机数字换成另一组数字，我们就不会看到与没有水标记的响应相比有任何这样的区别。
他们继续写道：
在各种情况中，这可能会有问题，因为相互反应的多样性很重要，或者可能会降低产生许多反应的系统的总体质量，然后选择最佳反应系统。
因此，用他们自己的话说，它毕竟可以“降低整体质量”。 但是，没有什么可担心的，因为只有你关心“反应间多样性”，他们只提供了一个例子：一种产生和选择反应的系统。
但“反应间多样性”也影响到大多数人如何使用这些系统。 特别是要写创意的任务。
有人回应Gruber关于牺牲质量的观点，认为这个模型没有notion，他们肯定会同意你，个人
“我不喜欢这句话的结尾。 请提出一个新的建议：[判决]。” 不过你不喜欢。 没问题，按下重试按钮现在，你受到水印的影响了，因为你碰到了“反应间多样性”的问题：就像投出一个满载的死亡一样，你会比没有水印的模型看到更多的类似的反应。
Natzir Turrado，一名技术顾问，通过在纸的设置和水印密钥的开放式模型上复制SynthID-Text，实际上测试了这个(西班牙文的文章). 他五次要求水印模型描述一次对一个沿海城镇的难忘之旅。 他的五个回答中，有四个几乎以同样的方式打开了：“当我回想起我前往奇特的沿海城镇。.....时，我的心仍在唱歌/跳舞”。 对于没有水印的模型，有三个以不同的方式打开。 2个
虽然他们似乎没有对此进行测试，水印在选择下一个符号时只看一个小的滑动上下文窗口，所以无论何时，两块书写经过相同的短符号序列，都可能重新创建相同的上下文窗口，导致输出更可预测，更不多样化。 而写作中充满了相同的短句，这些短句可能会引发这种行为。
这也是这个现象影响的另一个例子：人们试图确定反应的信心水平，降低幻觉的几率，就是重复多次同样的提示，探索模型的不同路径。 然后，他们看看结果，看看共识在哪里。 嗯，这也受到影响。 有了水印，你更不能信任你所看到的共识，因为答案现在不那么独立：它们受水印键的引导。
令人感兴趣的是，虽然合成体文件指出各种反应的质量下降，但我却在文件中找不到他们用人类测试水标记方法这一特定方面的地方。 他们指出的人类试验结果并不包括测试水标记方法中受影响最大的部分。 3个
所以在我看来，他们正在牺牲质量，通过水印来限制他们的模型的创造性范围，他们认为你不会注意到或关心。
一个图案茉莉珊发现，看看AI模型为什么写得不好。 以下是一位诗人和计算机科学家Katy Gero所说的话：
让GPT-2的声音变得新鲜，Google或OpenAI等大型公司，Gero 称是一个怪胎
Natzir Turrado指出他从合成ID报的一位作者Sumanth Dathehri收到的关于X的答复：
多样性问题有很多简单的解决办法。 如果你用Gemini/Claude多次尝试同样的提示
他的“轻松解决”是否将问题转移给用户，但解决办法只是翻滚“很多次”(我的强调)，如果这是他所说的，为什么不承认这样的改变确实使系统比以前更糟糕？
Turrado回应：「好奇：哪一个容易解决
达塔赫里至今没有答复
最近一篇论文承认，这是“经过研究的LLM水标的基本权衡：质量-多样性-能力，即水标文本的质量、水标方案下的输出多样性以及整体水标信号强度之间的权衡。”
有一些方法可以减轻一些影响，使使用聊天应用程序的个人不那么明显，例如，在他们每项请求之间旋转水印键。 但这些都不是固定的。 它们只会淡化效果，并有其自身的成本，例如，使标注水在较短的文本上不太有效，而且检测成本更高。 正如另一项研究所指出的：“实现完美的无扭曲水印是不可能的”。
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
After the EU AI Act’s transparency guidelines went into effect, and following the announcement that most major AI labs will soon be watermarking their text output to comply, there has been some debate about how much of an impact this is going to have, if any, on the quality of the text produced by AI models.
Since the focus of AI labs shifted to code production, I think the written output of the models has gotten much worse.1 But it’s still interesting to try to understand if watermarking could make it even worse.
I found the discussion around technology writer John Gruber’s recent post, Anthropic’s ‘Watermark’ Text Adulteration in Claude Is a Perversion of Writing, very interesting.
On watermarking, Gruber wrote:
It’s unacceptable for a tool to sacrifice an iota of clarity, coherence, meaning, quality, etc. for the purpose of embedding hidden clues within the text to suggest its provenance.
He got a lot of pushback, with many debating how you would even measure quality, or define it.
But some argued that watermarking does not affect quality at all: it’s essentially just using different random numbers to generate the text. Daniel Jalkut, whom Gruber linked to in his follow-up, published a piece titled AI Can’t Adulterate its Own Writing, in which he wrote:
…there are also ways of tweaking the algorithm that would not diminish its randomness. For example if a random number generator routinely reversed the order of the digits in a generated number, the randomness would remain the same.
Similarly, in another discussion, someone described it as essentially swapping two faces of a die.
In a follow-up piece, Gruber wrote:
I hope that’s true. I believe it’s possible that it is true. I think it’s highly unlikely that it is true.
I think Gruber is right.
Quality loss across responses
Anthropic make a bold claim: “Watermarking does not impact the quality of Claude’s output.” The paper they link to and based their design on (Google’s SynthID-Text, published in Nature in 2024) assures everyone that the method “preserves response quality”.
So it’s easy to assume there’s been a lot of fuss over nothing, that they really are just swapping some random numbers around.
But their no-quality-impact claim, which they tested with humans, is based on single responses. Not the way millions of people actually use these systems: as back-and-forth conversations, over time, over multiple prompts and responses.
The 56-page supplementary PDF they link to at the bottom provides some more info on the method used, and it suggests the no-quality-impact claim isn’t telling the full story. Notably, it says that there is a “diversity/detectability trade-off”:
While single-sequence non-distortion guarantees the quality of each individual response, it does not necessarily preserve diversity across multiple responses
Okay, so what does that mean? They go on to explain:
when sampling several responses to the same prompt, the similarity between the responses is greater for the watermarked responses than the unwatermarked responses.
Now, if they had truly only swapped one set of random numbers for another, we would not be seeing any such difference compared to the unwatermarked responses.
They go on to write:
This could be problematic in scenarios where inter-response diversity is important, or could lower the overall quality of a system which generates many responses then selects the best one
Okay, so in their own words, it can “lower overall quality” after all. But nothing to worry about, because this would only concern you if you cared about “inter-response diversity”, for which they only provide one example: a system which generates and selects responses.
But “inter-response diversity” affects how most people use these systems too. Especially for writing tasks where you want creativity.
In response to Gruber’s point about sacrificed quality, some argued that the model has no notion of the best next word to use for what each individual is trying to convey in their specific work, and so we shouldn’t get too hung up on the next word choice. But then surely they’d agree that you, the individual, would know if it’s suitable or not once you see it.
Let’s say you prompt: “I don’t like the ending to this sentence. Please suggest a new one: [sentence].” The model suggests an alternative, but you don’t like it. No problem, hit the retry button. Well, now you’re affected by watermarking, because you have hit the “inter-response diversity” problem: like throwing a loaded die, you’ll see similar responses more than you would with an unwatermarked model.
Natzir Turrado, a technical consultant, actually tested this (article in Spanish) by reproducing SynthID-Text on an open model with the paper's settings and a watermark key. He asked the watermarked model five times to describe a memorable trip to a coastal town. Four of the five answers he got back opened in a near-identical way: “My heart still sings/dances when I think back to my trip to the quaint coastal town of…”. For the unwatermarked model, three opened in different ways.2
And though they don’t appear to have tested for it, the same thing that makes answers to the same prompt look similar could happen across different prompts too. The watermark only looks at a small sliding context window when choosing the next token, so any time two pieces of writing pass through the same short token sequence, they could be recreating the same context window, leading to a more predictable and less diverse output. And writing is full of identical short phrases that could trigger this behaviour.
Here’s another example of what this affects: one way people try to determine the confidence level of a response, to lower the chances of hallucinations, is to repeat the same prompt a number of times, to explore the different pathways the model takes. They then look at the results to see where the consensus lies. Well, this is also affected. With watermarking, you’re less able to trust the consensus you see forming, because the answers are now less independent: they’re steered by the watermark key.
What’s interesting is that while the SynthID paper points to the loss of quality across responses, I couldn’t find anywhere in the paper where they tested this particular aspect of the watermarking method with humans. If they didn’t, the wonderful human test results they point to did not involve testing the most affected part of the watermarking method.3
So it seems to me that they are sacrificing quality, by limiting the creative range of their models through watermarking, and they don’t think you’ll notice or care much.4
A pattern Jasmine Sun found looking at why AI models don't write well. Here’s what Katy Gero, a poet and computer scientist, had to say:
The same whimsicality that made GPT-2’s voice fresh also made it prone to other unpredictable behavior. “If you’re a big corporation like Google or OpenAI, you want a chatbot that’s going to make money. The chatbot that’s not going to make you money is the one that’s a weirdo,” Gero said.
Natzir Turrado pointed me to a reply he received on X from one of the SynthID paper’s authors, Sumanth Dathathri:
There's a lot of easy fixes to the diversity issue. I think if you tried the same prompt with Gemini/Claude many times, you'd get very diverse responses :)
It’s a bit ambiguous, but I wonder if his “easy fix” is shifting the problem to the user. It reads to me like he’s acknowledging that they’ve replaced the previous die with a loaded one, but the solution is just to roll it “many times” (my emphasis), and it’ll eventually land on each of its faces. If that is what he’s saying, why not admit that such a change does make the system worse than it was before for the user?
Turrado responded: “Curious: which easy fixes don’t trade off detection strength?”
No reply from Dathathri so far.
A recent paper recognised that this is an “understudied fundamental trade-off in LLM watermarks: quality-diversity-power, i.e., the trade-off between the quality of the watermarked text, the output diversity under the watermarking scheme, and the overall watermarking signal strength.”
There are ways to mitigate some of the effects to make them less noticeable to an individual using a chat app, e.g. by rotating watermark keys between each of their requests. But these are not fixes. They only dilute the effect, and have their own costs, e.g. making watermarking less effective on shorter texts, and more costly to detect. As another study pointed out: “achieving a perfect distortion-free watermark is impossible”.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.