Presenter: Xuandong Zhao

This section provides the background for the tutorial, presenting the challenges posed by machine-written text and the potential ethical issues arising from Large Language Models (LLMs). We will introduce two primary approaches to addressing these issues: post-hoc detection and watermarking methods.

Presenter: Xuandong Zhao, Yu-Xiang Wang

We delve into the process and evolution of watermarking for natural languages. We further explore watermarking methods specifically designed for Large Language Models. We will also cover the theoretical analysis of each watermarking method.

• Early stages of text watermarking
• Watermarking for Large Language Models:
  • KGW (Green-Red) Watermark: Kirchenbauer et al. (2023) A Watermark for Large Language Models
  • Unigram (Green-Red) Watermark: Zhao et al. (2023) Provable Robust Watermarking for AI-Generated Text
  • Gumbel Watermark: Aaronson (2023) Watermarking of Large Language Models
  • Undetectable Watermark: Christ et al. (2023) Undetectable Watermarks for Language Models
  • Distortion-free Watermark: Kuditipudi et al. (2023) Robust Distortion-free Watermarks for Language Models
  • PF Watermark: Zhao et al. (2024) Permute-and-Flip: An Optimally Robust and Watermarkable Decoder for LLMs
  • Unbiased Watermark: Hu et al. (2023) Unbiased Watermark for Large Language Models
  • Mark My Words: Piet et al. (2023) Mark My Words: Analyzing and Evaluating Language Model Watermarks
  • PRC Watermark: Christ and Gunn. (2024) Pseudorandom Error-Correcting Codes
  • Other methods...
• Trade-offs among watermarking methods:
  

Presenter: Lei Li

This part explores other related watermarking methods for AI models. We will discuss various methods used to safeguard intellectual property using watermarks.

• Copyright protection against model extraction attack:
  • CATER: He et al. (2022) CATER: Intellectual Property Protection on Text Generation APIs via Conditional Watermarks
  • DRW: Zhao et al. (2022) Distillation-Resistant Watermarking for Model Protection in NLP
  • Ginsew: Zhao et al. (2023) Protecting Language Generation Models via Invisible Watermarking
  • Radioactivity: Sander et al. (2024) Watermarking Makes Language Models Radioactive
• Model detection against finetuning or pruning:
  • DeepJudge: Chen et al. (2021) Copy, Right? A Testing Framework for Copyright Protection of Deep Learning Models
  • Instructional Fingerprinting: Xu et al. (2024) Instructional Fingerprinting of Large Language Models

Presenter: Lei Li

This part focuses on post-hoc detection methods, detailing the use of binary classifiers and statistical outlier detection techniques. We will discuss the theoretical and empirical limitations.

In the final section, we will provide a summary of the tutorial, discuss the best practices for implementing text watermarking, and explore potential future developments in the field of text watermarking.