Skip to main navigation Skip to search Skip to main content

Did I really agree to that? making sense of policies you didn’t read with models that actually did

Research output: Contribution to journalArticlepeer-review

Abstract

Introduction. While Privacy Policies and Terms of Service (ToS) are intended to inform users; they often overwhelm, mislead, and confuse in practice. This work investigates automated techniques for analyzing such legal documents, with the goal of supporting user compre hension and regulatory auditing. Method. We use expert-driven annotations from Terms of Service; Didn’t Read (ToS;DR) to train classification models that assign case labels to individual sentences. We also develop a classifier to distinguish document types: Privacy Policy, ToS, or other legal te xt. Analysis. Models were evaluated using F1 score, and we compared traditional fine-tuned models (RoBERTa, PrivBERT) against GPT-4 Turbo. We then applied the best-performing models to real-world policies to uncover conceptual overlaps between Privacy Policies and ToS. Results. Our case classifier achieved a 0.73 F1 score, while the document classifier reached 0.79. GPT-4 performed worse on case classification (0.58 F1). We found that GDPR-relevant clauses often appear in both Privacy Policies and ToS, blurring distinctions and raising risks for user misinterpretation and regulatory non-compliance.-type Conclusion. By surfacing hidden structures and overlapping clauses, our system enhances transparency and supports digital literacy by increasing accessibility of complex documents. This work lays the foundation for tools that promote user agency and platform accounta bility.

Original languageEnglish (US)
Pages (from-to)699-717
Number of pages19
JournalInformation Research
Volume31
Issue number2026
DOIs
StatePublished - 2026

All Science Journal Classification (ASJC) codes

  • Library and Information Sciences

Fingerprint

Dive into the research topics of 'Did I really agree to that? making sense of policies you didn’t read with models that actually did'. Together they form a unique fingerprint.

Cite this