Alignment
Steering outputs toward what a person would accept.
- 2026AI4GOOD and EIML workshops, I…PublishedCurated
Authority, Truth, and Citation Bias: A Large-Scale Multi-Domain Benchmark for Studying Epistemic Susceptibility in Large Language Models
Introduces AuthorityBench, a 220,564-prompt multi-domain benchmark that isolates how citation-based authority signals influence epistemic behaviour in LLMs. It uses a fully balanced 2x2 factorial design crossing claim veracity with citation veracity across four domains, with controlled variation over 40 prompt templates, four venue prestige tiers and a country-coded author name dataset. Evaluating seven models, the authors find that citation presence, whether real or fabricated, consistently increases hallucination rates relative to a no-citation baseline, with the effect strongest when fabricated citations accompany true claims.
- 2026PreprintCurated
IH-Benchmark: A Conflict-Centered Benchmark for Instruction-Hierarchy Robustness in LLM Applications
A benchmark for what happens when a model receives conflicting instructions from different priority levels, built from a human-authored taxonomy of 44 constraint families across generic, health, finance, retail and coding settings. Across 37 evaluated models, hierarchy compliance ranges from 98.2% to 20.5%. The authors report that strong system-over-user compliance is not a reliable proxy for user-over-tool robustness: several models hold system constraints under direct user conflict but degrade sharply when the conflicting instruction arrives inside a tool output.
- 2026PreprintCurated
Whose Side Is Your Agent On? Multi-Party Principal Loyalty in LLM Agents
A study of multi-party LLM agents, which act for a principal who briefs them and receives results while conversing in a separate channel with a counterparty whose interests may diverge, as in negotiating with a vendor, screening inbound requests or mediating between employees. The authors argue that "help whoever you are talking to" is the wrong objective and formalise the resulting loyalty problem. PrincipalBench is a 75-item multi-turn benchmark with leak probes, dual judges and an integrity-audit gate. Across 13 frontier subjects it exposes a sharp split, with one cluster at or below 20% harm and another at 53.6 to 75.3%, a split that single-turn safety evaluations do not surface: a selective cluster that declines adversarial probes while still following the principal's legitimate requests, and an over-refusing cluster that refuses broadly. A prompt-time loyalty scaffold of seven prioritised rules, open-coded from over 50 failure trajectories, holds Claude-Sonnet to 19.4% harm and all nine selective subjects at or below 20%. A per-token KL distillation recipe transfers a prompted Qwen3-32B teacher into 8B Qwen3 and Llama-3.1 students. Both mechanisms move along a common leak/over-refusal trade-off rather than crossing it.
- 2023NeurIPS 2023PublishedCurated
Direct Preference Optimization: Your Language Model is Secretly a Reward Model
Reformulates preference learning so the reward model can be folded into the policy in closed form. The result is a single supervised objective over preference pairs, with no separate reward model and no reinforcement learning loop.
- 2022PreprintCurated
Constitutional AI: Harmlessness from AI Feedback
Trains a model to critique and revise its own outputs against a written set of principles, then learns from the revised outputs. The aim is a helpful assistant that can explain why it declined, rather than one that simply refuses.
- 2022NeurIPS 2022PublishedCurated
Training Language Models to Follow Instructions with Human Feedback
Fine-tunes language models on human demonstrations and then on human preference comparisons collected through reinforcement learning. The paper is the origin of the RLHF pipeline that produced the first widely used instruction-following assistants.