4. Refusal in Language Models Is Mediated by a Single Direction (arxiv.org)

Conversational large language models are fine-tuned for both instruction-following and safety, resulting in models that obey benign requests but refuse harmful ones. While this refusal behavior is widespread across chat models, its underlying mechanisms remain poorly understood. In this work, we show that refusal is mediated by a one-dimensional subspace, across 13 popular open-source chat models up to 72B parameters in size. Specifically, for each model, we find a single direction such that erasing this direction from the model's residual stream activations prevents it from refusing harmful instructions, while adding this direction elicits refusal on even harmless instructions. Leveraging this insight, we propose a novel white-box jailbreak method that surgically disables refusal with minimal effect on other capabilities. Finally, we mechanistically analyze how adversarial suffixes suppress propagation of the refusal-mediating direction. Our findings underscore the brittleness of current safety fine-tuning methods. More broadly, our work showcases how an understanding of model internals can be leveraged to develop practical methods for controlling model behavior.

7. Good developers learn to program. Most courses teach a language (evilgeniuslabs.ca)

A bootcamp can teach you the syntax of a language in six weeks. The part that takes a decade is everything else: where the seams go, where the data flows, which decisions you are stuck with for the life of the codebase. This is a senior developer's argument for what learners should actually be looking for, after thirty years of watching the difference.

21. Welcome to Hell Developer (noahclements.com)

Reverse engineering the Wahoo ELEMNT Bolt v3 BLE protocol to unlock a hidden developer debug mode

30. What did you love about VB6? (evilgeniuslabs.ca)

Two open questions for anyone who shipped real work on Visual Basic 6 and is now writing C# against modern .NET. What specifically did VB6 get right that you miss? And what do you find frustrating about the modern toolchain that VB6 didn't make you fight? I shipped about a hundred VB3-through-VB6 line-of-business systems between 1995 and 2010, and I'm trying to get to the root of what was actually good before too much of the institutional memory leaves the room.

Article Details

Browse by Date

Type to search HN and engineering blog articles
Press Escape to close  |  ↑↓ to navigate  |  Enter to open