Extracting Steering Vectors from J space

I was reading about the jacobian space and how it can be used to verbalize the intermediate activations of an LLM to decode what it is most likely going to say or is thinking about. I wanted to test if we can use the j lens to arrive at a general activation steering vector from a couple of tokens related to the concept towards which we wanted to steer the model i.e inverting the j lens to have a general method of finding steering vectors from the concept tokens
53 points by martianvoid 45 days ago darshanmakwana412.github.io 12 comments

Summary

The article discusses using jacobian space to interpret the intermediate activations of large language models and proposes a method for generating steering vectors based on specific tokens. It offers insights into how this approach could enhance our understanding of model behavior and improve interaction with AI systems.

Comments (12 of ) ⌛

Loading more comments... ⌛
Scroll down to load more comments
No comments available.
Type to search HN and engineering blog articles
Press Escape to close  |  ↑↓ to navigate  |  Enter to open