Why this blog exists

I’ve been keeping research notes in scattered markdown files for about two years now, and the pattern is always the same: I work something out, understand it for a week, and then lose it. Writing for an audience — even a hypothetical one — forces the kind of clarity that private notes never demand.

So this is the place for that. Mostly notes on interpretability and inverse reinforcement learning, which is what my master’s work at CTU is about, plus whatever else I end up building. I’d rather post something half-finished and be corrected than polish a draft nobody reads.

No comments, no newsletter, no analytics. If you want to tell me I’m wrong, GitHub is the place.

Open Table of contents

Code renders
Math renders

Code renders

Syntax highlighting comes from Shiki and needs no configuration:

def value_iteration(P, R, gamma=0.99, tol=1e-6):
    V = np.zeros(len(P))
    while True:
        Q = R + gamma * P @ V
        V_new = Q.max(axis=1)
        if np.abs(V_new - V).max() < tol:
            return V_new
        V = V_new

Math renders

Inline math works: the discount factor $\gamma \in [0, 1)$ keeps the sum finite.

And so does display math — the Bellman optimality equation:

V^*(s) = \max_{a \in \mathcal{A}} \left[ R(s, a) + \gamma \sum_{s' \in \mathcal{S}} P(s' \mid s, a) \, V^*(s') \right]

Which is the thing inverse RL tries to run backwards: given $V^*$ , or just trajectories that look like they came from it, recover $R$ .