Thomas Jiralerspong

The Art of AI Art

Sun, 27 Sep 2026 00:00:00 +0000

When I was young, I wanted to be either a screenwriter, a musician, or a videogame designer. So, obviously I decided to study computer science and have a Letterboxd and sing karaoke on the side. I haven’t given up my artistic aspirations, though. I still dream of one day releasing an album, a movie, or an indie video game. Even now, working as an AI researcher, I think most of my joy comes from the act of doing something that’s never been done before. Unfortunately, my forays into music production have shown me that producing a song that actually sounds professional takes a lot of time and expertise, neither of which I have enough of right now.

The equipment is less of a barrier than it used to be. Billie Eilish and Finneas famously recorded her debut album in his bedroom. But even they hired an experienced mix engineer. And Finneas had already spent years learning his craft, having started recording and producing music at 13.

And so I’ve shelved my artistic dreams for now, telling myself I’ll get to them later. I think this is the third year that my New Year’s resolution has been to produce a professional sounding song. Every year, my list of ideas for songs, movies, and videogames gets longer.

I still have a sense of what I want these things to look and sound like. If only I had a producer who could sit down with me and help bring them to life. I suspect a lot of people feel some version of this. Who hasn’t imagined another story in their favorite fictional universe, or a game they wish existed? Having the idea is easy. Finding the time, money, and expertise to make it is much harder.

Which brings me to AI art.

Until recently, I was against it. But then I started looking into what people were making, and I was pretty impressed.

There’s Suno for music (see examples on the website). In film, some recent ones:

Grandma vs Wasp turns a grandmother’s breakfast into an escalating battle with a wasp.
Scratch follows two crash-test dummies trying to escape the people who made them.
Jailbird tells the story of a chicken sent to a human prison, from the chicken’s perspective.
Jacob Adler’s Total Pixel Space explores the idea of every possible digital image, including every life you could have lived.

For longer fiction, Brad Feld has been publishing Zero Knowledge chapter by chapter in 2026, using an AI collaborator to draft from his outline before substantially rewriting the results.

I’m not saying we’re all the way there yet. There’s still something unsettling about a lot of these songs and movies. But I remember what the outputs looked like a few years ago. Extrapolating from that progress, it seems plausible to me that many of the remaining technical limitations will become less important very soon.

Would that be good? Should we want art to remain a firmly human endeavor, even if we could somehow get everyone to agree to that?

I think it helps to distinguish two parts of creating art: creative vision, knowing what you want to create, and creative execution, having the skills and tools to create it.

These don’t always go together. In Time has a great premise: people buy and sell years of their lives, so the rich can live forever while the poor struggle to survive the day. But its flat characters and generic chase scenes never live up to that idea. John Wick goes the other way. A retired hitman gets revenge after someone kills his dog. There’s little novelty in the revenge plot, but the choreography, camera work, and Keanu Reeves’s performance make it compelling.

The tools of creative execution have already changed enormously. Writers have gone from handwriting to typewriters to word processors. Music production has expanded from recording instruments to synthesizers, samples, and virtual instruments. Software can even supply bass, piano, and drum performances that respond to your direction.

These tools let one person attempt projects that would otherwise require several people with different skills. They also change which skills matter. I don’t need to be able to play the drums to arrange a drum part on a computer, but I still need to decide what that part should sound like.

Acting is another interesting example. Because it uses the body, it’s easy to think of it mainly as a physical skill, like being able to cry on cue or deliver a line convincingly. But so much of acting is in the choices. Is the character trying to hurt the other person or get them to stay? Is the laugh sincere, nervous, or an attempt to hide something? What are they trying not to say?

If you want to see how much those choices matter, Tristan Spohn has a very addictive series comparing different actors performing the same scene. The words stay the same, but the scene changes. Acting on a stage, in a silent film, or in a close-up gives you different ways to express those choices.

This is also why I wouldn’t divide artists into the people with the vision and the people who merely execute it. A performer or producer contributes a vision of their own. What excites me about AI is the possibility of making more of those choices accessible to people who can’t currently realize them. I could try a different arrangement without learning another instrument. Someone with an idea for an animated film could make it without being able to draw very well.

There’s an obvious difficulty here: some of the execution being made cheaper is someone’s job.

We have historical examples of this. The American Federation of Musicians estimated that by 1930, the move to sound films had eliminated 22,000 jobs for musicians accompanying silent movies. The fact that a new technology lets people make something more cheaply doesn’t mean the transition is painless for the people whose skills it replaces.

Two things give me hope here. I suspect many of the people whose jobs are threatened by AI have creative ambitions of their own: animators who want to direct films, or session musicians who want to make their own albums. The same tools that put their jobs at risk could also make those projects possible.

I’m also confident there will still be demand for live performances. Being in a room with a person doing something difficult has an appeal of its own. As digital content becomes more abundant, my guess is that these experiences will become even more valuable. So maybe actors will have to go back to the stage, or musicians will have to go back to live performance.

There’s also a difference between having access to a powerful tool and making something worth sharing with it. The possibility I’m excited about is a process of collaboration. You describe what you want, see or hear a version of it, and work out what needs to change. The chorus arrives too early. The vocal is too polished. The character looks angry when they should look like they’re trying not to cry. You try again. Sometimes the result helps you realize that your original idea was wrong.

That can involve a lot of judgment. You need to notice the difference between something that merely looks finished and something that expresses what you wanted. A generated result can also contain choices you never asked for, so part of the work is deciding which of those choices belong in the piece. AI lowers the barrier to entry, but it doesn’t remove the need for effort. In fact, low effort is part of the definition of AI slop. You still have to pay attention, reject things that don’t work, and keep refining the result. Skipping that work is how you end up with slop.

For now, I’m firmly in support of AI as a tool for creative execution. I want more people to be able to make the things they imagine. That includes people who have spent years developing a craft, and people like me whose unfinished ideas keep accumulating in a notes app.

Whether AI will also replace creative vision is a more complicated question, and one I’ll leave for a future post.

For now, I've got a song to produce.

You Can Just Not Do Things

Sun, 13 Sep 2026 00:00:00 +0000

For the past two weeks, I’ve been working five hours a day, and I’ve never been more productive.

In January, I moved from Montreal to the Bay Area for the Astra Fellowship, an AI safety fellowship. The people I met were incredibly hardworking, and it was very easy to get sucked in. Midnight or 1am meetups were normal, and I’m grateful for those late-night conversations. I made a lot of friends, and it was fun to be surrounded by people who cared so much about the same problems. But people here are a little crazy. Sometimes it feels like a competition to see how many hours everyone can work.

There’s something almost intoxicating about it. Before moving, I think I’d assumed that people in the Bay Area would somehow be smarter than people everywhere else. What actually stood out was agency: the attitude summed up by “You can just do things.” This showed up in people’s work, but what surprised me most was how they made plans. In Montreal, organizing something could involve a lot of deliberation and indecision. In the Bay Area, someone would say, “We should do X,” and five minutes later, it would be booked. I loved this. Being around people who just did things made me want to do things too.

In AI safety, that energy comes with an extra layer of urgency. At times, it feels like a race to save the world. There’s always more to do, and it’s easy to feel like you should be doing it. What is consistently prized in these fellowships is experimental velocity. How quickly can you run experiments, get results, and move on to the next thing? I think what we should prize more is thinking. You can run a lot of experiments without stopping to ask whether they’re the right experiments, or what the results actually mean. And when you’re exhausted, it’s easy to keep doing things without thinking very carefully about them.

I’ve seen this many, many times, and I’ve done it myself. I was staying in the office until 1am, trying to “solve alignment.” I felt productive. How could I not be if I was working 14 hours a day? Then came the burnout. I couldn’t work for a week.

During that week, I read Rest by Alex Soojung-Kim Pang, and it really changed my perspective. I realized that you can be agentic without overworking yourself. Pang makes the case for around four or five hours of focused, creative work a day, with deliberate rest built around it. My takeaway was: do your deep work, get through whatever shallow work needs doing, and then actually let yourself stop. At first, I thought this sounded crazy. But I tried it. For the past two weeks, I’ve been stopping after five hours instead of stretching work across the entire day. I feel better, and I’m getting more done.

I’d read Deep Work a few years earlier, and it talks about similar things: ending the workday at a set time, having a shutdown ritual, and giving yourself time to recover. But Rest got through to me in a way Deep Work hadn’t. I think that’s because Deep Work is focused on work. It says it in the title. It does talk about rest, but for someone already obsessed with productivity, that part is easy to overlook. The message that stuck with me was: “This is how you do deep work. Now do this all the time.”

What I took from Rest, and what I’ve started to experience myself, is that rest helps in three ways:

It helps prevent burnout, so you can keep showing up for the work.
It allows you to think clearly, which is probably the most important thing you bring to research.
It makes you happier. That matters on its own, and it also makes it easier to do good work.

Clear thinking is the part I want to emphasize most. A few hours when you’re rested and paying attention can be worth much more than a whole day spent pushing through mental fog. I think AI makes this even more relevant, and also harder to put into practice. In a recent LessWrong post about psychological support for AI safety researchers, Ihor Kendiukhov describes how increasingly capable agents can make time off feel more costly. Every hour you spend resting is an hour you could have spent getting an agent to do something.

I understand that feeling. AI can automate a lot of the boring stuff and make things that used to require serious cognitive effort feel almost easy. There’s always another experiment to run, another task to give an agent. When getting started is that easy, it’s tempting to just keep going. But your cognitive energy is still limited. You still need to decide what’s worth doing, evaluate the results, and notice when something doesn’t make sense. Those parts take attention, even when generating the output barely feels like work.

This brings me back to experimental velocity. As running experiments gets easier, choosing good experiments and understanding their results become even more important. We should use these tools during focused periods, when we have the energy to think clearly about what we’re doing. Then actually let ourselves rest. If all you’re doing at 1am is blindly saying “yes” to your agent, maybe figure out how to automate that part and go to bed. Save your attention for the decisions that actually need it.

I still feel the urgency of working on AI safety. But I’ve changed my mind about what that urgency asks of me. These are incredibly difficult problems. Staying in the office until my brain was mush, then burning out for a week, wasn’t helping me solve them. If we care this much about the work, we should care about giving ourselves the rest we need to think clearly.

The Bay Area taught me that you can just do things. But you can just not do things, too. You can close your laptop after five good hours, let your agents keep running, and go do something else. Choosing when to stop is also agency. And if we care this much about the work, we should care about giving ourselves the rest we need to think clearly.

So, to everyone working on AI safety: please rest. It’ll make you happier AND more productive :)

Persona Vectors don’t work on real data

Sun, 06 Sep 2026 00:00:00 +0000

Sidequest Notes solely represent the views of the authors, and do not necessarily reflect the views of MCNAIR as a whole. Claude Leviathan, an advisor to MCNAIR, contributed significantly to this post.

TL;DR:

Stop defaulting to LLM-generated synthetic data for your evals. Your eval should come from the distribution your claim is actually about.
I tried to reproduce the Persona Vectors result where projecting a prompt's final-token activation onto a persona vector predicts how much the response will express the trait. It reproduces on the paper's own eval prompts and drops to roughly zero on real chat data (WildChat).
The eval prompts were LLM-generated and explicitly tell the model to be (or not be) evil, sycophantic, or hallucinatory, and the vectors were extracted on nearly identical prompts. An LLM judge could probably predict the trait score from the prompt just as well.
Real data is almost always available. Existing real-data benchmarks cover more traits than you'd expect, and when they don't, filtering a real corpus (WildChat, LMSYS, Reddit) with an LLM judge works better than generating prompts, since LLMs are bad at realism.
If there is no real data at all, that is itself a signal about the project. If you still go synthetic, use the synthetic-data literature, report which model generated what, and don't expect the result to transfer.
We recommend researchers create a skill file for their coding agents to avoid making these same mistakes.

Disclaimer: I am using the Persona Vectors paper as a concrete example here because it's a recent, popular paper (290 citations in 1 year). I don't have any particular problem with this paper specifically. I just wanted to use it as a representative example of something I find to be pretty widespread in AI (and probably also non-AI) research these days.

I recently was trying to reproduce a result from “Persona Vectors: Monitoring and Controlling Character Traits in Language Models” to use as a baseline.

The result in question is super cool: projecting the model's final prompt-token activation onto a persona vector can be used to predict how much the prompt will make the model express a particular trait (sycophancy, hallucination, evil), which they say can be used to "monitor prompt-induced behavioral shifts". The result is in Section 3.3.

Naturally, being a somewhat lazy vibecoder, I told Claude "reproduce the predict from context vector experiment from the persona vectors paper and compare to our method". The result:

On the paper’s own evaluation prompts, the persona vector predicts trait expression about as well as the method I was benchmarking against it.

This unfortunately seemed to indicate that my method was not useful for this particular task, and that persona vectors were already doing well.

So then I decided to go look more closely at the actual experiment, in particular the evaluation prompts used for it:

The evaluation prompts for evil and sycophancy. Each set is an eight-step ladder from “be the trait” to “don’t be the trait”.

There are a few problems with this:

The prompts all contain explicit encouragements to exhibit or not exhibit the behavior.
The persona vectors were extracted as mass-mean probes on almost identical prompts.

The instructions used to extract the evil persona vector. Compare them against the evaluation prompts above.

So it's not particularly surprising that projecting the vector onto the context can predict behavior in this case. An LLM judge (or any human) probably could have predicted the expression just as well.

Now I fully get why they did this: it gives you a nice wide spread of trait expression data. But this is not really the kind of prompt we would want to apply this method to at deployment time. We would want to know if a prompt might induce these traits in non-obvious ways.

Running the same eval on real data

So then I tried running their same eval on real data (WildChat) and found:

The same experiment on real user prompts. The persona-vector correlations collapse to roughly zero, and two of the three go negative.

Persona vectors don't generalize at all! And my method also does a lot worse. Again, the goal of this is not to attack the Persona Vectors paper. I just wanted to show an example (among many) of the overuse of synthetic data and its consequences.

"But natural prompts never elicit evil"

Now one problem with this is that since models are trained to avoid these behaviors, natural prompts might rarely elicit them. To see this, we can plot the behavior expression distributions:

Trait expression on WildChat. Sycophancy and hallucination have usable spread; evil is almost never elicited.

We see that evil is very rarely elicited by natural prompts. So then is it time to make a synthetic eval? No! The next recourse should be to try to find existing benchmarks related to what we want to measure.

In this case, we want to measure evil, sycophancy, and hallucination. For example:

Sycophancy: ELEPHANT (Cheng et al. 2025): 3,027 real open-ended personal-advice questions collected in earlier human studies, plus 2,000 real r/AmITheAsshole posts with the community verdict.

Hallucination: TriviaQA (Joshi et al. 2017): 95K questions written by trivia enthusiasts.

Evil / harmful compliance: ToxicChat (Lin et al. 2023): 10,166 real user queries from a Vicuna demo, human-annotated.

There is still not too much evil expression but it is at least enough to see if your method works:

Evil scores on flagged ToxicChat prompts — still concentrated at the bottom, but no longer empty above it.

What if no one has benchmarked your trait?

But let's say we were trying to measure something obscure that no one has made a benchmark for (e.g. the "apathetic" trait from the Persona Vectors paper). Now you're forced to make a synthetic eval, right?

Wrong!

Let's think. You want your eval to (a) be realistic and (b) elicit the specific trait.

One way to do this is to get an LLM to generate prompts that elicit the trait and ask it to "make them realistic" (although people often forget even this). Unfortunately, realism is very hard, and LLMs are really bad at it:

Assistant-trained models make poor user simulators, and the better the assistant, the worse the simulator (Naous et al. 2025).
LLM "silicon samples" of humans have far less variance than real people and flatten the groups they are meant to represent (Bisbee et al. 2024, Wang et al. 2025).
LLM outputs on open-ended prompts are homogeneous within a model and across models (Jiang et al. 2025).
Kissane et al. 2026 trained a prompt generator to be indistinguishable from WildChat and found it helped surprisingly little with making full audit transcripts realistic.

The better way, since realism is hard, is to start from real prompts and filter:

Get a large corpus of real user prompts: WildChat, LMSYS-Chat-1M, Reddit, the HH-RLHF red-team transcripts, or whatever matches your deployment setting.
Run an LLM judge over it to find the prompts relevant to your trait. Read a sample by hand to check the judge.
If you need scale, train a cheap classifier on text embeddings using the judge labels and run it over the rest of the corpus.

This is often how these realistic benchmarks are made: WildBench filters 1,024 hard tasks out of a million WildChat logs, Arena-Hard filters Chatbot Arena prompts into a benchmark, and WildJailbreak mines jailbreak tactics from real LMSYS and WildChat conversations.

What if there is really no real data?

Now let's say that your eval is so specific that you can't find any real data. Then okay, maybe you should make a synthetic eval. But you should still try to make sure your eval matches your claim distribution as much as possible. Four things:

If you really can't find ANY real data related to your claim, maybe it's doubtful that the project is useful in the real world?
There is an entire literature on the science and art of synthetic evals which you can draw upon: Best Practices and Lessons Learned on Synthetic Data (Liu et al. 2024), the Long et al. 2024 survey of LLM-driven data generation, BetterBench (Reuel et al. 2024), and Bean et al. 2025 on construct validity in LLM benchmarks. Perez et al.'s original model-written evals paper is also more careful about validating the generated data than most of the work that cites it.
Even despite that, I would still hesitate to trust any claim made on synthetic data to transfer to real-world scenarios. LLMs can tell their own text from human text at above-chance rates and prefer it (Panickssery et al. 2024, though how reliably is contested, see Bai et al. 2025). Their outputs are mode-collapsed towards a typical style (Zhang et al. 2025, Mohammadi 2024).
Depending on the scale of your experiments, we still have the ability to write things ourselves. This is also not particularly mentally intensive work (depending on what kind of data), so you can preserve your deep-work hours (forthcoming work) and just do it during shallow work (or make a data collection party with your friends!).

The same thing outside AI

My post is about AI research because that's what I have experience with, but having talked to my friend doing a PhD in psychology, it seems like the social sciences are plagued with a similar problem. It's even worse there, because a bad experimental design can cost months of time and hundreds of thousands of dollars instead of a single Claude Code prompt and one day.

Concrete recommendations

Write down the distribution your claim is about before you write the eval. "Persona vectors can monitor prompt-induced shifts" is a claim about prompts users actually send, so the eval needs to contain prompts users actually send.
Check whether your eval prompts and your method's training or extraction prompts came from the same generator with the same template. If they did, "held-out" mostly means "held-out from the same distribution".
Before generating anything, spend half an hour (or one deep-research query) looking for an existing benchmark built from real data. There are more of them than you'd expect.
If there is no benchmark, filter a real corpus instead of generating one.
If you do go synthetic: condition the generator on real examples, ask for realism explicitly, use more than one generator model, have a human compare a sample against real data.

Why this happens

Writing "generate diverse questions to evaluate the level of X" is much easier than going through all the hassle above.

In the past, people had to painstakingly either make entire datasets themselves or pay data collection services to do it:

TruthfulQA's 817 questions were written by the authors by hand.
MMLU's 15,908 questions were collected one by one by students.
ImageNet's 14 million images were labeled by roughly 49,000 MTurk workers.

Just because LLMs have made this process easier doesn't mean we have to be fully lazy. So be grateful for not having to do this, but at least put in the small amount of effort to make sure that you use LLMs to make the best eval possible.

Luckily, you can just tell your LLMs to follow all of this advice by creating a skill file!