AI rebuilt the picture a person was looking at

Researchers at Osaka University paired an image-generating model, Stable Diffusion, with fMRI readings taken while people looked at photographs, training the system to match its internal representations to patterns of brain activity rather than training a new model from scratch for the task.

The combined system rebuilt recognizable versions of the images each person had been shown, matching rough layout and semantic content, using each person's scans from a large public dataset. No additional fine-tuning of Stable Diffusion itself was needed to make the reconstructions work.

The approach also let the researchers probe which parts of a diffusion model's internal process track which kind of brain activity, linking specific computational steps to specific visual functions. The reconstructions are blurry approximations of the original photograph, not exact copies, and every result is tied to one individual's scanner data, not transferable to a different person's brain.

A machine sketched what the eye alone had seen.