AI Guide Totem — Museo Emilio Carafa

For the opening of a new show at Museo Emilio Carafa, the museum and the Secretaría de Cultura wanted something that would bring art and technology together. More than a guide, they asked for a consultant: someone visitors could ask where a work was, where the bathroom was, what was on the third floor, whether the catalogue held anything by a given artist. Out loud, in real time, on a touchscreen totem.

The bar was a guide that felt present, not a kiosk with a menu. It had to speak and show text at once for accessibility, stay in character at all times, and — most importantly — never make anything up about the collection.

The first obstacle was scale. On the Academia's VR piece, animating a character's face for a handful of lines had already taken several days; doing that for a guide's endless possible answers was unworkable. So we switched technologies and used Convai, which runs in the cloud with an Unreal Engine plugin, its own language model and text-to-voice.

Instead of a parametric MetaHuman, the on-screen character is a photogrammetric twin of a real member of our design team, scanned and brought into Unreal Engine. The likeness was high enough that standing the actual person next to the totem got a reaction from the crowd. It was also our first time designing an interface inside a 3D environment — a HUD — and we kept it deliberately plain: a touchscreen, the instruction "tap and hold to talk," and a hand icon. That was enough for almost everyone to understand how to interact, with no extra peripherals.

Hallucinations, and specifically about authorship and dates. A language model's instinct is to always give you an answer, and here that was a live wire: asked about an artist it had no data on, it would confidently invent one. Attributing the wrong painting to the wrong artist — the equivalent of saying Picasso painted The Birth of Venus — would have been an embarrassment for the museum. And it got worse with updates: a version that behaved well would start slipping after an update. We fought it on two fronts: shaping the database format so it produced as few hallucinations as possible, and writing explicit, repeated restrictions about what to say and what not to say, plus limits to keep it away from anything political, violent or off-character.

It ran at the exhibition opening and drew a real crowd. We took away two things. A practical one: in a noisy hall, room acoustics wreck voice capture, so bring a proper external mic. And one that opened another project: the same character was extended into an agent people could carry on their phone, with the same identity and the same database (see the mobile guide project).