Eloi Maduell and Santi Vilanova are Playmodes – an audiovisual research studio developing self-made, open-source technologies. Focused on an exploration of the materials they work with, Santi’s sound design and Eloi’s visual expressions have found their form in mediums like light sculptures, audiovisual instruments, projections onto the stars or even pebbles and twigs. Nxt Museum called the artist duo to talk about ‘Espectres’, their newest work on show in the 370m2 projection in Nxt Stage.

NXT: Can you tell us about your work, ‘Espectres,’ which is on show at Nxt Museum?

SANTI: So, ‘Espectres’ is a work that explores synthetic spectrograms. We’re used to spectrograms as a visual representation of the spectrum of frequencies of existing sound. In ‘Espectres’, we turn that around. We first create the visual representation of the sound and then transform this image into sound. ‘Espectres’ is an exploration of that. It’s a generative work. That means that when it works in real time, the machine decides what the musical and visual results will be. We don’t have 100% control, and yeah, it’s all driven by randomness and probabilities.

Is it central to your work that the viewer understands immediately what they’re looking at?

SANTI: The beauty of ‘Espectres’ is that you can understand a part of it. Still, there is a complexity there which can be overwhelming. You can arrive to understand that all that is music, that all that is creating the sounds you’re gonna hear, but you cannot grasp everything. And this overwhelming complexity – it’s part of the beauty.

I think it is possible to read it, but you would need training and understanding of what a spectrogram is and how it works. Of course, there are very obvious elements, like when you see the snare drum coming. You know that’s going to be a snare drum because you previously heard a couple, and you know that gradient on that side of the screen will sound like a snare drum. However, there are many details that are very difficult to anticipate. 

ELOI: Having ‘Espectres’ displayed on such a big screen at Nxt Museum and having people radically inside this universe of visual music and its complexity works quite perfectly. It’s an audiovisual music shower.

How do you explore the world of audio-visual art? 

ELOI: Santi and I met in the party scene in Barcelona in the early 2000s and connected through our interest in the junction of sound and visual inputs. And in building an audio-visual language together. In the past 20 years, we explored many possibilities for combining sound and visuals. We worked with video and mapping, with light, and some kinetic art, but no matter the medium we always create this audio-visual thing that drives our performances.   

Is there a broader ethos and approach to your work that may be reflected in the free party scene? Does that also inform your creative process?

ELOI: When we met in the underground of Barcelona it was with many people involved in software development related to visuals, sound, new music, and stuff, and that’s also part of our philosophy. So we’ve learned, and we grew up with free parties and open source, and that’s part of our core in how we work. 

SANTI: The free party scene was a form of movement that started in the 1990s, at least in Barcelona. It was a social movement based around the need of the youth to have their own autonomous spaces where societal norms didn’t exist. It was this creation of a space without rules, outside space and time, where you had the freedom to relate to each other the way you wanted, without the influence of capitalism. 

ELOI: We were lucky that computers started to be powerful enough to drive video and sound at that moment. It was affordable for many of us to have a computer and be able to do weird things with them. The free party scene and rave culture helped us to find ways to explore these languages. We didn’t know where we were going; we just wanted to explore the possibilities of using computers to drive media in these environments. The open source community, or the open source movement, helped us to learn and collaborate with other people without real monetary interest. We were not doing this for money or as a job. We were just having fun.

What is the artist’s role when it comes to artworks like ‘Espectres’ that are driven by chance and probability? 

SANTI: All of it. Random is just a function. We declare the rules of the game. The machine has a very narrow set of possible decisions: You can choose randomly between one and 23, but not outside one or 23; or you can choose this musical scale and this musical scale but nothing else. 

With ‘Espectres,’ we created that framework of possibilities—and, of course, all the programming. So, the way the machine behaves is part of our decision. The results can be surprising, but they never depart from what we defined. 

ELOI: My rules, but you can do it. Intrinsically, the process and the results are different. That doesn’t mean that the computer is deciding. The number of hours behind controlling the parameters is huge, and we just use random data as a kind of child play to see what the machine comes up with. We take all decisions, but we use probability to give dynamics to the whole thing.

SANTI: Latent spaces. It’s like we designed the space of possibilities. We do not design all possible outcomes, but we design the space of possibilities.

Do these generative processes strongly influence the future of music making, music notation, and how we produce and understand music generally?

ELOI: I think the future is AI-driven, sadly – or not sadly. I don’t know. It’s difficult to talk about the future with the AI revolution happening right now. For my taste, from what I’ve heard now, the outcomes of AI-generated music are kind of cheesy. Maybe that’s because of the cheesy decisions of the ones training them. We’ll see what happens when someone with a cool taste trains these models. But in the end, it’s like statistics. I mean, it’s very difficult to hear something radically new coming out from these trained models because, in the end, it’s responding to training. 

New, radical ideas still come from human intelligence, not artificial intelligence. But I think it will be a matter of time. It will come. And then we should ask ourselves: What’s our role other than being pets of the machines?

I’m looking forward to seeing what that brings for creativity, regardless of the fear that may be hidden behind it.