The box says “Sonic Fiction: Synaesthetic Videos from Austria.” Synaesthesia is a hot topic right now. Currently an exhibition entitled “Visual Music” is making its way around the US, chronicling the history of attempts by the visual arts to evoke the idea of music. These are attempts at creating synaesthesia, and there have been many. But why? What is so fascinating about one sense trying to reach into the realm of another? For many of us no answer is necessary. We simply feel intuitively that transsensorial experience (as Michel Chion would call it) is like a kind of magic, or alchemy: the key to unlocking the truth that there is no death, only the transfer of energy from one being to another. We can be unmade and remade; the limits of our present bodies are essentially meaningless. Pretty heady stuff. Not surprising then that, as Gene Youngblood noted in Expanded Cinema, the idea of synaesthesia is often associated with psychedelic experience. [1]

I’m all for the freak-out version of synaesthetic experience, but I’m also interested in a slightly more tangible form of the concept. To me, synaesthesia should not be thought of as some esoteric realm of experience that certain obscure avant-gardists explore in their work. Rather, it is my opinion that it’s the cinema’s job to consider synaesthetic experience as fundamental to the medium. If one sense evoking the other is not what the cinema is about, then its two channels of transmission – sound and image – shall remain forever separate. This would be a shame. I align myself with Chion in my belief that we do not hear and see in the cinema. Rather we “hear-see.” Image and sound are part of the same thing on film, but this is not how they are treated by the majority of filmmakers who have worked throughout history. In general the idea of an audio-image, that audio-visual entity that we hear-see in the cinema, is something that is constructed by the spectators as a function of being exposed to sound and image at the same time. But the idea of the audio-image should be cultivated actively by the filmmakers from the moment of a film’s conception. This is the difference between artist intention and audience reception.

In “Designing a Film for Sound,” noted sound designer Randy Thom advocates thinking of sound at the script stage so that the film can evolve in audio-visual terms over the course of its production rather than treating sound as an afterthought. Thom has, of course, been inspired by Walter Murch, a practitioner whose resumé includes a variety of films for which sound was thought of as an integral part from the very outset. Of course filmmakers have been building their films around sound long before these two American magnates appeared on the scene. We need only look to Fritz Lang’s The Testament of Dr. Mabuse for a good example of the importance of the auditory realm in the earliest days of synch sound. Indeed, as recent research by Rick Altman, Tom Gunning, and others suggests, filmmakers like Germaine Dulac were extremely conscious of sound even in the so-called silent era.

Yet the idea of film as being constructed of audio-images has not caught on to the degree that you’d expect after a century of filmmaking practice. As support for the image, sound remains separate from what we see on screen. This may seem counterintuitive; you’d think that if a sound supports an image then they’d have some sort of fundamental connection to one another. But they don’t give Best Supporting Actor awards to players considered to be on equal footing as their Lead Actor counterparts. There must be an equal partnership for sound and image to be considered two parts of the same thing. This equality is what lacks in most cinema, and it is this equality that the idea of synaesthesia evokes. It is in search of such partnership that the present Index DVD compilation is founded.

As someone greatly invested in the study of film sound I found myself challenged by the pieces included on this disc. The challenge was to try to refine my articulation of what it is that I’m looking for in audiovisual art. When I can’t point to something and say “that’s it,” what means can I use to pass judgment? My usual feeling in work that I don’t like is that there is little substance behind the correlation between sound and image. But what is this mysterious substance? What kinds of correlation am I looking for? Much of my instinctual criticism of sound in film is generally targeted at what I consider to be a lack of interesting points of crossover between the heard and the seen. I like the idea put forth by Pudovkin and co. that sound in the cinema should always be asynchronous, for it is the space between sound and image that the interesting stuff happens. But it seems to me that sometimes this space is just too big, and I’m left floating in the abyss wondering just what it’s all about.

The potential for this space to be overwhelming is increased in situations where the creators of sound and image do not actually collaborate, where image is put to existing sound or vice-versa. In much narrative cinema, the sound team adds their work to the image. In music, whether video or performance, image is generally added as an extra feature. Chion addresses this problem in his book Audio-Vision: Sound on Screen, and as such he is mentioned in the liner notes to this DVD. Clearly there has been some thought by the editors of this compilation about the theoretical implications of the work they are presenting. Their use of Chion suggests that these films answer the problem the author has identified. Yet I still feel generally disappointed by the failure of these films to live up to the potential that I expect. The problem for me is that I’m not entirely sure what the substance of this potential is. Perhaps this is why I remain so interested in sound/image relationships within the cinema. It’s like alchemy. There is a mysterious and elusive substance out there somewhere and it is akin to a kind of magic, where the joining of sound of image creates more than the sum of its parts: an audio-image.

Perhaps the most direct address of the distance between sound and image can be found on the track blinq conceived by Billy Roisz. He commissioned a variety of sound-makers to create short samples which he then translated into images via some mysterious process. But he doesn’t present the sounds and images together. Instead he alternates between them, giving us sounds in darkness and images in silence. The notes suggest that the sound/image correlation is obvious at first, the images we see clearly stemming from the sounds heard just prior. But then slowly this cause and effect relationship reverses and we start making connections between the images that precede the sounds instead. To me this is a classic example of a gap between concept and execution. I caught on right away that we were supposed to draw some connections between the sounds and images, but the connection for me simply wasn’t there, regardless of a sound’s temporal relationship to the image. Perhaps my imagination simply isn’t up to the challenge of drawing the two together, but in my defense I think there is something missing here. Conceptually the work is right on the money for an exploration of synaesthesia, but that illusive audio-image is perhaps obscured by the fact that sounds were literally translated into images. One becomes the other with no room for the two to co-exist as equal partners.

blinq

The idea of sound and image as equal partners is evoked immediately in the opening seconds of Cubica by m.ash and Chris Janka. The video starts with a little white square appearing on a black background along with a corresponding short staccato sound. My first impression was that it was very hard to tell precisely which came first, though I had a sense that they were not simultaneous. The square then moves to the right, and the same sound is heard once again. But now there is more of a disjunction between the two since the sound is relatively instantaneous compared to the time required for the visual gesture to complete. As the video progresses, the sound/image correlations become less based on synchronization and more open. The squares become cubes and group to form lines which start moving in three dimensions. They map out their individual cubic forms through groupings of lines in three dimensional space. This is evocative of the idea of the two dimensions of sound and image creating a third: the transsensorial dimension. The video ends with two lines of cubes crossing over one another and coming to rest in the shape of a crucifix.

I find the crucifix to be particularly evocative. In a recent essay I wrote on the use of sound in Scorsese’s film The Last Temptation of Christ, I suggested that the shape of the cross makes a perfect metaphor for sound/image relationships in the cinema: two independent layers that branch off in their own directions, yet whose intersection creates a single entity. This thought was inspired by a description of sound/image relationships in the cinema once given by Walter Murch:

Image and sound are linked together in a dance. And like some kinds of dance, they do not always have to be clasping each other around the waist: they can go off and dance on their own, in a kind of ballet. There are times when they must touch, there must be moments when they make some sort of contact, but then they can be off again. There are some films where the contact is unbroken: the image leads and the sound follows – it never deviates from what you actually see, what is directly indicated. Other films are way out there – what you are hearing has only the smallest physical relationship to the image. Yet there is – there has to be – some kind of connection being made, a mental connection. Out of the juxtaposition of what the sound is telling you and what the picture is telling you, you ( the audience) come up with a third idea which is composed of both picture and sound and resolves their superficial differences…The relationship is always shifting, though, in any film. Sometimes it is very close and then it will open way up and the sound will do something completely off the wall, and then zoom back in again. But that’s where it starts to be like a dance. I mean, they’re dancing together and then they go off, and then they come back, and cross, and go in different directions again [2].

Cubica offers a formal exploration of these very ideas. Unfortunately I don’t find the film particularly enjoyable to watch or listen to. As is so often the case with concept-heavy works of art, the ideas are there but they don’t translate into an equivalent experience. Perhaps this is what I am most interested in finding: the concept/experience equivalent of an audio-image, something that stimulates the intellect along with the rest of me to such an equal extent that there is no need to differentiate between the two. And that is rare bird indeed.

On the subject of rare birds and enjoyable experiences, one of the three pieces on this disc that I found more holistically fulfilling is Falcon, by Karo Goldt and Rashim: a still image consisting of a swath of abstract colors that change in various intensities throughout the piece. Very simple, but very effective. Rashim’s soundtrack was nice, but I failed to find any real correlation between the two. So here I have the inverse problem that I had with blinq and Cubica: conceptually I was not engaged whatsoever, and thus I have nothing really to say about it.

Falcon

Equally enjoyable yet slightly more conceptually engaging is trans by Michaela Grill and Martin Siewert. Siewert is co-credited for the image and is the sole credit on the soundtrack, so here is the only example on this disc of a piece in which the boundary between the creation of sound and image has been crossed. As with Falcon the piece starts with an abstract still image, but it doesn’t hang around for very long. A succession of similar images proceed one after the other, and the time between each grows increasingly short as the similarly abstract soundtrack increases in intensity. There is a peak in the middle at which point the images move very quickly, and this calls attention to the basic perceptual fiction that we call cinema: the illusion of motion created by a rapid series of still images. But we certainly don’t need another experimental exploration of persistence of vision in this day and age. The liner notes suggest that the purpose of the soundtrack is to “not so much accompany the visuals, but to enrich them, to expand upon and complement them, to provide a contrast.” The idea of providing a contrast is in keeping with the philosophy behind the slide-show aspect of the piece, the idea being that “its main place of action is the space between two images, and its defining order is the neither-nor of two positions imagined as one.” It is this “imagining as one” aspect to the work which I find most intriguing. Trans reaches for the principle of the audio-image, but I still don’t get it. While I enjoy this work quite a bit, I don’t think it achieves what the liner notes suggest that it sets out to do. I experience it rather as a nice series of images accompanied by yet another amorphous ambient soundtrack without much to bind the two together.

trans

My favorite work on the disc is Neon: images by Nik Thoenen and Timo Novotny, and sound by Wolfgang Schloegl. It is comprised of a bank of neon lights in various stages of on and off captured at various speeds by a super 8 camera. The result is strikingly beautiful, and not without conceptual interest. Here the spaces between the positions of on and off are fleshed out, the magic of the camera allowing us to perceive states of intermediary light that our eyes are not ordinarily tuned into. This is further explored by the soundtrack which begins with very concrete synch points in which moments of flared light on the screen are accompanied by a kind of mechanical locking sound as though a scanning device had successfully locked onto an image and holds it for a moment before proceeding further. But these synch points become increasingly vague until the correlation seems almost entirely gone. Interestingly, it is towards the end of the film that the light-generating apparatus is revealed. At first we see only flares of light, but by the end we can see fluorescent tubes housed in their casing. So there is a kind of demystification going on, and it is important that a separation of sound from image accompanies the revelation of the apparatus. Sound and image are always separate in the cinema which is why illusions of synchronicity actually take away from the potential to achieve the alchemical audio-image. Alchemy is a science, and it can only be achieved when a full understanding of the component parts is achieved. This is Chion’s premise in “Audio-Vision”: to reveal the workings of the soundtrack so that we might relearn to understand cinema as an audiovisual art rather than a visual art with accompanying sound. Neon gets the closest to this ideal of all the films on this disc, and it might serve as a basic model for the philosophical formation of a generation of filmmakers who might yet reveal the hidden potential in the medium.

So I am left with the question: what is meant by the title “Sonic Fiction?” If we are to think of the visual aspect of these films, perhaps the idea of “fiction” has to do with the impossibility of translating sound into image. If we align synaesthesia with the idea of psychedelic hallucination, then perhaps the fiction here suggests that synaesthesia is impossible to create and remains a function of audience perception. You can’t turn an image into a sound, and so the sounds here are fictional accounts of the images we see on screen. I don’t believe this to be the case, though it might explain why so few filmmakers have risen to the challenge of really working sound/image relationships to their fullest potential. Whatever the implications of the compilation’s title, this disc proves yet again that there are no easy solutions to the problem of treating cinema as a truly audiovisual medium. I always appreciate outstanding efforts to push the envelope, and a few of these works have made just such an effort. But I reserve my right to be picky, and my pickyness is not in vain. Every once in a while I find another equivalent to In Absentia by the Brothers Quay and Karlheinz Stockhausen, and my mind is put at ease for I know that what I seek does exist. It goes just as quickly as it comes, and I am left to continue my search for more. Luckily I find the search as enjoyable as its rewards.

To purchase this, or other Austrian Avant-Garde DVDs, please visit the Index website.

Endnotes

¹ Youngblood, Gene. Expanded Cinema (Expanded Cinema is now available in part or whole as an online book)

² Paine, Frank. “Sound Mixing in Apocalypse Now: An Interview with Walter Murch.” Film Sound: Theory and Practice. Elizabeth Weis and John Belton, eds. New York: Columbia UP, 1985:356.

Volume 9, Issue 11 / November 2005 Essays   austrian cinema   austrian experimental cinema   blinq   chris janka   experimental   film theory   genre_experimental   m.ash   michel chion   sound   sound theory   sound-image   synaesthetic   walter murch