False relations: On the Problem of Translation Between the Sonic and the Visual Sign

by Erkki Veltheim

(This text relates only to the tradition of notation in Western music, and makes no claims about the universality of these observations.)

The Paradox of Notation (1)

The notation of music presents itself as something of a paradox; an attempt to represent a fleeting series of sounds by means of a fixed spatial diagram.

Certain musical parameters, such as pitch and rhythm, at first seem to lend themselves to a more consistent translation into stable notation than others, such as timbre and dynamics, but only in a contingent way; that is, by presupposing a standardised and stable relationship between a particular written sign and its sonic rendering.

Natural languages similarly rely on a collectively agreed method of translation between the written and the spoken sign; but while these signs encode conventionally established meaning that can be recombined productively to refer to things in the world, musical signs are intrinsically meaningless and refer only to themselves, as sounds.

(A musical phrase is pure phenomenon; it does not point to anything else in the world. It may encode implicit information, such as cultural context, social status or level of competence, but does not have a clearly defined referent that would constitute meaning in the same sense as that encoded by signs in a natural language.)

The fact that sounds can be represented in written notation, which allows for their free recombination and permutation within a given logical scheme, seems to deceptively bestow on music an ontological status similar to that of natural languages; as musical phrases can be constructed from smaller units much the same way as words and sentences, it seems plausible that different combinations of pitches and rhythms would create different meanings.

(This results in a category mistake that is compounded by conflating musical and linguistic signs' contrasting modes of reference in music that uses text; in other words, interpreting the music and text to refer to the same thing together, as if the words could by some contiguous magic permeate a melodic phrase with their meaning.)

##Origins of Notation

The word music is derived from the Greek mousikē tekhnē, ‘the art of the Muses’. The Muses were the daughters of Mnemosyme, the Ancient Greek personification of memory, suggesting that the origins of music are rooted in oral culture; musical works being committed to memory and passed down the generations in an immediate way, from teacher to student.

The earliest known example of music notation is a Hurrian song from around 1400 BC; a set of instructions for a singer and a harpist carved in cuneiform on clay tablets found in the ancient Syrian city of Ugarit. Whilst any transcription of such instructions into modern notation is necessarily speculative, these tablets attest to a very obvious yet radical role of music notation; that of a vehicle for the recording, studying and reproduction of music that frees it from a dependance on one-to-one contact between musicians. This opens up the prospect of individual authorship of a mediated form of music that can be disseminated at a distance, both physical and temporal; the birth of a 'composer'.

(The development of writing in Ancient Greece has been seen to accompany a transition from the collective consciousness of oral culture to the self-consciousness of literate culture; the written word becoming a concrete record of an individual's thoughts, without the need for an interlocutor. Compare for instance Plato's writings, devised as imaginary debates between agonistic philosophers, to those of Aristotle, who writes in his 'own voice'.)

Notation codifies musical ideas and henceforth becomes an effective normative and evaluative standard for performance; individual and collective memory can only regulate musical works contemporaneously, while a score does so as an enduring and immutable law; it creates a new concept and mythology of the musical work.

The Uses of Notation

Whilst various traditions of music notation arguably originated as mnemonic and communicative tools for the reproduction of musical works, some of these traditions have developed into sufficiently general systems of symbolic representation to enable them to be used beyond their original context. These uses fall into two mutually exclusive categories: reactive or transcriptive notation (for instance the transcription of environmental sounds, such as birdsong) and generative or prescriptive notation (for instance the composition of musical works).

The mutually exclusive character of these categories is obscured by the standardised form of notation; for instance, the reactive transcription of a pre-existing musical work as a mnemonic device can be retrospectively mistaken for an example of a generative, prescriptive act of composition, as there is nothing in the appearance of such notation that necessarily distinguishes one from the other. Notation as a generalised system ignores the inherent tension in the switching polarity between these two categories and their relation to performance, neutralising their antithetical entanglement with a veneer of mute impartiality.

(Transcription, which presumes the performance to be the primary creative act, in fact already hints at the potential for a reversible function; that is, the prescriptive use of notation in the creation of a musical work, prior to any performance. This reversibility also leaves open the theoretical possibility that notation in fact originated from the latter category, as a generative, creative tool, and was only subsequently reconfigured for use as a reactive, mnemonic tool.)

Each notational tradition has its own limitations given by its internal logic and the parameters of sound that it is able to represent more or less accurately, and therefore any transcription in some way contorts sonic phenomena to fit within these parameters; much like the way Euclidean and non-Euclidean geometries map the same terrain to fit into their own, contrasting forms. Similarly, any notational system a priori limits the musical ideas it can support, and hence always already determines the kinds of aesthetic result that can be achieved in employing that system.

(No sound can be fully represented by any given notation, nor by any other recording device; this incomplete translatability between a sound and its visual representation has contributed to the confusion in the ontological relation between the two.)

The internal logic of a given notational system, abstracted from the temporality of music performance, can also propagate new approaches to the manipulation of different musical parameters.

The geometrical properties of music notation invite experimentation with various kinds of symmetries and processes that would not be available to a musician purely thinking and unfolding musical ideas in real time.

(This results in another category mistake, where temporal processes are assumed to be analogous with spatial proportions.)

Typology of Notation

Different types of music notation can be seen to fit Peirce's typology of signs, categorised as:

-symbolic, exhibiting an arbitrary link with the referent (established by convention); for instance, phonetic notation, using letters of the alphabet, and diastematic notation, representing sonic relationships geometrically.

-iconic, exhibiting some form of similarity with the referent; for instance, pictographic or ideographic instructions for actions or construction of a sonic object.

-indexical, exhibiting a causal or physical link with the referent; for instance, tablature notation, where diagrammatic instructions applied to a specific instrument or object cause a certain sonic result.

The Epochs of Notation

Radical historical changes in music notation have coincided with large-scale political, technological economic and cultural shifts, each such epoch reinventing the social role of music and its manner of reproduction and distribution.

-The Carolingian Renaissance: Up till Charlemagne's reign in the 8th Century, sacred vocal music, the precursor to what we now think of as classical music, was by and large an oral tradition, studied and performed in the context of a stable monastic milieu. Charlemagne's push for cultural reformation and standardisation across his empire resulted in the institution of cathedral schools with transient ecclesiastical populations, necessitating the development of mnemonic devices for the learning and transmission of officially sanctioned versions of plainchant.

-The Renaissance: The invention of the printing press in the 15th Century led to another wave of standardisation, accompanied by the commercialisation, of music. The advent of music printing gave composers access to the general public as a viable market, leading to the rise of instrumental music; a popular pastime for the aristocratic classes as an index of their social status. This began the elevation of the composer into an almost mythical figure; a divinely-inspired creator passing down a perfected piece of music to the interpreter in the mediated form of a score. This leads to a concept of the musical work as somehow being embodied by the score as the primary and eternal document, relegating any given performance to a mere shadow of its ideal form.

-The Golden Age of Capitalism: In the decades after World War II, Western music notation experienced perhaps its most radical period of transformation, fuelled by the collective individualism of modernism (composers being compelled to create idiosyncratic compositional systems ex nihilo) and a reaction against the inherited norms of a culture that came close to self-annihilation twice within the space of 30 years. The 1950s and 1960s became a age of experimentation with various forms of graphic notation that reimagined how musical ideas could be represented visually. This period coincided with the commercialisation of the magnetic tape, which democratised the recording and distribution of music and implicitly questioned the need for notation to retain its function as a way to faithfully inscribe musical ideas in a durable medium.

-The Digital Revolution: The technological advances of the late 20th Century brought with it another wave of democratisation in the creation and reproduction of music. The proliferation of computer-aided compositional tools has returned music-making to a more immediate form of cultural practice (something that was never lost in various folk music traditions), where the triadic relation of composer-score-interpreter is no longer the norm. If the creation and recreation of musical works once more becomes an integrated activity through the rise of self-sufficient composerperformers (acting individually or collectively), music notation may ultimately lose its use value and become obsolete; a situation that is already a given for practitioners of improvised music.

A Brief History of Notation

Ancient Greek music notation followed a process of development from ideographic signs to a phonetic system that worked combinatorially to produce larger units, much like the alphabet. This phonetic scheme represented each pitch with a unique symbol, and included a different set of symbols for vocal and instrumental music. In other words, the 'same' pitches in different octaves were represented by different symbols, and vocal and instrumental parts represented the 'same' pitch with a different symbol. Greek notation ceased to be used around the 4th Century, and was replaced gradually by neumatic notation (from the Greek πνεῦµα, 'breath' or νεῦµα, 'sign'), which was derived from Ancient Greek pitch accent diacritics.

The earliest records of music notation using neumes come from the 9th Century, during the reign of Charlemagne, which also saw the birth of simple polyphony. This notation was used as a mnemonic device for the study of plainchant, and for a long time was not completely standardised; each monastery having its own variations on the generally adopted system. The neumes were designed to show the relative shapes of musical phrases, and relied on a choir master to give the initial pitch and gesture the size of the intervals.

The neumatic system followed a trajectory of gradual increase in its specificity, complexity and standardisation, through the invention of heightened neumes to show intervals, the addition of (initially 4) horizontal guidelines to show relative pitch movement more clearly, and the inclusion of an opening clef to show the main pitch of the chant. This process of greater precision in the pitch domain (imagined as the vertical axis) was accompanied by parallel developments in the rhythmic domain (imagined as the horizontal axis). Rhythms were implied by natural speech patterns until around the 10th Century, when 'longa' (long) and 'breve' (short) rhythmic notation were incorporated into scores, followed by modal rhythmic patterns based on Ancient Greek lyric metres (Iambic, 'short-long', and Trochaic, 'long-short') in the 11th Century, which employed ternary rhythmic units thought to symbolise the holy trinity. From the 13th Century, plainchant became ever-more melismatic, requiring new notational strategies, including ever-smaller rhythmic divisions and a five-line stave to fit the wider range of melodies.

The ars nova and ars subtilior musical styles of the 14th Century revised some features of the neumatic system, facilitating the development of a florid type of polyphony where each vocal line demonstrates a high degree of independence. Some of the notational inventions of the period were time signatures and barlines, visual groupings to simplify reading, the use of colour notation for certain rhythmic features, and progressive further division of rhythmic values. There was a general push from contextual and relative notation to absolute notation in both pitch and rhythm, which allowed for more abstract compositional techniques to be employed. Here the symbiotic nature of the evolution of musical ideas and their notation becomes quite transparent, as the new notational features lent themselves to the use of mathematical procedures such as augmentation, diminution, inversion and mirroring of both pitch and rhythm. Musical materials could now be viewed as a graphical puzzle, out of time, to be combined and permutated in various ways whilst maintaining a certain level of logic and symmetry; a logic that could possibly be heard and recognised by the expert, if not the novice, creating an aura of learnedness for those who could hear and understood these procedures in a performance of the musical work. Through such compositional strategies, the musical work became conceptualised as if it were a static physical object, with quantifiable spatial dimensions that could be rearranged and viewed from different angles; a notion that was analogous with the contemporaneous experimentation with perspective in painting.

There was also an interest in the general design and specific typography of musical scores, which at times resulted in graphically elaborate examples that compromised their role as a tool for the communication of musical ideas. Some such scores bear a striking resemblance to music from the second half of the 20th Century, which experienced a resurgence of interest in the visual design of music notation. These two eras share many similarities in the general tenor of their musical practice, both overseeing an increasingly multilayered complexity based on a reappraisal of the notational system through the prism of mathematical logic. Whereas in the 21st Century, this complexity has diffused into a plethora of idiosyncratic compositional techniques, the 15th Century experienced a trend back towards simplicity, as the new notational system ossified into an accepted standard across Europe.

During the Renaissance, instrumental music took hold in the European courts and amongst the rising bourgeoisie. Aided by the ease of reproduction and distribution of scores afforded by the printing press, composers began creating large repertoires of music for newly invented instruments, such as viols, that were used to accompany social dancing. These modern instruments utilised the 5-stave vocal staff, with absolute pitch notation, while older instruments such as the lute still used tablature notation, with the resultant pitch being relative to specific tunings.

While the historical trend throughout the Renaissance was for notation to gradually morph into a fixed, absolute system, certain discrepancies between written music and its sonic rendering remained. For instance, the prevailing performance practice dictated the chromatic alteration of identically notated pitches in specific contexts, in order to achieve more aesthetically appealing horizontal (voice-leading) or vertical (harmonic) results. These inflections were known as musica ficta, 'false' or 'fictitious music', falling outside the 'true' notes of a given mode; the notational invariance of these musical structures belying their malleability as sonic material.

In the process of the old church modes coagulating into the tonal system of major and minor scales towards the end of the Renaissance, the practice of such pitch alterations came to be systematically codified into written notation. This resulted in situations where the 'same' pitch occurred simultaneously or nearsimultaneously in two 'contradictory' versions (for instance 'C' and 'C#') in different voices or contrapuntal lines; one example of a phenomenon known as false relation. This conceptual clash (a non-identical identity), made explicit by the demands of standardised notation, demonstrates a fundamental problem with the evolution of a relative notational scheme into an absolute one.

The entire logic of the former is predicated on a specific performance practice (sacred vocal music), which necessitates the adoption of various ad hoc measures in its recasting as a generalised notational scheme; a permanently makeshift construct that can never completely hide the inherent contradictions at its core.

During the Baroque, a shorthand script for chord structures, called figured bass, was introduced into the notational scheme.

This system of numbers, which was written below the bass part of a piece of music, provided a harmonic skeleton that was fleshed out by the keyboard player with various idiomatic ornamentations, reflecting the role of limited improvisation that was central to the performance practice of the time. Similar chord-number notations are still in use in types of music that maintain a level of specialised extemporisation as part of their language, for instance jazz and country music.

The Baroque period also introduced explicit dynamics into music notation, concurrently with the invention of the pianoforte, the first keyboard instrument able to play with dynamic variation.

Such markings became more widespread and explicit during the Classical and Romantic Eras, including 'impossible' (or psychological) instructions such as a crescendo on a sustained piano note, which were designed to convey to the performer a more detailed sense of the composer's intentions; developments that went hand in hand with the waning of improvisation as an essential part of the performance of composed music. This was concomitant with a creeping sense that the score, however precisely notated, could never fully express the true essence of the composer's musical ideas. The positivism of the score as an eternal mirror of the composer's vision was thus slowly being eroded by the very process of its becoming an increasingly sophisticated and nuanced expressive tool; as if the ability of notation to generate and reflect more complex musical ideas was in fact directly linked to self-destructive doubt about the solidity of its foundations.

In the Romantic Era, many instructions such as speed and expression markings began to be written in the composer's own language, instead of Italian as had been the custom, coinciding with the burgeoning nationalistic sentiment across Europe towards the end of the 19th Century. This portended the impending fracturing of the universal notational scheme and paved the way for the birth of modernism in music around the turn of the 20th Century, inspiring ever more individual attempts at systematising the dense and complex chromatic tonality that had been passed down by the Romantics.

One such system, serialism, renewed the ars nova fascination with symmetry, abstracting pitch and rhythm into matrices of numbers (such as magic squares) that could be manipulated without reference to temporality and then used as raw materials for composition. These permutational schemes recall anagrams and other wordplays, such as the 'Sator Square' of classical antiquity, which intriguingly combines strict (mirrored) graphemic symmetry with a meaningful (if trivial) sequence of words. Such puzzles can only be constructed in a written form, demonstrating an uncanny independence of the script from its sonic counterpart, whether as speech or music; as if the written sign were inhabited by a ghostly secondary logic that mere sounds could only ever brush up against but never fully embody.

There were also attempts at creating new notational systems that weren't based on tonality, instead setting out to show features such as chromatic movement without the use of accidentals and attendant contradictions like the false relation. It was becoming clear that the logic of the inherited notational scheme was unable to support some of the new ideas in music, its internal logic (a relic of medieval modal music) becoming outgrown by experiments in areas such as microtonality, aleatorism and the use of new technologies, including the magnetic tape and electronic sound generators and processors.

After World War II, many composers began developing increasingly idiosyncratic notational systems, which compromised the universality of notation as a simple and effective means of communicating musical ideas. This proliferation of notational strategies reflected the plethora of new aesthetic and conceptual approaches to music, including many that stood in polemical opposition to one another. As a consequence, the universal standard of the five-line staff gave way to ad hoc solutions that required specific instructions and detailed study for the interpretation of each sign, which were in many cases not applied consistently by different composers, or even by the same composer across different works.

Some of the new notational strategies involved: the addition of staves for information that could not fit on the standard stave; various text scores that described actions and processes; diagrammatic scores for building a sonic situation or object; tablature notation for instruction of physical actions on instruments; using visual artefacts such as painting, film or sculpture as a 'score' that demanded a significant level of interpretation from the performer on how to translate these visual stimuli into musical material; algorithms and punch cards as scores for mechanically or electronically produced or reproduced music; and computer-generated notation that could transcribe elements of live performance into digital data, which could then be decoded for use with other media.

The Fall of Notation The tradition of Western music notation, which had forged a sufficiently close relationship between the written mark and a corresponding sound to be considered a functioning notational symbol scheme (in philosopher Nelson Goodman's terminology), lost its unquestioning representative fidelity in the face of the postWorld War II avant-garde's challenge of the foundations of its historical continuum. This crisis heralded an age of deconstructive experimentation with notation (and the very concept of the musical work), leading to the widespread employment of different types of graphic scores, which could incorporate a broad range of visual signs from traditional notation to text, pictographs and abstract geometric shapes. The vital question of notation became less about how to create strict correspondence between a visual and sonic sign, and more about how different notational strategies could instigate new modes of music-making.

These developments resulted in notation functioning not just as instructions for performance, but also as an independent aesthetic object in its own right, with serious repercussions for the ontological status of the musical work; if the score is an object of aesthetic contemplation itself, in the visual domain, it severs its strict contract with its sonic counterpart, with which it traditionally had merely a regulatory relationship. In other words, the aesthetic effect of a musical work previously relied ultimately on its performance (whether imaginary or real), whereas a graphic score marks it as oscillating vertiginously between the visual and the sonic realms.

Graphic notation has a much more tenuous relationship with its sonic instantiation than conventional notation, as it may be difficult to perceive a correlation between a score and its performance. There is no longer a one-to-one representative scheme at work, and the score can no longer act out its normative function, as a prescriptive and evaluative document (a law); it can not be used to judge a performance, as there are no clear criteria that determine success or failure. It becomes more of a ludic contract between the composer and the performer, allowing the latter a much greater role in the interpretation, and indeed the composition, of the work.

(As a thought experiment, one can imagine two performances of the same score yielding versions where no single element could be perceived to be replicated, and performances of two different scores that sound identical.)

Many graphic scores in reality feature elements that retain some general characteristics of conventional notation, as well as instructions that refer to very specific actions, even if they may be open to varying sonic outcomes. (Notation of actions rather than sounds results in a choreography of gestures that confuses the concept of the musical work as a sonic phenomenon, recontextualising it as a much more open performative form that begins to converge with other disciplines, such as theatre and dance.) For instance, a common way to conceptualise the twodimensional space of the score is still to see it mapped onto an invisible Cartesian axis, with pitch as the vertical and time as the horizontal domain. These scores still appeal to an intuitive visual logic in the arrangement of the musical ideas on the page and remain faithful to the original metaphor of conventional notation that translates musical phenonema into spatial relations.

The relationship between a graphic score and its performance could be seen to be akin to that of a genotype to a phenotype, the score acting as a kind of genetic code that can be expressed in different instantiations with different traits that cannot be fully predicted beforehand. The score thus specifies a potential for a unique situation that involves an element of chance beyond the composer's control. This creates a new paradox for the roles of the composer and performer, and the ontological status of the musical work. On the one hand, the composer retreats from the dictatorial role of controlling all the musical parameters, allowing (forcing) the performer to express a greater level of agency and spontaneity in the study and performance of a work. On the other hand, the composer still maintains authorship of the musical object, and becomes in some ways an even more mythical figure in the process, circumscribing the performer's liberty with his invisible wand. (The composer frees the performer from the bonds of notation, but binds the performer's freedom to the composer's name.)

The Paradox of Notation (2)

Notation performs a double sleight of hand that detaches music from its essence as a concrete sonic (and social) phenomenon; firstly by abstracting sound from its spatio-temporal reality (through transcription), and secondly by reifying musical ideas in a physical object, the score (through composition).

The development of notation as a mnemonic device begins as a way to record a musical work through transcription; this necessarily involves a process of filtering only the elements deemed to be essential to the work's reproduction (ignoring all others), and the invention of a spatial metaphor for their representation.

As these essential musical elements are mapped onto a twodimensional axis with pitch as the vertical and time as horizontal, a further abstraction takes place; a concept of music as an idealised set of geometric proportions that are represented by this method of notation.

Notation becomes not only a way to transcribe sounds, but a generative and normative principle for new musical ideas and new ideas about music; it bends the composer's imagination into its own form.

Notation not only facilitates thinking about music in abstract terms; it demands it.

(This produces the fallacy of imagining a direct connection between a set of spatial proportions and the experience of a sonic phenomenon.)

Composition becomes the construction of idealised spatial symmetries using the elements of the given notational scheme; these are reified in the musical score, which acts like a fetish, magically embodying the composer's creative act in its geometric designs.

The score-fetish, existing outside the temporality of musicmaking, reveals itself as the new mythical locus of the musical work; a self-contained, complete and perfect object traceable to a single point of origin, its author.

(The score-fetish appeals to our desire for certainty; it clarifies our social roles as a composer, performer, audience member or musicologist and vouches for the quality of the musical work.)

The score-fetish simultaneously conceals its own contingency on a specific concept of music and set of social relations that uphold its status as an immutable, magical object; it feigns ignorance of the possibility of a different way of music-making and its own role in upholding and perpetuating notation as a normative ideological construct.

(The score-fetish replaces the fleeting concreteness of sound as phenomenon with its own solid, comforting concreteness as a physical object; it reassures us that the musical work really exists.)

Erkki Veltheim