Synaesthesia refers to the crossing of senses that affects an estimated 1 in 2,000 people, including the visually impaired and even the blind (Kher, 2001; Nold, 1997). In synaesthesia, the stimulation of one sensory system induces a perception in a different, physically unstimulated sensory system (Wager, 1999). For example, as cited by Wager (1999), Cytowic reported on a synaesthete who perceived the touch of a glass column when he tasted mint. The variations and combinations of senses are numerous in synaesthesia, with no two synaesthetes reporting the same associations (Kher, 2001). A music piece may induce a visual scene of color explosions for one person while inducing a scene of colored waves for another. In color-hearing synaesthesia, an auditory stimulus creates a visual perception. In other words, when color-hearing synaesthetes hear different sounds, they perceptually see different colors and shapes that are not physically presented to them.

Scientists have not yet discovered what causes synaesthetic perception. One theory is that synaesthesia is the norm in lower vertebrate animals; it is a quality that humans have left behind in the process of evolution (Freeman, 1998). Most researchers, however, seem to agree that the answer to the question lies in the concept of modularity. The human brain is efficiently organized so that specific mental tasks are processed in specific areas of the brain. Each of the sensory systems has a corresponding area, or module, of the brain. While there are normal interactions between the brain activities of different modules, as when using vision to detect movement, the current understanding of human perception cannot account for the stimulation of one module inducing brain activity in a different module.

There are conflicting theories of modularity as the source of synaesthesia. Lloyd (1996) describes the brain activity of a synaesthete as "confused," but not as the intermingling of different modules. In contrast, Gray (2001) proposes two possible models of modularity that may explain the occurrence of synaesthesia: (1) a breakdown in normal modularity, or (2) an extra module designed to process more than one sensory modality that occurs in only some individuals. Both of these possibilities would, theoretically, be feasible because neither circumstance would violate the current understandings of the rules governing the modularity of human brains. Anderson (1998) proposed the idea that some neural modules are not innate&emdash;they can be learned. If supported, this theory would suggest the possibility that synaesthesia develops as the result of an extra, learned module. There is no theory regarding the period of life in which this extra module would develop since all synaesthetes report having experienced their synaesthetic perceptions as far back in time as they can remember (Cytowic, 1995).

The experience of color-hearing synaesthesia and imagery can be compared in that both are a visual representation of an unreal stimulus. The difference is that, in synaesthesia, the images are actually seen outside of the individual, whereas in imagery they are merely imagined within the individual's mind (Cytowic, 1995). Brandt and Stark (1997) show in their study that patterns of eye-movements are similar when an image is imagined, and thus not physically presented, as when the image is actually viewed. Researchers already know that the eyes follow a systematic pattern of movement when viewing an image. It is also understood that most people follow a systematically similar pattern of eye movement when scanning the same image. These relatively standardized patterns of eye movement are the brain's way of organizing an image (Brandt & Stark, 1997). Since eye patterns are indicative of image content during imagery tasks, it is reasonable to believe that synaesthetes will also exhibit eye movements representing the content of their visual scene, even though the images they see are not physically present.

Several methods of tracking eye-movements have been used in research (Mulligan, 1997). Choosing a single method of eye-tracking involves the consideration of many advantages and disadvantages associated with each available system. Most commercial systems are expensive and invasive because they involve the use of head gear, chin rests, and bite-down armatures as well as requiring that the head be stationary during testing ( ; Krugman & Fox, 1994). Currently, no system is appropriate for use in every research situation. Of these disadvantages, the largest obstacle facing the presently available commercial systems is their inability to separate head movements from eye movements (Mulligan & Beutter, 2002). This limitation means that these systems can only be used in studies where the participant's head is immobile, prohibiting research of eye movements in realistic situations. Research on the eye-movements in infants or young children is also limited because of the difficulties involved in keeping maintaining head immobility in very young participants while awake. A few of the recently developed techniques of eye-tracking have overcome these obstacles, including the Applied Science Laboratories Model 425OR eye tracker and the use of compressed video images (Krugman & Fox, 1994; Mulligan & Buetter, 2002).

Eye movements are an observable behavior resulting from the subjective experience of perception. Richardson (1999) explains that subjective experiences will produce observable behaviors. In the case of this study, the subjective experience of extra-sensory, synaesthetic perceptions should produce observable eye-movement patterns different from the patterns of eye movements in non-synaesthetic individuals. Using this logic and the knowledge that imagery will produce patterns of eye-movements similar to those of actual vision, it is reasonable to assume that color-hearing synaesthetes would exhibit a systematically different pattern of eye movements than non-synaesthetes receiving the same visual and auditory stimulation. With this assumption in mind, it is hypothesized that color-hearing synaesthetes will have significantly more visual fixations as well as a significantly higher number of areas of visual interest, or areas with many visual fixations than non-synaesthetes.

Participants

Twenty non-synaesthetic participants will be recruited from the local university and will receive class credit for their participation. Twenty color-hearing synaesthetic participants will be recruited through the American Synaesthesia Association and will be paid for their participation. The number of participants will be relatively low due to the expensive and time-consuming procedures available for the use of measuring eye movements (Krugman & Fox, 1994). The participants are expected to range in age from 20 to 30 years old. Approximately 20% of the participants will be male and 80% of the participants will be female. More females will be studied because synaesthesia is more common in females than in males (Cytowic, 1995). The sample will include a number of different ethnicities as well as a number of different socioeconomic classes.

Materials

The materials used to stimulate the senses will be a black and white photograph of a woman's face and tape recording of "Romance," the second movement from Wolfgang Mozart's Eine Kleine Nachtmusik. The auditory stimulus will be heard through earphones connected to a tape player.

The eye movements will be monitored and recorded using the Applied Science Laboratories Model 425OR eye tracker (Krugman & Fox, 1994). This system reports points of visual fixation without the use of head-worn equipment, meaning participants are free to move as they might move in the normal environment. Participants will sit in front of a screen on which the woman's photograph will be projected from a projection unit located beside the participants' seat.

Procedure

Participants will first be asked to view a photograph of a woman's face for 90 seconds while receiving no auditory stimulus. This is to establish a baseline and to ensure that the any difference in the number of visual fixations or in the number of areas of visual interest is due to the added auditory stimulus and not due to innate differences between the two groups of participants. Participants will wear the headphones in the trials where no auditory stimulus is presented to ensure that neither group of participants is exposed to noises within the room. Extra noise exposure could increase the number of visual fixations and the number of areas of visual interest in the synaesthete group, giving a false baseline and possibly implying a difference between the groups when no auditory stimulus is presented. Next, participants will be asked to view a second, similar photograph of a woman for 90 seconds while listening to "Romance" through headphones. Points of visual fixation will be recorded during both the baseline measurements and the experimental measurements.

The data will be analyzed using a between subjects 2x2 ANOVA procedure. The two variables will be synaesthete condition (whether the participant was synaesthetic or non-synaesthetic) and presence of the auditory stimulus (whether the auditory stimulus was present or not present).

The difference in the number of visual fixations (see Figure 1) will be analyzed by comparing the mean number of visual fixations in synaesthetes to the mean number of visual fixations in non-synaesthetes. There will be no significant difference between synaesthetes and non-synaesthetes for the baseline measurement of the number of visual fixations when auditory stimulus is not presented to the participants. It will be found, however, that color-hearing synaesthetes have a significantly higher number of visual fixations than non-synaesthetes when asked to view the photograph while the auditory stimulus is presented to participants.

Figure 1: A schematic showing fixations.

The same statistical analysis will be applied to analyze the difference in the number of areas of visual interest. For our purposes, areas of visual interest are the areas where the most densely packed points of visual fixation are grouped (see Figure 2). When no auditory stimulus is presented to participants, there will be no significant difference between synaesthetes and non-synaesthetes for baseline measurement of the number of areas of visual interest. In contrast, synaesthetes will have a significantly higher number of areas of visual interest than non-synaesthetes when auditory stimulus is presented.

Figure 2: A schematic showing a areas of visual interest.

Discussion

When viewing the photograph alone during baseline measurements, both synaesthetes and non-synaesthetes will show a similar number of visual fixations. This indicates that there are no inherent differences between synaesthetes and non-synaesthetes that should cause them to have different patterns of eye movements without the extra sensory stimulation. When the musical stimulus is also presented, however, synaesthetes will exhibit significantly more visual points than non-synaesthetes as well as a significantly higher number of areas of visual interest than non-synaesthetes.

Although the relationship implied by this study is correlational and not a cause-and-effect relationship, it is assumed that synaesthetes show extra visual points and extra areas of visual interest because they experience the visual stimulus, common to both groups, as well as visual sensations induced by the auditory stimulus. This supports Richardson's theory that all subjective experiences will produce observable behaviors (Richardson, 1999). The subjective synaesthetic experience will in fact induce an observable behavior, described in this study as patterns of eye movements.

These results will add to the empirical evidence that synaesthesia is a real phenomenon. Previously, the only objective means of measuring sensory experience was through brain imaging. Eye tracking is a less expensive and more accessible method of objectively identifying synaesthesia than brain imaging techniques.

Tracking eye movements can also make the identification of color-hearing synaesthesia in individuals more systematic. Currently, identification of synaesthesia is based solely on subjective reports. I dentification may be possible through the use of brain imaging, but these techniques are expensive and, in many circumstances, inaccessible and unrealistic. The compressed video images technique of tracking eye movements is a relatively inexpensive alternative with the necessary PC software available for less than $1,000 (Mulligan & Beutter, 2002). This technique would also make it easier to identify synaesthesia in people who cannot give subjective reports, as in the cases of young children or adults without speech.

Accurate and inexpensive identification of synaesthesia can benefit scientists who research the phenomenon. Because synaesthetic research participants are often paid, it is possible that non-synaesthetes can claim to have synaesthesia and participate in research for the money. Using eye-movement tracking to identify color-hearing synaesthesia can prevent this relatively easily and inexpensively, which would in turn help researchers to be sure that they are actually dealing with color-hearing synaesthetic participants.

Because the study of synaesthesia is still relatively new, several areas of research are still open for scientists to pursue. It would be interesting to see research done comparing the eye movements of different synaesthetes. Synaesthesia is currently described as being unique in experience from one synaesthete to another, but this is based on the subjective reports of synaesthetes (Kher, 2001). Objective observations through the use of eye-tracking could provide empirical data that would either support the current theory that synaesthesia is unique to each individual or show that there are in fact systematic similarities between individual color-hearing synaesthetes that could not have been previously recognized.

The use of eye-tracking as a means of identifying synaesthesia could also help researchers to determine when synaesthesia develops in synaesthetic individuals. Since synaesthesia is proposed to be inherited from synaesthetic parents, further research could include longitudinal studies of synaesthetes' offspring to determine when in life their children develop synaesthesia (Cytowic, 1995).

Also of interest for future research would be the adaptiveness of synaesthesia. How does synaesthesia give synaesthetic individuals an advantage over non-synaesthetes? Or, if Cytowic's theory that synaesthesia is a trait that humans have evolved away from, what advantage do non-synaesthetes have over synaesthetes? This is a topic that has received little attention because there is no empirical data on the subject. Research comparing synaesthetes' and non-synaesthetes' abilities to perform different tasks may show minor differences.

References

Anderson, M. (1998). Ask not what can you do for modularity but what can modularity do for you. Learning and Individual Differences, 10, 251-258.

Brandt, S. A., & Stark, L. W. (1997). Spontaneous eye movements during visual imagery reflect the content of the visual scene. Journal of Cognitive Neuroscience, 9, 27-38.

Cytowic, R. E. (1995). Synesthesia: Phenomenology and neuropsychology: A review of current knowledge. Psyche, 2. Retrieved April 20, 2002 from http://psyche.csse.monash.edu.au/v2/psyche-2-10-cytowic.html

Freeman, W. J. (1998). The neurobiology of multimodal sensory integration. Integrative Physiological and Behavioral Science, 33, 124-128.

Gray, R. (2001). Cognitive modules, synaesthesia and the constitution of psychological natural kinds. Philosophical Psychology, 14, 65-82.

Kher, U. (2001). Ah, the blue smell of it. Time Atlantic, 157, 64.

Krugman, D. M., & Fox, R. J. (1994). Do adolescents attend to warnings in cigarette advertising? An eye-tracking approach. Journal of Advertising Research, 34, 39-52.

Lloyd, D. (1996). Consciousness, connectionism, and cognitive neuroscience: A meeting of the minds. Philosophical Psychology, 9, 61-89.

Marks, D. F. (1999). Consciousness, mental imagery and action. British Journal of Psychology, 90, 567-585.

Mulligan, J. B. (1997). Image processing for improved eye-tracking accuracy. Behavior Research Methods, Instruments, and Computers, 29, 54-65.

Mulligan, J. B., & Beutter, B. R. (n.d.). Eye-movement tracking using compressed video images. Retrieved April 8, 2002 from http://vision.arc.nasa.gov/personnel/jbm/home/mulligan.html

Nold, M. G. (1997). Synesthesia and blindness: A personal account and informal survey. Journal of Visual Impairment and Blindness, 97, 14-16.

Richardson, A. (1999). Subjective experience: Its conceptual status, method of investigation, and psychological significance. Journal of Psychology, 133, 469-485.

Wager, A. (1999). The extra qualia problem : Synaesthesia and representationism. Philosophical Psychology, 12, 263-281.