Acknowledgements: This paper summarizes a class research project. In addition to the above authors, John Aston, Molly Daniel, Sophia Hussein, Jeffrey Oldenkamp, Ugochi Onyejiaka, Taylore Sloan, and Todd Ward also made meaningful contributions to the project.
Return to class web page: http://hubel.sfasu.edu/courseinfo/SL03/sl2003front.html
Abstract
The purpose of this experiment was to determine how email text format and instructions affect reading times, content retention, and reader preference, where text format is as follows: all capital letters (AC), normal mixed case (MC), capital letter emphasis (CE), and spacing emphasis (SE). Fifty-seven students from an East Texas university were presented with twenty adapted, non-classified U.S. NAVY emails; approximately half of this group was instructed that they would be asked questions about the content of each email, and all participants were asked subjective questions about format preference. A Web-based computer program calculated the time participants spent reading each email and collected answers to all questions. It was hypothesized the AC format would result in the slowest reading speeds, the most incorrect answers, and be rated as the least preferred format. ANOVAs indicated neither text format nor instructions had significant effects on reading times or content retention; however, Chi Square analyses indicated readers liked text written in all upper case letters significantly less than the other formats, and that it was harder to find important information when using all capital letters. Thus, while the use of all capital letters does not significantly affect accuracy or reading times, it may affect readers' likelihood to read an email in the first place.
Electronic messages are an increasingly important form of communication. One major benefit is the immediacy with which information is transmitted. Thus it is easy to understand why time-sensitive information, such as meeting times and locations or military orders, would be best disseminated via email. However, if the message text is not easily readable, and the reader must spend additional time deciphering the content, the time-saving aspect of electronically sent information is lost. Readability is the ease with which a text may be comprehended (Samuels, 1983). Factors external to a reader, (e.g. text topic and structure, environmental characteristics, and the goal imposed on the reader by an external source), and factors internal to a reader, (e.g. background experiences, knowledge base, and awareness of text structure), dually affect the reading process (Samuels). Internal factors could be further categorized into physical and cognitive influences. For example, the structure of the eye, visual cortex, and other cortical areas involved in reading vary with the individual's learning style, word familiarity, and grammar usage to influence the reading process. The scope of the current research is limited to the effects of external factors, (specifically letter case, text format, and "imposed goals") on reading times, comprehension and reader text preference; thus no physiological functions will be measured.
The current study stemmed from sample U.S. NAVY emails; all NAVY emails are written in exclusively capital letters, and paragraph spacing is not always preserved upon receipt of a message. The primary purpose of this research was to determine if this inherent writing style affects the time a reader needs to finish reading an email as well as the ability to identify important information presented in an email. Secondly, readers' preference of text style was determined. Understanding these factors is important, especially when little time is available to decipher an email message, as might be the case in hostile or unexpected military situations.
According to Samuels (1983), there is an interaction among external factors to influence overall reading speed, comprehension, eye movements, and reading strategies. These factors include: illumination, print size, font style and legibility (letter recognition), format design (page width, size, placement of text, etc.), the use of titles and headers, text style (word frequency, sentence construction, and text construction), and the goal or direction imposed on the reader. A reader whose text has a less legible font style but important information separated spatially, may have an equivalent reading speed as another reader whose text has a more legible font but unorganized content. Because this interaction is present, the factors influencing reading speed, comprehension, and reader text preference are not segregated for discussion. All information below refers to sighted readers with normal vision.
Historically, documents printed in English have been in mixed case (some upper and lower case letters), thus learning and continuing to read effectively requires some ability to recall differences between letters of different cases. MacKeben (2000) stated that without good letter recognition, reading is severely impeded. With frequent exposure, the perceptual system becomes tuned to font regularities in order to effectively process visual information (Sanocki, 1987 & 1988 as cited in MacKeben). These regularities act as cues in word recognition, and can include letter ascenders (vertical extensions above the letter body), descenders (vertical extension below the letter body), curves, dots, and overall word shape. The shape of a lower case word provides significant clues to the reader as to its identity, whereas a word in all capital letters has a destroyed shape (no ascenders or descenders), so the word appears to be rectangular (Williams, 2000). When unable to use previous experience and knowledge of word shapes, reading time slows as a result of slowed processing. Upper case type is read about 13 percent slower than lowercase type (Tinker, 1955 as cited in Williams).
Additional problems arise when reading text presented in exclusively capital letters due to the amount of similarity between these letters. Briggs and Hocevar (1975) conducted a study in which the letters of the capital letter alphabet were defined based on common and distinctive features such as horizontal angularity (horizontal line at the top, middle, or bottom of letter) and curvature (small curve-convex right, closed curve, etc). Each letter was tested for confusability with all other letters. Their study showed more effort was required to decipher the differences between two capital letters, because they are more similar to each other (have fewer distinguishing features). Staats (1968) too suggests the reading process is affected by the difficulty in learning to distinguish between similar letters in the English alphabet.
Finally, upper case letters are simply larger than the corresponding lowercase letters. With each eye fixation, a limited amount of material can be resolved, recognized, and used to guide the reading process. Letter distinctions can be made for four to six letters from the fixation point, but seven to fourteen letters away could be useful for planning the next eye movement, or saccade (McConkie & Rayner, 1975 as cited in Bailey et al., 2003). Thus, the size of the text can influence the efficiency with which the visual system combines previous information with current information and plans to incorporate new information by controlling eye movements. Interruption of this process with the introduction of difficult-to-read text may influence the frequency of eye movements and could ultimately affect reading speed.
In addition to text case affecting reading speed, text organization can affect reading speed and content retention. For example, aesthetically pleasing visual organization and design can impact cognitive learning (Martin, 1986 as cited in Haag & Snetsigner, 1994). In fact, Samuels (1973) believes text construction/coherence is the most important aspect of readability. In describing information presentation on web pages, Williams (2000, pp.384) stated "Good design reveals structure when it visually mimics the logical relationships that exist among elements…the human visual system attempts to find the structure of information… it does so by looking for visual patterns." Logical relationships are assigned based on a text's visual characteristics, and readers assume placement of any element on a screen as intentional and as a result, try to assign meaning to it (Williams). Williams identifies spacing as an important tool for organizing information, suggesting the visual system tries to group elements that are close together or separated by white space. This spacing simulates a "blocking" visual effect, helping the reader break down the information into smaller more manageable pieces of information, and in turn can facilitate better content comprehension. In addition to manipulating text case, the current study will also include conditions where some text is emphasized using either all capital letters or spacing. This allowed us to examine whether the use of selective text emphasis would affect both reading speed and retention of emphasized and non-emphasized information.
As discussed above, text readability can be affected by an interaction of many format and organization factors. However, few studies have focused on how more subjective influences (i.e. reader preferences) impact text readability. Gump (2001) conducted a study in which participants rated various fonts as either easy or hard to read and indicated the "mood" created by each of the fonts. While ninety-eight percent of participants rated Arial as easy to read, approximately seventy percent of participants rated Arial as plain. The findings of this survey suggest differences in assessments of font readability and basic font aesthetics. As noted above, text organization and layout influence aesthetics and retention. Thus, it is possible that basic font styles and letter formats (all capital letters versus normal mixed case) may also influence retention and reading speed simply because they influence a reader's subjective responses. To determine if text format affects readers' subjective responses as well as objective performance (reading times and retention), subjective questions will also be included.
Finally, reader motivation will be manipulated in order to determine its influence on reading times. Previous research on reading comprehension indicates the purpose of the reading task may dictate how quickly a participant reads text; thus affecting overall reading performance. Samuels and Dahl (1975, as cited in Samuels, 1983) found that when participants were issued reading tests and asked to read for a general as opposed to detailed information, large differences in reading speed and amount of information learned were produced. Because participants did not know how detailed the post-reading questions would be, they did not know whether to read quickly or carefully (Samuels & Dahl). This research suggests if a reader is looking for specific information in preparation of test questions, more time is required in gathering more information for retention. Therefore, one half of the current participants will answer retention questions following each email, while the remaining participants will read the emails without answering any retention questions.
Due to word shape unfamiliarity and a resulting difficulty reading all upper case text, we hypothesized participants would yield significantly slower reading times and more incorrect answers when reading all upper case text than when reading the emails presented in mixed case. We anticipated that participants who knew they would be questioned about the email content would have significantly slower reading times than participants who would not be questioned, because they would spend more time reading and trying to remember details about the text. Additionally, we hypothesized that participants would have faster reading speeds and more correct answers when answering questions about emphasized text, than when answering questions about text with no distinguishing features. Finally, it was hypothesized that text written in exclusively upper case letters would be the least preferred style as a result of this format being the most difficult to read.
Participants
Sixty participants from a mid-sized, East Texas university were recruited from the Psychology department student subject pool. A majority of the participants in this study were between 18 and 20 years old. Participants were compensated with a one-hour experiment participation credit and were required to have 20/20 vision normally or corrected.
Design and Materials
The current research employed a 4x2 mixed design. Because the sample texts were adapted from NAVY emails written in Arial 10 point type using black text on a white background, the current test conditions used the same font, size, and colors. The independent variables were Text Format (all capitals (AC), mixed case (MC), capital letter emphasis (CE), and spacing emphasis (SE)), and Instructions (participants questioned about the material they read (Q), participants not questioned (NQ)).
The following describes each level of the Format variable: AC- email text written exclusively in upper case letters with single blank lines between paragraphs (ex: THIS IS JOHN'S BOOK); MC-email text written in standard format (some lower and some upper case letters) with single blank lines between paragraphs (ex: This is John's book); CE-email text written in standard format with four sentences or phrases in upper case letters and single blank lines between paragraphs; SE-email text written in standard format with an additional blank line separating the same information capitalized as in the CE conditions.
Twenty NAVY emails were each adapted to the four formats described above (AC, MC, CE, SE). Each participant was presented with twenty emails; five in each of the four formats. The specific emails that each participant received in each format was counterbalanced across participants. Thus, eight versions of the program were created to eliminate any effects of email length and content difficulty. Versions one through four were identical to versions five through eight, however they did not contain questions. To minimize order effects within the experimental versions, the experimental computer program randomized the order in which the emails were presented to each participant.
All participants received all treatment combinations for the Format variable and only one level of the Instructions variable. The three dependent variables were reading speed, content retention, and the reader preference of text format. A participant's reading speed was measured in milliseconds and calculated as the time between the complete loading of a web-page that displayed a single email and the participant's request to go to the next page. The timer began as soon as a page was completely loaded (based on the browser and not the server from which the page was hosted) and ended when the participant clicked a button at the end of the email. Content retention was measured by the number of correct answers to questions pertaining to the text content. In the Q condition, each email was followed by four multiple-choice questions presented on the computer screen and answered electronically. To better estimate the effects of text case and emphasis on comprehension in the CE and SE levels, two of the four answers came from capitalized/spatially separated information, while the remaining two came from information not emphasized by capitalization or spatial separation. At the end of the twenty emails were four subjective questions about the readers' preference of text format. Participants indicated their most (SubQ1) and least (SubQ2) preferred formats and the format they thought was the easiest (SubQ3) and most difficult (SubQ4) to read.
The twenty non-classified NAVY emails were incorporated into a computer program executed via the World-Wide-Web. The emails varied in their length, with a minimum of 84 words and a maximum of 844 words (median of 273). A laboratory with 20 single-user computer workstations was used to collect data. Each computer was equipped with Internet access to the program developed for this study. The program collected information about each participant's reading time, content retention, and format preference.
Procedure
Before beginning each experimental session, the computers were turned on and one of eight versions of the data collection program was started on each machine so that the experiment instructions were displayed. After participants were seated, they were given consent forms and provided with answers to any questions they had. Participants were then given instructions emphasizing the relationship between the experiment and the NAVY, the importance of using three initials for data coding purposes, and that the on-screen instructions would indicate whether or not there would be questions after each email presentation. Participant then entered their initials and completed the twenty trials. At the end of all twenty trials, each participant was presented with the four subjective questions about their preferences among the four formats. Finally, participants were debriefed and provided with documentation of experiment completion. Participants took between 10 and 45 minutes to complete the experiment.
The scores for three participants were not included in the data analysis because they did not follow instructions. Of those remaining, 28 participants received experiment versions including questions and the remaining 29 received versions without questions. One ANOVA was performed for the dependent variable of reading speed and another was performed using question accuracy (content retention). For each participant for each condition, the median reading speed, the total number of correct answers from emphasized text, and the total number of correct answers from non-emphasized text were calculated; these values were used in the ANOVAs. Chi-square analyses were used to analyze the subjective data.
A 4 (AC, MC, CE, SE ) x 2 (Questions, No Questions) ANOVA revealed no significant effects of text format on reading times. See Figure 1 for a graph of mean reading times (seconds) for all conditions. Although it was not significant, there was a tendency for those who received questions to read emails using all capital letters or capital letter emphasis more slowly than those not receiving questions. Overall, emails using normal mixed case format were read most quickly.
Figure 1 - Reading Times (seconds) as a function of text format and instructions.
A 4 (AC, MC, CE, SE ) x 2 (Emphasis, No Emphasis) ANOVA also revealed no significant effects on accuracy. See Figure 2 for a graph of accuracy scores for all conditions. Although it was not significant, there was a tendency for the spacing emphasis to improve accuracy for the emphasized questions, but decrease accuracy for non-emphasized questions.
Figure 2 - Accuracy as a function of text format and instructions.
Before conducting Chi Square analyses on each of the four subjective questions, the answers were tallied for each question (see Table 1). Only 56 participants were included because one participant neglected to answer the fourth subjective question. All four questions showed significance at the p<.005 level. Participants most strongly preferred (easiest to read) the two formats using emphasis. The all-capital-letters format was the single least preferred format of the four format types. A similar pattern was found for the formats for which it was easiest to find important information (capitals emphasis and spacing emphasis), and for which it was the most difficult (all capital letters).
Table 2 - Tally of Answers to Subjective Questions.
|
Question 1 |
Question 2 |
Question 3 |
Question 4 |
|
|
|
Format Most Preferred (Easiest to read) |
Format Least Preferred (Hardest to read) |
Format Easiest to find Important Information |
Format Hardest to find Important Information |
|
AC |
4 |
46 |
5 |
46 |
|
MC |
7 |
6 |
4 |
5 |
|
CE |
22 |
2 |
27 |
1 |
|
SE |
23 |
2 |
20 |
4 |
Although no significant difference was found between groups with respect to reading times, there were in fact trends that were noticeable and worth discussing. E-mails that were formatted using normal mixed case tended to be read more quickly than the other three format options. Perhaps this trend existed because readers are more accustomed to reading mixed case text, and because the use of lower case letters allows word shapes to be used to help recognize words. In contrast, readers tended to take the longest to read text that was presented using all capital letters, where word shape cues were missing and familiarity was not as strong. It is also important to note the interaction trend between format and the presence or absence of questions. Readers took their time while reading e-mails in all caps or e-mails with case emphasis if they knew they would be asked questions. However the presence of questions had no effect on reading time of e-mails in mixed case or spacing emphases. These results suggest that text presented in capital letters may require more attention and focus in order to accurately retain information.
With respect to accuracy, the trend regarding the use of spacing emphasis is also noteworthy. It seems that while spacing emphasis somewhat helped readers accurately recall information of the emphasized material, it seemed to hinder their recall of non-emphasized material. It is possible that the emphasized material caused readers to pay close attention to that information and in the process of doing so may have caused participants to inadvertently disregard information that was not emphasized. Therefore, it is crucial when emphasizing material to be sure and emphasize all of the important material. A similar trend did not exist for the emphasis conditions using all capital letters. This lack of effect for the all capital letter emphasis may have occurred because these particular emails often contained Navy acronyms in all capital letters. Thus, readers may not have been able to efficiently scan for and use the all capital letters emphasis and, in turn, they effectively ignored it.
In contrast to the objective measures, participants showed strongly significant preferences for the different formats. Participants liked and found it easiest to find important information using mixed case with capitals emphasis and mixed case with spacing emphasis. Further, they specifically disliked and found it difficult to find important information using the all capital letters format. While they did not strongly like the normal mixed case, they also did not strongly dislike it. The fact that they least preferred the all capital letters format may be due to the fact that this is the most unfamiliar way to read text and that reading this format was more frustrating due to the lack of word shape cues.
Although readers did not perform significantly differently on comprehension tasks as a result of emphasized text, they still felt that it was easiest to find important information in a paragraph that contained emphasized sections. In the future, experimenters may want to require participants to read material with more difficult content. Emphasis may have more of an impact on the reading comprehension of more complex messages. Additionally, the use of the all capital letters emphasis should be tested using emails that do not contain acronyms.
The results of this project have implications regarding the Navy's use of the all capital letter format. Because subjective responses can influence the likelihood of reading and attending the text content, it might be beneficial for the Navy to consider changing its case format. The trend toward an interaction between format and answering questions about the email supports this conclusion. Participants only took extra time to process the all capital letters format when they knew they would be questioned over the content. Without questions, they spent the least amount of time reading emails using that format, most likely because they did not like reading that format. Because readers generally are not quizzed over e-mail content, the Navy might inadvertently be encouraging its personnel to superficially skim any administrative e-mails not prejudged to be important to the reader.
Because emails are so widely used, the results of this experiment also extend beyond the realm of the Navy. General users sometimes write emails in all capital letters, either to suggest emphasis (such emails are often interpreted as "shouting"), or because it is easier to type without using the shift key as often. Such emails may not be read as efficiently, or readers may choose not to read them at all. In general, it is important to know how to best present e-mails to optimize the transfer of intended information.
Bailey, I.L, Lueck, A.H., Greer, R.B, Tuan, K.M., Bailey, V.M., & Dornbusch, H.G. (2003). Understanding the relationships between print size and reading in low vision. Journal of Visual Impairments & Blindness, 97(6), 325-334.
Briggs, R. & Hocevar, D.J. (1975). A new distinctive feature theory for upper case letters. The Journal of General Psychology, 93, 87-93.
Gump, J. (2001). The readability of typefaces and the subsequent mood or emotion created in the reader. Journal of Education for Business, 76(5), 270-273.
Haag, B. B. & Snetsigner W. (1994). Aesthetics of screen design: an integration of principles. In D. G. Beauchamp, R. A. Braden, & J. C. Baca (Eds.), Visual Literacy in the Digital Age: Selected Readings from the 25th Annual Conference of the International Visual Literacy Association (pp. 92-97). Blacksburg, VA: International Visual Learning Association.
MacKeben, M. (2000). Enhancement of peripheral letter recognition by typographic features. Visual Impairment Research, 2(2) 95-103.
Samuels, S. J. (1983). A cognitive approach to factors influencing reading comprehension. Journal of Educational Research, 76(5), 261-266.
Staats, A. W. (1968). Learning, Language, and Cognition. New York: Holt, Rinehart, and Winston, Inc.
Williams, T.R. (2000). Guidelines for designing and evaluating the display of information on the web. Technical Communication, 7(3), 383-396.