A Checklist for the Evaluation of Web Accessibility and Usability for Brazilian Older Adults

Sandra Souza Rodrigues* and Renata Pontin de Mattos Fortes

ICMC, University of São Paulo, São Carlos, SP, Brazil

E-mail: ssrodrigues@usp.br; renata@icmc.usp.br

* Corresponding Author

Received 14 August 2019; Accepted 30 March 2020; Publication 19 May 2020

Abstract

The constant evolution of the Web has covered various segments of society and, as it evolves, its content must be accessible to different users’ profiles. Older adults (60+) are the fastest growing population of users who face difficulties in interacting with websites, due to limitations in their abilities caused by the aging process. Despite the legislation and guidelines established for the development of accessible and usable web contents, several problems of accessibility and usability still must be solved. In particular, most website designers do not take into account the older adult’s profile. This paper introduces Sene-check checklist, a support to web accessibility and usability evaluations for the Brazilian older profile developed according to scientific procedures, including the following three perspectives: (1) literature review, (2) point of view of experts and developers and (3) point of view of Brazilian older adults. The results enabled an investigation into the main problems encountered by older adults regarding web interaction and reinforced the relevance of a support for the evaluation of web accessibility and usability.

Keywords: Web accessibility, usability, older adults, evaluation methods.

1 Introduction

The Web undergoes constant technological innovations, and its adoption in the different segments of society has been irreversible. Although many proposals and standards have been developed towards improving Web browsing by users, a major challenge related to web accessibility and usability still remains [23, 28]. According to Power et al. [41], different users’ profiles should be addressed in the design of web content, so that all information can be widely accessed.

In many countries, the Web is a main vehicle through which governments disseminate information and public services. It is also an important way for the obtaining and supply of updated news and serves as a means for the society to interact. Therefore, it must be usable and accessible for providing equal access and opportunities for individuals with or without disabilities. An accessible Web can potentially aid people with disabilities and older adults to participate more actively in the society [11, 13]. Older adults (deemed here as people aged 60+) are identified as a specific user profile that should be taken into account in accessibility and usability issues. The aging process affects the individuals’ sensory, physical, and cognitive abilities and hampers their interaction with computer systems [19, 26, 42, 47].

In the particular case of Brazil, the country has experienced unprecedented growth in the number of older adults. The definition of older adults in this work is following the definition of Brazilian Law 10,741 sanctioned on October 1st, 2003, that is also known as the Elderly Statute.1 The older adults are those above 60 years of age.

Population aging is a global phenomenon that occurs rapidly due to factors, such as increased life expectancy, lower birth rates and improvements in the quality of life of older people. According to the United Nations (UN) [46] and the World Health Organization (WHO) [50], the worldwide number of older people is expected to double by 2050 and reach nearly 2.1 billion. In Brazil, according to population projections based on the National Census 2010, conducted by the Brazilian Institute of Geography and Statistics (IBGE) [10], the number of individuals aged 65 or older will be four times greater by 2060. Such a change in the Brazilian demographic profile is due to the lower fertility rates associated with a decrease in mortality rates in recent decades [12].

The increasing use of the Web is parallel to population aging. Although new online web services have been offered in many areas in a sophisticated way, most websites have not been designed for older users [25]. Despite accessibility and usability guidelines, such as Web Content Accessibility Guidelines 2.0 (WCAG 2.0) [52], Making Your Web Site Senior Friendly [32] and the Brazilian Electronic Government Accessibility Model (e-MAG) [9], further studies are necessary for a deeper understanding of the difficulties faced by such users when interacting with the Web.

Studies have reported a lack of significant evolution of accessibility evaluation methods. While some practices, as use of more than one evaluation method, seem to provide reliable results, they are not practical in the long term due to the substantial improvements in web technologies. Further efforts are required towards enhancing evaluation techniques and facilitating the accessibility of websites [6].

This article addresses the development of a support that aids developers and experts to evaluate web accessibility and usability for older adults. The support, designed as a checklist (called Sene-check), was based on scientific procedures that involved three different perspectives: (1) literature review, (2) point of view of experts and developers, and (3) point of view of the target audience, the Brazilian older adults. The contributions of this study are an investigation into the main barriers and difficulties faced by Brazilian older adults with web interaction and development of a support for the evaluation of web content for Brazilian older adults. This paper is an extension of previous work (removed for blind review) which presented a literature review regarding the difficulties reported by older adults when interacting with websites and proposed a first version of the checklist. Therefore, we proceeded the research to improve our first checklist, from the following procedures: interviews conducted to investigate the most frequently websites accessed by the elderly and to obtain a sample for evaluations; conformance review, automated tests and user testing to verify the accessibility and usability problems in a sample; and an improved version of the checklist that integrates the main observed issues from the evaluation results.

The remainder of the paper is organized as follows: Section 2 discusses the main related work; Section 3 presents a review of older adults and web interaction; Section 4 describes the method used in this research and the evaluations performed; Section 5 addresses the development of Sene-check checklist and its validation; finally, Section 6 is devoted to discussions and conclusions.

2 Related Work

Many studies have investigated the main barriers and difficulties faced by older adults when interacting with the Web, and proposed different guidelines, methods and techniques that support evaluations of web accessibility and usability, and more senior friendly websites. The Web Accessibility.

Initiative (WAI/W3C) is one of the four domains of the World Wide Web Consortium (W3C), which has developed an international set of accessibility guidelines, called WCAG. The second edition, WCAG 2.0 [52], comprises 12 guidelines organized under 4 principles, namely perceivable, operable, understandable, and robust. WCAG 2.0 also includes guidelines towards making websites more accessible for older users based on WAI-AGE project [48]. Making Your Website Senior Friendly recommendations, version 2016, developed by the National Institute on Aging (NIA) [32], is another important set of guidelines for best practices and assist developers in the creation of websites that meet the demands of older users. They are available online and provide an example of a website accessible to older adults.

In 2003, a checklist composed of 38 questions was developed by Sales and Cybis [16] for the evaluation of web accessibility for older people. The process involved activities of observation in senior centers, a literature review and a recommendations review followed by stages of elaboration, review and validation until the final version. Its creation was based on recommendations, as WCAG 1.0 [51], the Portuguese Group for Initiatives in Accessibility (GUIA) [40], NIA recommendations, version 2002 [31], studies of Czaja [14] and others. It was a relevant effort towards helping developers to better understand the needs of older adults and provide a rapid diagnosis for site analyses regarding usability and accessibility issues for elders. However, this support was created more than 10 years ago and new demands must be addressed.

Lara et al. [1, 2] investigated problems of accessibility and usability of older people and identified resources and mechanisms that might help interaction with the Web. They also suggested a set of success criteria for older adults to be inserted into the WCAG 2.0 guidelines and highlighted the need for other studies with older users, since other difficulties might be identified. Lynch et al. [29] developed a weighted heuristic measure for evaluating the websites usability for older adults and created a heuristics based on the NIA recommendations, version 2002 [31]. This evaluation technique is composed of a list of 32 characteristics that represent the most important senior-friendly recommendations, organized into four categories, namely Readability, Navigation, Content/Organization, and Accessibility. Each heuristic evaluated has a weight and a presence score, which composes a formula to provide a grade of complying accessibility and usability requests; the final score produces a percentage mensuration, called “Usability Index”, that represents a quantitative measurement for the usability of a website.

According to Nielsen [35], an evaluation method must help developers in identifying, count, and classify usability problems found during the inspection and selecting problems that must be corrected. After identifying the problems, the project team should redesign the interface to correct the most significant number of problems. To correct the selected problems, the designers and evaluators ought to prioritize them according to their severity and the cost associated with correcting. Rosa and Veras [43] performed a Nielsen’s heuristic evaluation in two important newspapers’ sites “Estado de São Paulo”2 and “Folha de São Paulo”3 (Brazilian). The evaluation’s result shows a concentration of usability violations in heuristics related to the visibility of system status, consistency and standards, and error prevention. However, the evaluation has not detected accessibility problems like changing the font size of texts or providing navigability with keyboard, for example, problems that are detected just by a human. Santos et al. [18] investigated the effect of eight different types of menus when used by people aged over 40 that had some experience in using the Internet. The experiment revealed that in menus commonly found in Web applications, the task completion time and the number of errors were lower. However, features were lacking as a distinct presentation of menu options already visited, and speed reduced to show menus enabled to users who have little coordination.

The work performed by Freire et al. [21] presented a metric based approach for evaluating municipalities Web pages using automatic accessibility evaluation tools. The results show that the use of automatically generated accessibility metrics is a powerful tool for helping to measure and monitoring the accessibility of e-gov Web sites. However, a usability evaluation requires more information, and evidence of that is that not all accessibility checkpoints could be automatically verified. There are many ways to evaluate usability and accessibility characteristics separately. However, since the professionals are unaware of how usability and accessibility guidelines could join together, the evaluators face difficulty in evaluating a web system. In general, web developers are not users of the web applications they develop, so they have to prioritize the productivity in terms of correctitude and how much time it takes. Therefore, instruments for assessing individual items related to accessibility and usability issues are necessary. Dias et al. [17] proposed the Heuristic Evaluation with Usability and Accessibility Requirements to Assess Web Systems (HEUA) to help to close this gap.

During the development of the HEUA, Dias et al. [17] took into account the primary references of accessibility and usability. HEUA is a questionnaire to assess the usability and accessibility of existing Web systems. More specifically, it is a direct instrument to be applied, since it contributes to the improvements made in the versions of Web systems, indicating a diagnosis in terms of accessibility and usability issues. As systems Web have a development process of perpetual beta, HEUA can be seen as an effortless record to evolve with the quickly built versions of the sites. The 23 patterns of [30] were used to measure usability, and all WCAG 2.0 success criteria [52] were considered for accessibility, as well as the success criteria for older adults developed by [2] and [1]. The patterns and success criteria were named as requirements in HEUA; the requirements were organized into the ten heuristics of [33]. Thus, HEUA presents 93 requirements distributed in ten questions (Table 1).

Each requirement has an example, motivation and sources. These information help to clarify any doubts that the requirement could present when a heuristics specialist apply HEUA. To use HEUA, the specialist needs to answer all the questions, deciding if the system satisfies, does not satisfy or the question is not applicable in each of the requirements classified in the 10 questions. After applying the HEUA, we can calculate a set of measures to obtain a quantitative result that indicates how much of accessibility and usability requirements are fulfilled by a Web system. In this sense, HEUA was proposed as a questionnaire to assess the usability and accessibility of existing Web systems, allowing a means to compare their usability and accessibility levels. By this point, applying HEUA serves as a first view but lacks information to developers and evaluators.

The related works addressed in this section have shown a growing interest in the development of aid supports for evaluations of web pages for older people. Moreover, new approaches that take into account difficulties faced by older users must be updated due to the rapid evolution of web technologies, and usability and accessibility issues must be considered.

Table 1 The number of requirements in each HEUA usability question [17]

Questions of the Heuristic Evaluation No. of Requer.
Q1: Does the web app always keep the user informed about what is happening through feedback in real time? 10
Q2: Does the web app use the language and the mental model of the user with interface features that are similar to the real world? 8
Q3: Does the web app provide the user control and freedom to get out of undesirable states? 13
Q4: Is the web app consistent and follows the same pattern in all the interface to facilitate the recognition of the user? 11
Q5: Does the web app present have a preventive and careful design that may be able to avoid any problems in the interaction? 19
Q6: Does the web app avoid overloading the user’s memory by providing contextual information for each action? 6
Q7: Does the web app provide users with flexibility and efficiency, streamlining the use for experienced users and maintaining the facility for beginners? 14
Q8: Does the web app provide aesthetic and design minimalist, keeping only useful information, direct and clear? 5
Q9: Does the system help the user recognize, diagnose and correct errors? 3
Q10: Does the web app provide a help document that can be easily found in case of necessity? 4
Total of requirements 93

3 Older Adults and Web Interaction

Changes in abilities caused by the aging process must be understood towards the design of inclusive interfaces, systems and services easy to be learned and used by older adults [47], identified as having specific characteristics, such as concerns about health and natural decline of their abilities, and experiencing social isolation and loss of independence. The decline of sensory, physical and cognitive capacities hampers their interaction with technologies [26, 42, 47].

Researchers have investigated the usability and accessibility of websites for older adults. Sayago and Blat [44] studied for three years the daily interactions of 388 older adults with the Web through in-situ observations and conversations. The results showed the main difficulties were remembering the steps of a task, understanding the languages and technical expressions, and using the mouse. Other less impacting barriers, as difficulties in perceiving visual information, understanding icons and using the keyboard were also observed.

Different ways of presenting information and interacting with elements in Web applications are often updated by new technological resources. However, the diversity of developers and designers’ technical knowledge results in inaccessible web pages. Moreover, Web developers generally do not consider older users in their projects. Arfaa and Wang [5] conducted a study with 22 older adults (aged 65+) using social networking sites, and the result showed those sites were difficult for older users to interact with, due to their computer illiteracy, lack of knowledge of Web 2.0 concepts, navigation and other accessibility and usability issues. Similarly, Finn and Johnson [19] developed a usability study in travel sites with nine older users (aged 55–80) and identified some of the most common problems, as confusing terminology, text with a small font size, hard-to-operate menus, information overload, among others. Web applications generally take into account young users, who are more familiar and experienced with the Web, in contrast to older adults [19]. Developers have still misunderstood the actual needs of older adults, and little attention has been devoted to web accessibility for their digital inclusion [15]. In fact, the concepts of web accessibility extend to the inclusion of older users. Petrie et al. [37] proposed the following unified definition of web accessibility: “all people, particularly disabled and older people, can use websites in a range of contexts of use, including mainstream and assistive technologies; to achieve this, websites need to be designed and developed to support usability across these contexts”. Older people have been encouraged to be more online. They use the Internet to send and receive e-mail, communicate with family and friends, search for information, play games, and for other activities, and also want to contribute through blogs and social networks. The number of users who are at least 65 years old that use the Web on computers, tablets, and smartphones continues to grow, therefore, developers must create more accessible sites for them [4, 32].

Hanson [24] highlighted the web applications accessibility may be related to the current generation of the elderly, which has provoked an important reflection on whether technological difficulties are inevitable to aging, or whether some factors will help future generations of older skillful adults eliminate or reduce such difficulties. The next generation of older adults probably will not find significant difficulties in terms of technological change, since they will have already developed a set of technology abilities than those who grew up without a computer [14, 15]. However, computers can also considerably evolve, so that the current web experience does not meet older users’ demands. The next generation of the elderly may encounter barriers with new technologies [24], therefore, accessibility issues should be continuously taken into account and investigated [14, 15].

4 Method

This section describes the methodological procedures used in our research, detailing the data collection, evaluations and results.

4.1 Design

Scientific procedures conducted included the following three perspectives for investigations on barriers and difficulties encountered by Brazilian older adults when interacting with the Web, and accessibility and usability issues involved: (1) literature review, (2) point of view of experts and developers, and (3) point of view of the target audience, i.e., older adults, as shown in Figure 1.

We performed a literature review and studied the related works, summarized in Sections 2 and 3. Interviews were conducted with Brazilian older adults on the websites they most frequently accessed for the obtaining of an actual sample for evaluation; four of such websites were selected. Accessibility and usability problems were then identified and automated tests with tools, a conformance review by experts and user testing on the sample were performed. Based on the results, we have developed a checklist, called Sene-check, which integrates the main issues observed as a support for evaluations of web accessibility and usability related to older adults. The checklist was validated by usability experts for the production of Sene-check 1.1.

images

Figure 1 Stages of the research methodology.

All the data were collected with the consent of the participants, whose anonymity was assured. The study protocol (875.356/2014) and informed consent form were approved by the Federal University of São Carlos Ethical Committee on Human Experimentation. Participation was voluntary and all participants were asked to read and sign the informed consent form before starting the tests. The participants were at least 60 years old, since Article 1 of the Brazilian Statute of the Elderly (Law 10.741), states Brazilian older adults are those aged 60+ [38].

4.2 Interviews

Our interviews inquired about websites mostly accessed by Brazilian older adults. Such data were relevant, since they composed a representative sample of websites actually used by Brazilian older people. The following subsections summarize the data collected.

4.2.1 Participants

Older adults that attended the educational program of the University of the Third Age in the São Carlos Educational Foundation (UATI/FESC), in São Carlos city, state of São Paulo, Brazil, were recruited. Thirty-eight participants aged 60 or older out of 87 invited from UATI/FESC participated in the study. They were 2 men and 36 women, whose ages ranged from 60 to 80 years, with a mean age of 67 years.

4.2.2 Materials

The interview script was conducted in two steps. First, a preliminary version was created and two participants were invited to take part of a pilot study. An improved version was developed according to the main issues pointed out by the first participants. The interview script was developed in partnership with the research group of the Gerontology Department of the Federal University of São Carlos (DGero/UFSCar). The interview was then fully structured and composed of six open-ended questions for the obtaining of the main websites accessed in the following categories: search engines, social networks, video sites, email services, and news sites. The participants were asked about other websites that were not in such categories, such as shopping sites, wikipedia, health sites, food sites, online banking and others.

4.2.3 Procedure

Interviews were first conducted face-to-face and individually at UATI/FESC. Prior to each interview session, we briefly explained the purpose of the research and asked the participant to read and sign the consent form. Next, phone interviews were made with the collaboration of a DGero/UFSCar researcher and at the laboratory of UFSCar and laboratory of the Institute of Mathematics and Computer Sciences at the University of São Paulo (ICMC/USP).

The interview period comprehended approximately four weeks (November to December 2014), which totaled 20.5 hours. The average time spent in each telephone interview was around 9.7 minutes. Some interviews lasted approximately 20 minutes, since the older adults interacted, talked, reported their experiences, difficulties, complaints, suggestions and aspirations regarding the use of the Internet and technologies. All the interviews we registered in annotations.

4.2.4 Results

By the end of the interview period, answers had been collected from 38 individuals and were considered valid. The most accessed website per category and its URL are shown in Table 2.

The participants were asked to comment on their use of the Internet. Older adults were highly interested in interacting with the Web, however, they faced problems and barriers that needed to be solved. Below are some of their remarks:

“Web developers forget about us when they develop a website as a product. They only think of young users. Products usually have a large amount of information in small font, and at our age, it is very difficult to read and interact with the webpages.”

“The Internet is a facilitator because it helps continuous education, since many Brazilian older adults usually have a low scholar level. But the fonts are still very small and there are many foreign terms that hamper our access.”

Table 2 Most accessed websites per category

Categories Website & (URL) Percentage of Access
1. Social networks Facebook (www.facebook.com) 87%
2. Search engines Google (www.google.com) 88%
3. Email services Hotmail (mail.live.com) 32%
4. Video sites Youtube (www.youtube.com) 42%
5. News sites São Carlos Agora (www.saocarlosagora.com.br) 31%
6. Shopping sites Magazine Luiza (www.magazineluiza.com.br) 11%
7. Banking sites Banco do Brasil (www.bb.com.br) 24%
8. Government sites Previdência Social (www.previdencia.gov.br) 15%
9. Health sites Dieta e Saúde (www.dietaesaude.com.br) 29%
10. Food sites Bem Estar (TV Globo) (globoplay.globo.com/bem-estar/) 8%
11. Others/Educational FESC (fesc.com.br) 15%

“Initially, I was afraid to use the Internet. But I cannot exclude myself. So, I try to adapt, and today I think the Internet is a very important tool. So I think sites should make it easier for us to access it, because we have difficulties in reading and understanding contents in small font.”

The set of most frequently accessed websites by Brazilian older adults of São Carlos/SP city was relevant for the next stages of the research methodology. Four of the websites composed the sample used in the evaluations, and are described in the next sections.

4.3 Evaluated Websites

We selected four websites to compose the sample for the evaluations, due to the difficulty in counting on experts and users and the time required for such evaluations. Based on the data provided by the interviews, we applied two selection criteria to identify the websites to be tested: (a) most frequently accessed category and governmental sites, according to the Brazilian legislation on the obligation to comply with accessibility on public administration portals and sites [39]; and (b) sites from regional and national area for composing a representative sample to be tested. Table 3 shows the sample of websites: São Carlos Agora is an online newspaper that provides local news from São Carlos; UOL (Universe Online) is an online Brazilian newspaper that provides world news; FESC is a local site of municipal information related to Education in São Carlos, and Previdência Social is an online national and governmental service with information on security and legislation for Brazilian citizens (Social Security services for Brazilian people). Although such sites exhibit a poor design, they are the most frequently accessed ones by the Brazilian older adults.

Table 3 Websites selected to compose the sample

Categories of Websites Area Websites URLs
News Regional [S] São Carlos Agora www.saocarlosagora.com.br
National [U] UOL www.uol.com.br
Government Regional [F] FESC www.fesc.com.br
National [P] Previdência Social www.previdencia.gov.br

After the selection of the sample, the initial pages of the websites were retrieved by HTTrack version 3.48-21 [53], a copier tool, to guarantee integrity and consistency among evaluations. HTTrack is a free and open source web crawler with an offline browser that enables the downloading of sites from the Internet to a local computer and organizes the original website into a relative link-structure [53].

We made a few modifications in the source code of the selected sites to make them similar to the original ones, since the copy did not guarantee their complete identical appearance. Subsequently, pages were stored on a web server located at ICMC/USP, and were downloaded on the following dates:

Next, we performed automated tests, conformance review and user testing, described in the following sections. Such three evaluation methods were used, since each one complements the other [8]. The automated tests are limited and alone cannot verify all the details about the accessibility of a web page. They require manual inspection for the identification of a more real set of problems. Tests must be conducted with users for the obtaining of the actual problems faced by the target audience when interacting with the web [20, 36].

4.4 Automated Tests

Automated tests aimed at obtaining a technical report evaluating the sample of websites to be used in the development of Sene-check Checklist (regarding the recommendation of the WCAG).

4.4.1 Procedure

We used TAW tool [22] to evaluate the sample of websites. It is a free and online service that produces web reports and tests web pages against WCAG 2.0.

The automated tests were applied in the homepages of the sample websites downloaded on a web server on Mar 11th, 2016 to guarantee integrity and consistency. The tests ran on TAW tool and were setup with level AAA in WCAG 2.0, on Firefox 45.0 and notebook with Microsoft Windows 10 Operating System. Only homepages were evaluated, since they are the entry points and the first impression for users. If homepages are inaccessible, users generally assume they will face the same difficulties on the following pages [3, 6].

4.4.2 Results and discussion

We used the detailed report provided by TAW, which shows the violated success criteria and the source code with the problems, warnings and non-verified points. The WCAG 2.0 success criteria were classified into:

All websites evaluated showed the violated success criteria (see Table 4). We observed all (100% of the pages) of them had WCAG 2.0 violations at level A, and none of the analyzed pages were in conformance with WCAG 2.0, as shown in Table 5. Although TAW did not indicate WCAG 2.0 violations at level AA, it classified some success criteria into such a conformance level, as warnings and non-reviewed (Figure 2). The success criteria require a human review, so that the evaluations can be complemented.

The results were also analyzed for the identification of the success criteria violated in the websites. Such criteria and related problems were mapped. The mapping comprised the evaluation of the technical report from the automated tests conducted in the sample of websites (Table 6).

Table 4 WCAG 2.0 success criteria violated in each homepage of the 4 websites, according to automated tests

WCAG 2.0 Success Criteria Violated FESC Previdência Social São Carlos Agora UOL
1.1.1 X X X
1.3.1 X X X X
2.1.3 X
2.2.1 X
2.2.2 X
2.2.4 X
2.4.4 X X X
2.4.9 X X X X
2.4.10 X X X X
3.1.1 X X X
3.2.2 X
3.2.5 X
3.3.2 X X
4.1.1 X X X X
4.1.2 X X X X
Total 10 6 8 13

Table 5 Results of the automatic tests for the homepages of the 4 websites

Number of Instances of Violations of WCAG 2.0 Success Criteria Number of WCAG 2.0 Success Criteria Violated by Conformance Level
Homepages A AA AAA Total A AA AAA Total
FESC 72 0 89 161 7 0 3 10
Previdência Social 50 0 111 161 4 0 2 6
São Carlos Agora 25 0 28 53 6 0 2 8
UOL 217 0 62 279 9 0 4 13

4.5 Conformance Review

A conformance review conducted obtained a more actual set of accessibility and usability problems in the homepage of the sample of websites and complemented the automated tests. The conformance review method detects problems in websites that cannot be verified automatically [20].

images

Figure 2 Number of violated success criteria in the analyses of Previdência Social website.

Table 6 List of WCAG 2.0 success criteria violated in the sample of websites

Success Criteria Violated
Guidelines Violate A AAA Problems in the Homepages
1.1. Text alternatives 1.1.1 Images without alt attribute, forms without label
1.3. Adaptable 1.3.1 Headers on the same level with no content between them
2.1. Accessible keyboard 2.1.3 Some features not operable by the keyboard
2.2. Enough time 2.2.1; 2.2.2 2.2.4 Automatic page refresh
2.4. Navigable 2.4.4 2.4.9; 2.4.10 Links with same text and different destinations
3.1. Readable 3.1.1 Undeclared page language
3.2. Predictable 3.2.2 Forms with unpredictable behaviors
3.3. Input assistance 3.3.2 3.2.5 No explanations or labels on the data format; existence of an auto-refresh
4.1. Compatible 4.1.1; 4.1.2 Incorrect language syntax

4.5.1 Participants

We invited experts based on their experience and knowledge of web accessibility and usability. First, a list of possible candidates was created and a selection followed two criteria, namely academic education and experience with web accessibility and usability. Twelve experts were invited to participate, however only 11 accepted. Such a number was enough for the evaluation of the websites.

Three master’s students and five Ph.D students from the ICMC/USP, three lecturers of Human-Computer Interaction (HCI) and specialists in web accessibility and usability with more than 5-year-experience in the area participated in the study.

4.5.2 Materials

Participants were asked to apply the HEUA. We chose this method due to its development based on the main references of accessibility and usability, such as patterns of [30], WCAG 2.0 success criteria [52], and success criteria for older adults proposed by [2] and [1]. Moreover, HEUA was developed by our research group [17]. Two examples of the HEUA requirements format are presented as showed in Table 7.

4.5.3 Procedure

We created a preliminary version of the evaluating protocol and invited two experts to participate in a pilot study. The answers obtained from the study were not considered in the final analysis. However, important suggestions were pointed out towards improving the protocol, and grammar errors were corrected at HEUA. An improved version of the evaluation protocol was then developed.

After the pilot study, we sent an email to the 11 experts to invite them to perform the conformance review on a website defined in the email using HEUA as support. Each expert evaluated the homepage of one website, since conformance review demands time for a complete evaluation. We also tried to guarantee consistency with the automated tests. We asked the experts to use Firefox browser, and the evaluation period comprehended 11 days in April 2016.

Table 7 Examples of HEUA requirements

(Q5) Does the web application have a preventive and careful design that may be able to avoid any problems in the interaction?
Requirement 5.13 – Does the Web application present differentiation for visited and unvisited links?
For example: change the color of the link that has already been visited.
Motivation: some users tend to forget the links they have already visited.
References: Lara 2.4.15 [1, 2]
Answer: ( ) It complies ( ) It partially complies ( ) It does not comply ( ) Not applicable
Observations:
(Q6) Does the web application avoid overloading the user’s memory by providing contextual information for each action?
Requirement 6.5 – Does the web application provide contextual help to the user?
For example: presence of contextualized help
Motivation: this functionality can help users operate without losing what they are doing, avoiding possible errors
References: WCAG 3.3.5 [52]
Answer: ( ) It complies ( ) It partially complies ( ) It does not comply ( ) Not applicable
Observations:

Table 8 Average time spent (hours) on the evaluation of each website

Websites Average Time Spent on the Evaluation
FESC 01:02:20
São Carlos Agora 01:33:00
Previdência Social 01:15:30
UOL 01:39:30
Total average time spent 01:21:36

4.5.4 Results and discussion

The evaluations conducted by the experts took, on average, 1 hour and 21 minutes. Most time was spent on the evaluation of São Carlos Agora and UOL websites, as shown in Table 8. The time was supposed to be longer because the two news sites had information overload, links and advertisements. FESC, Previdência Social and São Carlos Agora sites were evaluated three times and UOL was evaluated twice.

According to the results, 55 (65.5%) requirements (out of 84 of HEUA) were violated. Only 29 (34.5%) were not, as shown Table 9. The violated requirements showed “No” and “Partially” answers, whereas “Yes” and “Does not apply” were given to non-violated requirements. Just for exemplify, below, there are transcriptions of a few comments from experts during evaluations.

Table 9 Summary of the conformance review conducted by experts

Issues of HEUA Violated Requirements Non-violated Requirements
Q1 1.1; 1.3; 1.5; 1.6; 1.7; 1.8 1,2; 1.4
Q2 2.1; 2.2; 2.3; 2.4; 2.5; 2.6; 2.7
Q3 3.1; 3.2; 3.9; 3.11; 3.13
Q4 4.2; 4.4; 4.7; 4.8; 4.9; 4.10; 4.11 4.1; 4.3; 4.5; 4.6
Q5 5.1; 5.3; 5.4; 5.5; 5.6; 5.8; 5.10; 5.11; 5.12; 5.13; 5.15 5.2; 5.7; 5.9; 5.14; 5.16
Q6 6.4; 6.5 6.1; 6.2; 6.3
Q7 7.3; 7.4; 7.6; 7.7; 7.8; 7.11; 7.13; 7.14 7.1; 7.2; 7.5; 7.9; 7.10; 7.12
Q8 8.2; 8.1; 8.3; 8.4
Q9 9.1; 9.2; 9.3
Q10 10.3 10.1; 10.2
Total 55 (65.5%) 29 (34.5%)

“The visited options are not highlighted.”

“It does not offer help.”

“The web application shows the breadcrumb from the shortest route to the current page, which is not necessarily the path taken by the user.”

Another analysis of the results identified violated requirements of HEUA and WCAG 2.0 success criteria, which were then mapped in such criteria in each website evaluated. The mapping, shown in Tables 16 and ?? (Appendix A), composes the evaluation technical report from the conformance review, conducted by experts in the sample of websites.

4.6 User Testing

User testing identified the main barriers and actual problems faced by older adults when interacting with the sites and followed the stages recommended by [7, 27], namely: preparation and planning, tests and data collection, analysis, consolidation and reporting of results.

4.6.1 Preparation and planning

This stage involved the definition of the tasks to be performed, definition and recruitment of users, preparation of the material (questionnaires), equipment for recording and applying the tests, and execution of the pilot study, as described below (next Subsection). Prior to the user testing with the older adults, the defined tasks were tested with ten young undergraduates, for verifying if they would report problems to perform them. They argued although the tasks were simple, they did not access sites were not often, since the information was not familiar or even of their interest.

4.6.1.1 Evaluated criteria

A set of criteria was elaborated with two other researchers – one from HCI group and a gerontologist from DGero/UFSCar – for investigation, observation and reporting of the problems faced by older adults during the tests.

  1. Difficulties in locating the requested information;
  2. Difficulties in finding and using the site map and returning to the homepage;
  3. Forgetfulness and inattention;
  4. Presentation of links very near each other and links with activation problems;
  5. Use of scrollbars;
  6. Information overload, many links and advertisements;
  7. Problems with language, abbreviations and confusing terminology;
  8. Difficulties in finding menu options and links already visited;
  9. Perception of feedbacks;
  10. Difficulties in finding a specific place on the screen or places that users search less frequently;
  11. Lack of help resources;
  12. Presentation of subtitles in videos;
  13. Other problems verified through users’ comments during the tests.
4.6.1.2 Tasks undertaken

The task list was defined with the help of two researchers – one from the HCI area and a gerontologist from the DGero/UFSCar group. The tasks for each website were planned to represent typical activities that older adults would do when visiting websites on their own. They were organized in a scenario that involved searching specific news and finding important information for older adults on government sites. Attention was devoted to the writing of the tasks, so that they would not have specific terms, or require much time for their accomplishment. Below are the 4 sites selected with their tasks for user testing. The participants were instructed to not use any external search engines.

4.6.1.3 Materials

Two questionnaires with the closed and open-ended questions were applied to the users for verifying the usability of the evaluated websites. A pretest questionnaire obtained demographic and Internet-experience information and a post-test questionnaire detected difficulties in the users’ performing tasks, suggestions and comments on the sites. A paper version was created, and questions were written in a readable text size with 13-point Arial font. Each paper questionnaire consisted of two pages and additional two pages of informed consent form. Each page was single-sided printed.

4.6.1.4 Location equipment and software

The user testing sessions were held at the laboratories of UATI/FESC and UATI/USP senior activity centers, where the older adults attend Internet courses.

All tests were conducted on an HP Pavilion laptop with Microsoft Windows 7 Operating System and an AMD Dual-Core 2.30 GHz processor, 4 GB RAM, equipped with speakers, keyboard, a 14” LCD screen, a webcam and a 2-button mouse. The participants accessed the sites using Firefox 40.0.3 web browser. The computer also ran Morae version 3.2.1 [45], a screen capture program, that recorded the participants and researchers screens and voices. Morae was also set to record keystrokes and mouse events.

4.6.1.5 Participants

The users were older adults (people aged 60+) with some experience with the Internet and use of sites. They were recruited at UATI/FESC and UATI at the University of São Paulo (USP), in São Carlos city, São Paulo state, Brazil.

Fifty-three older adults were invited to participate, however, due to difficulties, such as time and rejection by some of them, only 20 accepted the invitation. They were 5 men and 15 women, whose ages ranged from 61 to 84 years, with a mean age of 67.5 years. Most participants were women, of whom 60% had graduated. Over half of the participants (55%) reported they had less than 10 years’ experience with the Internet and 70% indicated their frequency of use varied from once a day to three times a day. We observed most of them used the Internet every day and did not have significant experience, since they had been using it for less than 10 years, as shown in Table 10.

4.6.2 Procedure

The tests were conducted from May to June 2015 and the study sessions were held at the laboratories of UATI/FESC and UATI/USP senior activity centers. The participants were briefed on the purpose of the study and evaluation protocol. They were asked to read and sign the consent form and complete the pre-test questionnaire about their gender, age, educational competencies, profession, and time of Internet use, and frequency of visits to websites.

Table 10 Number of participants according to their characteristics

Characteristics Total
Gender Female
Male		 15
5
Age 60-69
70–79
80–89		 16
2
2
Education Level Elementary school
High School
University		 2
4
12
Internet experience 10− years
10+ years		 11
9
Frequency of Internet use More than 3 times a day
At least once a day, but not always more than 3 times a day (every day)
More than twice a week, but not every day
At least once a month, but not every week		 7
7
5
1
Total of participants 20

After all the settings on the computer had been made, the researcher opened the browser with the link of the downloaded pages stored on a web server located at ICMC/USP and began the recording session with Morae. For each website, the task was described to the participant and the use of the think-aloud protocol was explained. The researcher observed and registered all the comments and questions of the participants during the sessions.

After the evaluations of all websites, the participants answered the posttest questionnaire, reported the tasks of higher difficulties and suggestions for improvements, and commented on important aspects of the websites evaluated. The four sites were evaluated by 17 participants, and three participants evaluated two sites each. Due to equipment failure, data of two participants were lost and a participant gave up the test. The sites were evaluated in different cycles for each participant, and the order was reshuffled at each cycle for avoiding fatigue effects, as shown in Table 11. After the 8 cycles had been attributed to the first 8 participants, another round of 8 cycles was started.

Table 11 Cycles of evaluation of websites followed by users for avoid order effect. The sites are labeled by their first letters: [F] FESC, [P] Previdência Social, [S] São Carlos Agora, and [U] UOL.

Cycle First website Second website Third website Fourth website
1 F P S U
2 S F P U
3 F S P U
4 U P S F
5 P U F S
6 S F U P
7 P S U F
8 U P F S

4.6.3 Results and discussion

The total recorded video footage of the evaluation sessions lasted more than 15 hours. Each test session lasted 45 min on average, depending on the number of sites each participant chose to evaluate. The standard deviation was 15 minutes (SD = 15), since it took some participants approximately 1 hour and 19 minutes to complete the test, while the others spent 28 minutes. The total time spent refers to only the time of execution of tasks.

The results showed all participants faced difficulties in interacting with the sites. They were not satisfied with the evaluated websites and failed to complete the tasks. The problems and difficulties detected were then discussed. Some of the difficulties identified, as forgetfulness and inattention, could not be classified as usability problem or accessibility barriers, since they are subjective and cannot be controllable. Such characteristics are very common to older adults, since people’s aging affects their cognitive abilities. Two participants did not show any type of forgetfulness or inattention. The highest incidence of forgetfulness was related to the activity to be performed in the task. Participants lost attention browsing the website and asked the researcher to remind them of what should be done. Another occurrence refers to the participants’ not remembering they had already completed a task and beginning to do it again.

The following other 13 usability problems, difficulties and accessibility barriers were detected:

Such 13 main problems composed the report to be taken into account in the development of the checklist proposed.

5 Development of Sene-check Checklist

Sene-check checklist was based on the analyses of the results of the three evaluations and literature review (Figure 1), described in the next subsections.

5.1 First Version of Sene-check

Sene-check was elaborated to support developers and experts in their evaluations of web accessibility and usability for older users. It is an instrument to be applied objectively by experts, since it enables a more accurate and even-handed diagnosis of likely problems on websites, contains a minimum set of directly applicable recommendations, and supports a quick and inexpensive evaluation [49]. It was called Sene-check, since “sene” refers to the root of the Latin word senescentia, which means aging, and “check” refers to the well-known English term “inspection”.

The first version of Sene-check was composed of a set of questions defined from the following results of our evaluations:

At first, it was developed from the users’ testing results, followed by the automated tests results and conformance review, and according to the main accessibility and usability references. Ten usability heuristics proposed by [34] supported the usability evaluation. WCAG 2.0 success criteria [52], the success criteria for older adults proposed by [1, 2], and the NIA recommendations [32] supported the accessibility evaluation.

Sene-check questions were called checkpoints and organized into five of the seven difficulties identified by [1, 2]. Such difficulties were characterized as the main problems reported by Brazilian older adults when interacting with the Web. We disregarded two difficulties specifically related to shopping sites [1, 2] that did not apply to the general perspective of the requirements aimed at. Therefore, the checkpoints were organized according to the following five difficulties [1, 2]:

  1. Difficulty in reading and text comprehension (“L”): readability is still a problem faced by older adults in most web applications regardless of their visual quality, and can occur with available texts, system messages and menu options. Users prefer to read the content without much effort. Texts in small or blurred fonts are a recurring problem;
  2. Difficulty in recognizing and accessing links (“RL”): users face difficulties in distinguishing a common text from a link. Such a problem is more specific to novice users, who cannot notice the difference in the format of the mouse cursor when scanning the link area. In other words, the mouse cursor in the shape of a little hand is an insignificant symbol. Another important feature is the forgetfulness or distraction of the users about links they have already visited;
  3. Difficulty in navigating (“N”): the most efficient websites meet users’ expectations and take them to the right places. Once users cannot achieve their goals using navigation elements from the website, they usually believe the information is not there and change to another website. The creation of a solid and proper navigational structure provides higher reliability for users to navigate on the website, since they realize the ease of returning to the previously visited page, without barriers that hamper navigation;
  4. Difficulty in performing specific tasks (“RT”): such difficulties are faced when users are unable to perform tasks that would make their daily lives easier in web interaction;
  5. Difficulty in searching for and locating information (“B”): the display of a search engine on a website facilities access to the content and helps users know what they really want. One of the main problems reported by older adult is the location of the desired information among excessive information and links, which are usually presented as search results in a list or menu format.

Each Sene-check checkpoint was classified as one of the five difficulties, and is structured as follows (Table 12):

Table 12 Example of a structure of a checkpoint in checklist

Difficulty and checkpoint
Difficulty in reading and understanding texts (“L”)
L2 – Is the font size used in the Web application suitable for reading and understanding the textual content?
Level AA
Example: The font size of the textual content should have at least 16 pixels.
Motivation: To assist older adults of low vision and those who have lost some of the vision capabilities due to the aging process.
References:
(a) Success criteria 1.4.4, WCAG 2.0. World Wide Web Consortium, 2008.
Available at: www.w3.org/TR/WCAG20
(b) Making Your Website Senior Friendly. National Institute on Aging, 2016.
Available at: https://www.nia.nih.gov
Answer: ( ) Yes ( ) Partially ( ) No ( ) Not applicable
Observations:

Sene-check version 1.0 is composed of 51 checkpoints (a summary of their numbers is shown in Table 13). Based on WCAG 2.0 [52], the Senecheck 1.0 checkpoints were organized into three conformance levels, namely A (lowest), AA, and AAA (highest). The adequacy to the checkpoints will promote better accessibility and usability of the web content for the older profile. Their verification and validation procedures are described in the following section.

5.2 Validation of the First Version

Sene-check 1.0 was validated by experts in web accessibility and usability who investigated the comprehension, understanding and relevance of each checkpoint.

Table 13 Summary of Sene-check version 1.0

Difficulties of Sene-check Number of Checkpoints
Difficulty in reading and understanding texts (“L”) 15
Difficulty in recognizing and accessing links (“RL”) 2
Difficulty in navigating (“N”) 17
Difficulty in performing specific tasks (“RT”) 11
Difficulty in searching for and locating information (“B”) 6
Total of checkpoints 51

5.2.1 Materials

The validation was based on an online questionnaire that included one question for each checkpoint of each of the 5 difficulties (R, RL, N, RT and B). The questionnaire was comprised of 51 questions that asked the experts to rate the checkpoint regarding easy understanding, supply of clear examples and relevance. The options were provided on a Likert scale from 1 (I totally disagree) to 5 (I totally agree). An additional 1 open-ended question at the end of each difficulty asked the participant to comment on the checkpoint and another open-ended question at the end of the questionnaire asked the participant to submit more general comments, suggestions and criticisms.

5.2.2 Participants

Eight experts (3 women and 5 men) participated in this study. The selection took into account their academic formation, web accessibility and usability experience, and web development experience. Three master’s students and four Ph.D students from ICMC/USP and one professor, all specialists in web accessibility and usability over 5-years experience in the area were selected.

5.2.3 Procedure

After the publication of our questionnaire on the Web, an invitation email with a link to Sene-check (and attached file) was sent to the participants. Each participant validated it responding to the questionnaire and providing comments about it. The questionnaire was available to be filled in from August 28 to September 1, 2016.

5.2.4 Results

Six experts answered the whole questionnaire and two others reviewed each checkpoint. They provided comments and suggestions for reviews. All answers and comments were analyzed through measurements of the mean tendency. The average response value was selected as the result for the corresponding checkpoint. The comprehension, understanding and relevance of each checkpoint were checked.

The participants’ qualitative responses were also examined, and a set of modifications was suggested by the experts, who identified comprehension problems and ambiguities, and recommended joining similar checkpoints and adding new ones. The participants provided relevant comments, which were analyzed towards improving Sene-check. Below are some interesting comments made by the experts during the validation:

“The example in RT6 could be better described for avoiding confusion with RT8.”

“Checkpoint RL1 does not provide examples of errors to be mentioned in the description of the question. How confusing to users could this be?”

5.3 Sene-check Version 1.1

Each checkpoint of Sene-check 1.0 was reviewed from the validation data and the understanding and writing problems were tackled. Checkpoints of similar content were grouped. Those to be allocated in another difficulty level, as suggested by the participants, were rearranged and the ones reported as irrelevant were excluded. An expert suggested the inclusion of checkpoint R14, which was appropriately addressed according to the structure previously defined (illustrated in Table 12).

All WCAG 2.0 success criteria of conformance level A were covered in Sene-check to address the minimum level of compliance. Therefore, a web application accomplishes the minimum level of conformance with WCAG 2.0 if all Sene-check checkpoints are complied. In other words, it should be the minimum necessary for a user to be able to interact and reach his/her objectives during the interaction. In contrast to Sene-check 1.0, and since the new version includes the more basic criteria of WCAG 2.0, the checkpoints are not it does not have the organized in those three levels of compliance.

The modifications in Sene-check 1.0 are shown in Table 14: number of rewritten, excluded and added checkpoints in the new version. A new version, i.e. Sene-check 1.1 containing 52 checkpoints distributed in the 5 difficulties (Table 15) was developed and included another checkpoint, although some checkpoints were rearranged among difficulties, others were joined for avoiding ambiguities, and one was added according to an expert’s suggestion. Checkpoints were also included towards satisfying all level A WCAG 2.0 success criteria.

Table 14 Number of Sene-check 1.0 changes

Number of Checkpoints Modifications
13 Ls were rewritten
2 RLs
15 Ns
8 RTs
3 Bs
2 Ls were excluded
2 Ns
2 Bs
1 L were added
1 RL
1 Ns
3 RTs

Table 15 Summary of Sene-check version 1.1

Difficulties of Sene-check Number of Checkpoints
Difficulty in reading and understanding texts (“L”) 14
Difficulty in recognizing and access links (“RL”) 3
Difficulty in navigation (“N”) 17
Difficulty in performing specific tasks (“RT”) 14
Difficulty in searching for and locating information (“B”) 4
Total of checkpoints 52

Finally, the experts identified a significant number of problems in Sene-check 1.0 and evidenced the need for regular improvements and adaptations. The changes resulted in Sene-check 1.1 to be subjected to a new evaluation cycle for guaranteeing improvements according to technological advances and society demands. The new version (1.1) is available for researchers and interested people, and addresses the expectation to serve as a fair and objective instrument for websites evaluation, regarding Brazilian older users.

6 Discussion and Conclusions

A number of studies has investigated the main barriers and difficulties faced by older adults when interacting with the Web, however, much more work must be done due to the rapid evolution of web technologies. Many approaches (methods, recommendations, guidelines, etc.) have been developed for evaluations of web content for older adults [1, 2, 16, 29, 32, 48]. However, there is a lack of significant evolution in accessibility evaluation methods [6] and a growing need for more studies on issues of accessibility for older users [14, 15]. Older adults are a heterogeneous group of users who generally do not have a specific limitation, but a set of limitations caused by the aging process. Failures in complying with the web accessibility guidelines and lack of knowledge by developers on the difficulties encountered by older people when interacting with the web are a bottleneck for the development of web applications more accessible to the elderly.

We have followed a set of scientific procedures to find out the main perspectives a checklist support should address in a broad view of the works and approaches designed from the evolution of technological and human behaviors. We argue each procedure should be reviewed towards attending the technological advances and keeping our checklist version evolving.

This research is a starting point for the design of checklists for a specific user profile, and the following activities have been developed:

The results from the first activity (a) indicated a high preference for social media sites by the elderly in the city of São Carlos, as verified in the interviews performed. Such virtual environments promote more interaction and communication among people, since the elderly usually live the experience of social isolation.

The first results from activity (b) are limited, since they did not identify all accessibility problems of the sites. For instance, in the automated tests, FESC website violated 10 success criteria, where as in the conformance review, it violated 37 WCAG 2.0 success criteria, which indicates automated tests always require complementary evaluations. The conformance review complemented the automatic tests, however, it requires more time and shows mainly technical problems. Finally, user testing complemented the two other methods and detected the actual problems faced by users when interacting with the Web.

The results of the three evaluations showed the sites were not accessible and did not take into account the difficulties of older adults. Even sites such as FESC and Previdência Social, whose target audience is those users, have many barriers that hamper the access by the Brazilian elder users. The users’ demographics size reaches huge proportions, which requires an adaptation of the web content, so that this profile can be taken into account. Therefore, a more appropriate use must be provided for the older adults for avoiding their frustrations.

During activity (c), we have created an aid in the form of a checklist (Sene-check) for an objective evaluation of web accessibility and usability for Brazilian older adults. We have reviewed each checkpoint of Sene-check to attend the experts’ advice and improvements. The changes on Sene-check 1.0 resulted in a new version, Sene-check 1.1, that must undergo new cycles of evaluation and studies for guaranteeing improvements according to continuous technological advances and society demands.

Even with this validation that culminated in the new version of Sene-check, we recognize that it has been a limited validation, and it is still necessary to perform a more controlled and complete validation to the checklist. To conduct this evaluation, we intend to make the checklist available and disseminate it to developers for collect feedback and to know the acceptance of such aid. We pursuit to obtain the impression from the developers about how much the Sene-check could speed up the comprehension of the accessibility and usability problems faced by the older adults, as well as how the convincement of the web accessibility techniques is under evaluators view. In the next future, we intend to overcome the initial limits of our validation. We plan to conduct experiments, with different experts and developers, having various levels of knowledge about accessibility barriers for older adult interactions. The experimental scenario aims to observe the efforts that Sene-check requires from evaluators, as well as the weaknesses Sene-check 1.1 presents, in comparison with other user’s tests.

Studies on the interaction by older users must be conducted; it is a relevant topic in the area of HCI, which aims to contribute with research and develop a diversity of approaches and technologies that minimize the characteristic problems and losses caused by aging. As increasing use of the Web has accompanied the population aging, therefore, developers must use approaches that help them create more accessible sites to older users. This study aims at contributing to future research projects in this trend, e.g., enhancement of our validity procedures to be adapted to younger users or even children.

Sene-check was conceived to help a rapid and objective diagnosis of accessibility and usability problems of a site as a whole. It can be easily adapted to check only specific elements of an interface, as videos of a web page. Similar research can also be developed in other contexts, as different platforms from the Web for older adults. The use of mobile devices, as smartphones and tablets, has increased and become more popular and accessible to the general population. Therefore, future investigations should apply similar stages of our research, regarding evaluation of websites, but verifying if they can help the evaluation of products in other platforms towards a better diagnosis of their accessibility and usability and an objective dissemination of its knowledge.

Appendix

We described here the results of conformance review by experts, using HEUA and the sample of the websites. Table 16 shows the relation between violated requirements on each site and success criteria to which they report.

Table 16 List of requirements violated in HEUA

Requirements Violated in HEUA FESC Previdência Social São Carlos Agora UOL Correspondenting WCAG 2.0 Success Criteria
1.1 x x x x 2.4.8
1.3 x x 3.1.1
1.5 x x x x
1.6 x x x x 1.2.3; 1.2.5; 1.2.7
1.7 x x x x 1.2.8
1.8 x x x 1.2.9
2.1 x x 3.1.3
2.2 x x x 2.4.4; 2.4.9
2.3 x 3.1.4
2.4 x x x 2.4.6
2.5 x x
2.6 x x x x 3.1.5
2.7 x x 1.2.6
3.1 x x x
3.3 x x 2.1.2
3.4 x x 2.1.3
3.5 x x x x 2.4.1
3.6 x 1.3.2
3.7 x x 2.4.3
3.8 x x x x 3.1.2
3.10 x 1.4.2
3.12 x x 2.2.2
4.2 x x
4.4 x x 2.4.7
4.7 x x x x 3.1.6
4.8 x 3.2.5
4.9 x 4.1.1
4.10 x x x 1.2.2
4.11 x x 1.2.4
5.1 x x x 3.3.2
5.3 x 3.2.2
5.4 x 3.3.4; 3.3.6
5.5 x x x x
5.6 x x x 1.4.5; 1.4.9
5.8 x x x
5.10 x x
5.11 x
5.12 x
5.13 x x x x
5.15 x x x
6.4 x x x
6.5 x x x 3.3.5
7.3 x x x 1.1.1
7.4 x x 1.4.3
7.6 x x 1.4.4
7.7 x x 4.1.2
7.8 x x x 1.3.1
7.11 x x x
7.13 x x x x 2.2.1
7.14 x x x 1.2.1
8.2 x x
9.1 x x
9.2 x x 3.3.1
9.3 x x 3.3.3
10.3 x x

Acknowledgments

This study was supported by the grant #2015/24525-0, São Paulo Research Foundation (FAPESP), and was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior – Brasil (CAPES) – Finance Code 001.

1http://www.planalto.gov.br/ccivil_03/leis/2003/l10.741.htm

2www.estadao.com.br

3www.folha.uol.com.br

References

[1] Silvana Maria Affonso de Lara. Support mechanisms for usability and accessibility in the interaction of older adults on the web. Ph. d. thesis in computer science, Institute of Mathematical and Computer Sciences in São Carlos (ICMC/USP), University of Sao Paulo, São Carlos – SP, Brazil, 2013.

[2] Silvana Maria Affonso de Lara, Willian Massami Watanabe, Eduardo Pezutti Beletato dos Santos, and Renata P. M. Fortes. Improving WCAG for Elderly Web Accessibility. In Proceedings of the 28th ACM International Conference on Design of Communication, SIGDOC ’10, pages 175–182, New York, NY, USA, 2010. ACM.

[3] Basel Al Mourad and Faouzi Kamoun. Accessibility evaluation of dubai e-government websites: Findings and implications. Journal of E-Government Studies and Best Practices, 2013:1–15, 2013.

[4] Andrew Arch. Web accessibility for older users: Successes and opportunities (keynote). In Proceedings of the 2009 International Cross-Disciplinary Conference on Web Accessibililty (W4A), pages 1–6, New York, NY, USA, 2009. ACM.

[5] Jessica Arfaa and Yuanqiong (Kathy) Wang. A usability study on elder adults utilizing social networking sites. In Aaron Marcus, editor, Design, User Experience, and Usability. User Experience Design for Diverse Interaction Platforms and Environments: Third International Conference, DUXU 2014, Held as Part of HCI International 2014, Heraklion, Crete, Greece, June 22–27, 2014, Proceedings, Part II, pages 50–61. Springer International Publishing, Cham, 2014.

[6] Ibtehal S. Baazeem and Hend S. Al-Khalifa. Advancements in web accessibility evaluation methods: How far are we? In Proceedings of the 17th International Conference on Information Integration and Web-based Applications & Services, iiWAS ’15, pages 90:1–90:5, New York, NY, USA, 2015. ACM.

[7] Simone Diniz Junqueira Barbosa and Bruno Santana da Silva. Human-Computer Interaction (in portuguese). Elsevier (Campus), Rio de Janeiro – RJ, 2010.

[8] Giorgio Brajnik. Beyond conformance: The role of accessibility evaluation methods. In Sven Hartmann, Xiaofang Zhou, and Markus Kirchberg, editors, Web Information Systems Engineering – WISE 2008 Workshops: Auckland, New Zealand, September 1–4, 2008. Proceedings, pages 63–80. Springer Berlin Heidelberg, Berlin, Heidelberg, 2008.

[9] Brazilian Government. Accessibility model for e-government (e-mag). Technical report, Brazilian Government., Brasília, DF, 2014.

[10] Brazilian Institute of Geography and Statistics (IBGE). Censo Demografico 2010. http://censo2010.ibge.gov.br/resultados, June 2015. Accessed: 06-13-2017.

[11] Marina Buzzi, Barbara Leporini, and Clara Meattini. Design guidelines for web interfaces of home automation systems accessible via screen reader. Journal of Web Engineering, 18(8):477–512, 2019.

[12] Ana Amélia Camarano and Solange Kanso. Institutions for the elderly in brazil (in portuguese). Revista Brasileira de Estudos de População, 27(1):232–235, 2010.

[13] Daniel Costa, Nádia Fernandes, Sofia Neves, Carlos Duarte, Raquel Hijón-Neira, and Luís Carriço. Web Accessibility in Africa: A Study of Three African Domains. In Paula Kotzé, Gary Marsden, Gitte Lindgaard, Janet Wesson, and Marco Winckler, editors, Human-Computer Interaction – INTERACT 2013, volume 8117 of Lecture Notes in Computer Science, pages 331–338. Springer Berlin Heidelberg, USA, 2013.

[14] Sara J. Czaja. Computer technology and the older adult. Handbook of Human-computer Interaction, pages 791–824, 1997.

[15] Silvana Maria Affonso de Lara, Renata Pontin de Mattos Fortes, Cibele Maria Russo, and Andre Pimenta Freire. A study on the acceptance of website interaction aids by older adults. Universal Access in the Information Society, 15(3):445–460, 2016.

[16] Márcia Barros de Sales and Walter de Abreu Cybis. Development of a checklist for the evaluation of the web accessibility for the aged users. In Proceedings of the Latin American Conference on Human-computer Interaction, CLIHC ’03, pages 125–133, New York, NY, USA, 2003. ACM.

[17] Ana Luiza Dias, Renata Pontin de Mattos Fortes, and Paulo Cesar Masiero. Heua: A heuristic evaluation with usability and accessibility requirements to assess web systems. In Proceedings of the 11th Web for All Conference, pages 18:1–18:4, New York, NY, USA, 2014. ACM.

[18] Eduardo P.B. dos Santos, Silvana M.A. de Lara, Willian M. Watanabe, Mario C.A. Filho, and Renata P.M. Fortes. Usability Evaluation of Horizontal Navigation Bar with Drop-down Menus by Middle Aged Adults. In Proceedings of the 29th ACM International Conference on Design of Communication, SIGDOC 11, page 145150, New York, NY, USA, 2011. Association for Computing Machinery.

[19] Kate Finn and Jeff Johnson. A usability study of websites for older travelers. In Proceedings of the 7th International Conference on Universal Access in Human-Computer Interaction: User and Context Diversity – Volume 2, UAHCI’13, pages 59–67, Berlin, Heidelberg, 2013. Springer-Verlag.

[20] André Pimenta Freire, Silvana Maria Affonso de Lara, and Renata Pontin de Mattos Fortes. Accessibility evaluation of websites by disabled users. In Proceedings of the 12th Brazilian Symposium on Human Factors in Computing Systems, IHC ’13, pages 348–351, Porto Alegre, Brazil, Brazil, 2013. Brazilian Computer Society.

[21] Andre Pimenta Freire, Rudinei Goularte, and Renata Pontin de Mattos Fortes. Techniques for developing more accessible web applications: A survey towards a process classification. In Proceedings of the 25th Annual ACM International Conference on Design of Communication, SIGDOC ’07, pages 162–169, New York, NY, USA, 2007. ACM.

[22] Fundación CTIC. TAW tool. http://www.tawdis.net, June 2017. Accessed: 06-13-2017.

[23] Teresa D. Gilbertson and Colin H. C. Machin. Guidelines, icons and marketable skills: An accessibility evaluation of 100 web development company homepages. In Proceedings of the International Cross-Disciplinary Conference on Web Accessibility, W4A ’12, pages 17:1–17:4, New York, NY, USA, 2012. ACM.

[24] Vicki L. Hanson. Age and web access: The next generation. In Proceedings of the 2009 International Cross-Disciplinary Conference on Web Accessibililty (W4A), W4A ’09, pages 7–15, New York, NY, USA, 2009. ACM.

[25] Vicki L. Hanson. Influencing technology adoption by older adults. Interacting with Computers, 22(6):502–509, 2010. Special Issue on Inclusion and Interaction: Designing Interaction for Inclusive Populations.

[26] Leonard Hayflick. How and why we become older (in Portuguese). Campus, Rio de Janeiro RJ, 1996.

[27] Jonathan Lazar, Jinjuan Heidi Feng, and Harry Hochheiser. Research methods in human-computer interaction. John Wiley & Sons, United Kingdom, 1 edition, 2010.

[28] S. Luján-Mora, R. Navarrete, and M. Penñfiel. Egovernment and web accessibility in South America. In eDemocracy eGovernment (ICEDEG), 2014 First International Conference on, pages 77–82, Region 09 – Latin America, April 2014. IEEE Computer Society.

[29] Kyle R. Lynch, Diana J. Schwerha, and George A. Johanson. Development of a weighted heuristic for website evaluation for older adults. International Journal of Human Computer Interaction, 29(6):404–418, 2013.

[30] F. Montero, M. Lozano, P. González, and I. Ramos. A first approach to design web sites by using patterns. In Proceedings of the First Nordic Conference On Pattern Languages of Programs. VikingPLoP. Hojstrupgard, pages 137–158, Denmark, 2002. Published by Microsoft Business Solutions, ApS.

[31] National Institute on Aging. Making Your Website Senior Friendly. https://www.nlm.nih.gov/pubs/checklist.pdf, Sep. 2002. Accessed: 07-20-2017.

[32] National Institute on Aging. Making Your Website Senior Friendly. https://www.nia.nih.gov, June 2016. Accessed: 06-13-2017.

[33] Jakob Nielsen. Enhancing the explanatory power of usability heuristics. In Proc. of the SIGCHI Conf. on Human Factors in Comput. Syst., CHI ’94, pages 152–158, New York, NY, USA, 1994. ACM.

[34] Jakob Nielsen and Rolf Molich. Heuristic evaluation of user interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’90, pages 249–256, New York, NY, USA, 1990. ACM.

[35] Nielsen, J. Heuristic evaluation. John Wiley & Sons, New York, NY, USA, 1994.

[36] Helen Petrie and Nigel Bevan. The evaluation of accessibility, usability and user experience. In Constantine Stephanidis, editor, The Universal Access Handbook, volume 1, Chapter 20, pages 10–16. CRC Press, United States of America, 2009.

[37] Helen Petrie, Andreas Savva, and Christopher Power. Towards a unified definition of web accessibility. In Proceedings of the 12th Web for All Conference, W4A ’15, pages 35:1–35:13, New York, NY, USA, 2015. ACM.

[38] Planalto. Lei n. 10.741 de 1 outubro 2003. http://www.planalto.gov.br/ccivil03/leis/2003/L10.741.htm, Oct. 2003. Accessed: 06-13-2017.

[39] Planalto. Decreto n. 5.296 de 2 de dezembro de 2004. http://www.planalto.gov.br/ccivil03/ato2004-2006/2004/decreto/d5296.htm, Dec. 2004. Accessed: 06-13-2017.

[40] Portuguese Group for Initiatives in Accessibility. GUIA. http://www.acessibilidade.net/historia, Oct. 1999. Accessed: 06-13-2017.

[41] Christopher Power, André Pimenta Freire, and Helen Petrie. Integrating Accessibility Evaluation into Web Engineering Processes. International Journal of Information Technology and Web Engineering, 4(4):54–77, January 2010.

[42] Yoel Raban and Malcolm Brynin. Older people and new technologies. In Sara Kiesler Robert Kraut, Malcolm Brynin, editor, Computers, Phones, and the Internet: Domesticating Information Technology, pages 43–50. Oxford University Press New York, New York, 2006.

[43] Juan Miguel Rosa and Manoel Veras. Avaliação heurística de usabilidade em jornais online: estudo de caso em dois sites. Perspectivas em Ciência da Informação, 18:138–157, 03 2013.

[44] Sergio Sayago and Josep Blat. About the relevance of accessibility barriers in the everyday interactions of older people with the web. In Proceedings of the 2009 International Cross-Disciplinary Conference on Web Accessibililty (W4A) – W4A ’09, pages 104–113, New York, New York, USA, 2009. ACM Press.

[45] TechSmith Corporation. Morae. https://www.techsmith.com/morae.html, May 2017. Accessed: 05-30-2017.

[46] United Nations (UN). World Population Prospects: the 2017 Revision. https://esa.un.org/unpd/wpp/Publications/Files/WPP2017KeyFindings.pdf, June 2017. Accessed: 06-11-2019.

[47] John Vines, Gary Pritchard, Peter Wright, Patrick Olivier, and Katie Brittain. An age-old problem: Examining the discourses of ageing in hci and strategies for future research. ACM Trans. Comput.-Hum. Interact., 22(1):2:1–2:27, February 2015.

[48] Web Accessibility Initiative. Web Accessibility Initiative: Ageing Education and Harmonisation. https://www.w3.org/WAI/WAI-AGE/, Oct. 2010. Accessed: 07-24-2017.

[49] Marco Winckler and Marcelo Soares Pimenta. Avaliação de usabilidade de sites web. Escola de Informática da SBC SUL (ERI 2002) ed. Porto Alegre: Sociedade Brasileira de Computação (SBC), 1:85–137, 2002.

[50] World Health Organization (WHO). Ageing and health, Feb. 2018. Accessed: 06-11-2019.

[51] World Wide Web Consortium. Web Content Accessibility Guidelines (WCAG) 1.0. https://www.w3.org/TR/WCAG10/, May 1999. Accessed: 06-13-2017.

[52] World Wide Web Consortium. Web Content Accessibility Guidelines (WCAG) 2.0. https://www.w3.org/TR/WCAG20, June 2008. Accessed: 06-13-2017.

[53] Xavier Roche. HTTrack Website Copier. https://www.httrack.com/page/1/en/index.html, June 2017. Accessed: 06-13-2017.

Biographies

images

Sandra Souza Rodrigues is a Ph.D. candidate at the University of São Paulo at São Carlos, Brazil, since 2017. She attended the Federal University of Lavras, Minas Gerais, Brazil, where she received her B.Sc. in Information Systems in 2013; and received an M.Sc. degree in Computer Science from the University of São Paulo, Brazil, in 2016. Sandra is currently completing a doctorate in Computer Science at the University of São Paulo. Her Ph.D. work centers on analyze the forms of interaction in IoT systems and provide accessibility mechanisms that encourage the use of these technologies by older adults in their daily lives.

images

Renata Pontin de Mattos Fortes is currently an Associate Professor in the Computer Science Department of the Institute of Mathematics and Computer Science, at the University of São Paulo, Brazil. She holds a Ph.D. in Computational Physics from USP in 1996. Her research interests lie in Software Process and Human-Computer Interaction applied to the web systems and multi-devices domain. She is part of the Intermedia Laboratory, where she researches about accessibility, methods, and models applied to the web software processes domain. She has been a PC and senior PC member for a number of events and a reviewer of some prestigious journals. She is a member of the ACM and of the Brazilian Computer Society.

Journal of Web Engineering, Vol. 19_1, 63–108.
doi: 10.13052/jwe1540-9589.1913
© 2020 River Publishers

Abstract

1 Introduction

2 Related Work

3 Older Adults and Web Interaction

4 Method

4.1 Design

images

4.2 Interviews

4.2.1 Participants

4.2.2 Materials

4.2.3 Procedure

4.2.4 Results

4.3 Evaluated Websites

4.4 Automated Tests

4.4.1 Procedure

4.4.2 Results and discussion

4.5 Conformance Review

images

4.5.1 Participants

4.5.2 Materials

4.5.3 Procedure

4.5.4 Results and discussion

4.6 User Testing

4.6.1 Preparation and planning

4.6.1.1 Evaluated criteria

4.6.1.2 Tasks undertaken

4.6.1.3 Materials

4.6.1.4 Location equipment and software

4.6.1.5 Participants

4.6.2 Procedure

4.6.3 Results and discussion

images

images

images

images

5 Development of Sene-check Checklist

5.1 First Version of Sene-check

5.2 Validation of the First Version

5.2.1 Materials

5.2.2 Participants

5.2.3 Procedure

5.2.4 Results

5.3 Sene-check Version 1.1

6 Discussion and Conclusions

Appendix

Acknowledgments

References

Biographies