This page presents instructions for a joint study between RIAM (Reciprocal Interoperability between the Accessible and Mobile Webs) and SAMBA (A semi-automatic method for measuring barriers of accessibility) projects.

In this study, you will be asked to evaluate a Web page for both its accessibility for disabled people and mobile-friendliness. In summary, you will be given a list of barriers and a Web page to investigate and you are asked to see if those barriers exist on the given page.

Instructions

  1. Read the information sheet about this experiment.
  2. Fill in a screening questionnaire.
  3. When you were asked to participant in this study, you were given a judge number. Please download one of the following barriers sheet by using that given judge number (for the sake of counterbalancing several effects, each judge is asked to evaluate a given page against given barriers):
    Judge number Barrier sheet Web page Note
    J1 Barrier sheet-1 The Godfather at IMDB
    J2 Barrier sheet-2 "I love god father movies" Facebook group please make sure that you are NOT signed in to a facebook account
    J3 Barrier sheet-3 Sam's Chop House Manchester
    J4 Barrier sheet-4 Hall's Harbour Quilts, Halifax
    J5 Barrier sheet-5 The Godfather at IMDB
    J6 Barrier sheet-6 "I love god father movies" Facebook group please make sure that you are NOT signed in to a facebook account
    J7 Barrier sheet-7 Sam's Chop House Manchester
    J8 Barrier sheet-8 Hall's Harbour Quilts, Halifax
    J9 Barrier sheet-9 The Godfather at IMDB
    J10 Barrier sheet-10 "I love god father movies" Facebook group please make sure that you are NOT signed in to a facebook account
    J11 Barrier sheet-11 Sam's Chop House Manchester
    J12 Barrier sheet-12 Hall's Harbour Quilts, Halifax
    J13 Barrier sheet-13 The Godfather at IMDB
    J14 Barrier sheet-14 "I love god father movies" Facebook group please make sure that you are NOT signed in to a facebook account
    J15 Barrier sheet-15 Sam's Chop House Manchester
    J16 Barrier sheet-16 Hall's Harbour Quilts, Halifax
    J17 Barrier sheet-17 The Godfather at IMDB
    J18 Barrier sheet-18 "I love god father movies" Facebook group please make sure that you are NOT signed in to a facebook account
    J19 Barrier sheet-19 Sam's Chop House Manchester
    J20 Barrier sheet-20 Hall's Harbour Quilts, Halifax
    J21 Barrier sheet-21 The Godfather at IMDB
    J22 Barrier sheet-22 "I love god father movies" Facebook group please make sure that you are NOT signed in to a facebook account
    J23 Barrier sheet-23 Sam's Chop House Manchester
    J24 Barrier sheet-24 Hall's Harbour Quilts, Halifax
    J25 Barrier sheet-25 The Godfather at IMDB
    J26 Barrier sheet-26 "I love god father movies" Facebook group please make sure that you are NOT signed in to a facebook account
    J27 Barrier sheet-27 Sam's Chop House Manchester
    J28 Barrier sheet-28 Hall's Harbour Quilts, Halifax
    J29 Barrier sheet-29 The Godfather at IMDB
    J30 Barrier sheet-30 "I love god father movies" Facebook group please make sure that you are NOT signed in to a facebook account
    J31 Barrier sheet-31 The Godfather at IMDB
    J32 Barrier sheet-32 "I love god father movies" Facebook group please make sure that you are NOT signed in to a facebook account
    J33 Barrier sheet-33 Sam's Chop House Manchester
    J34 Barrier sheet-34 Hall's Harbour Quilts, Halifax
    J35 Barrier sheet-35 The Godfather at IMDB
  4. By using your judge number, visit the appropriate web page (URL given in the third column of the table above) and then fill in the barriers sheet. In this evaluation process, you can use any evaluation tool (e.g., a standalone evaluation tool or a browser extension) you like.

    You are asked to evaluate each barrier with respect to users that are blind, that have low vision, that are motor impaired, and those who use a mobile device. For each barrier and user group, please check whether that barrier exists. If it does not exist then give 0 or leave blank, but if it exists please specify the severity based on three point scale (1=minor, 2=significant, 3=critical) and also explain the rationale for your rating. Please avoid just to say yes or no as we are also quite interested in identifying the severity of a barrier. If a page contains several instances of the same barrier type (e.g., several clickable images with no text) and same user group, then enter the severity rating by referring to the worst-case scenario and explicitly indicating which case or specific element you are referring to.

    A barrier is minor if it is detected by the user, but there are simple ways to overcome it or to avoid it; it is easy to remember it, to learn how to avoid or get around it; it is significant when it heavily affects the task execution. To overcome the barrier the user has to back-up, follow a trial-and-error strategy, guess the proper action, repeat an action several times; the user may incur in errors. In many cases it is not possible to avoid the barrier; even if it can be avoided, this requires a substantial knowledge and/or memory (to recall that there is the barrier and on how to avoid it). Finally it is critical when users give up, or they cannot reach their goals. This can happen after users have spent considerable time and effort to try to overcome the barrier, perhaps with many errors. There are no alternative ways (known to the users) that can be followed to achieve the goals.

    In the evaluation process, you can use any evaluation tool or technique you like but please state them in the post-evaluation questionnaire.

  5. Fill in a post evaluation questionnaire.
  6. Once you complete the evaluation, please return us your screening questionnaire, the barriers sheet and the post evaluation questionnaire. We would like the evaluations to occur no later than Friday Nov. 07, and to be sent us soon after.

    If you have filled in electronic versions, please email them to Yeliz Yesilada or Giorgio Brajnik. If you have used hard copies, please either fax or post them to Yeliz Yesilada or Giorgio Brajnik, and email to notify that you posted or faxed them:

    Yeliz Yesilada
    School of Computer Science, Human Centred Web Lab (LF1),
    The University of Manchester, Oxford Road, Manchester, M13 9PL
    Email: yeliz.yesilada@manchester.ac.uk
    Fax: +44 (0) 845 139 5599

    Giorgio Brajnik
    Dipartimento di Matematica e Informatica
    University of Udine, via delle Scienze 206
    33100 Udine, Italy
    Email: giorgio@dimi.uniud.it