Characterizing the Software Acceptance Testing and the Inclusion of People with Disabilities by Means of a Systematic Mapping
Keywords:
Software Testing , Acceptance Testing , Disabled User, Systematic Mapping , Secondary StudyAbstract
Acceptance testing is a test technique where the final user evaluates the actual use of the software. In this test, software must meet the acceptance criteria defined on the requirement engineering phase to be approved. This paper describes the participation of the person with a disability in acceptance testing in terms of testing approaches, tools and user feedback. By performing a systematic mapping, we analyzed 609 articles, of which 58 were considered to answer four research questions. We identified that acceptance tests have no standard and present a dispersed variety of approaches and tools. We also few acceptance tests studies applied to testers with disabilities, although this approach has been rising in recent years. Considering accessibility for all users when including them in acceptance tests can ensure greater reach of users to systems
Downloads
References
P. Bourque and R. E. Fairley, eds., SWEBOK: Guide to the Software
Engineering Body of Knowledge. Los Alamitos, CA: IEEE Computer
Society, version 3.0 ed., 2014.
K. Naik and P. Tripathy, Software Testing and Quality Assurance: Theory
and Practice. Wiley, 2011.
D. Gotterbarn, “How the new software engineering code of ethics affects
you,” IEEE software, vol. 16, no. 6, pp. 58–64, 1999.
Brasil, “Lei nº 13.146, de 6 de julho de 2015. institui a lei brasileira
de inclusão da pessoa com deficiência (estatuto da pessoa com
deficiência).,” Diário Oficial da República Federativa do Brasil, 2015.
K. Silveira and R. Prikladnicki, “A systematic mapping study of diversity
in software engineering: A perspective from the agile methodologies,”
pp. 7–10, 05 2019.
G. Rodriguez, R. Nadri, and M. Nagappan, “Perceived diversity in
software engineering: a systematic literature review,” Empirical Software
Engineering, vol. 26, 09 2021.
P. A. d. M. Silveira-Neto, I. d. Carmo Machado, J. D. McGregor, E. S.
de Almeida, and S. R. de Lemos Meira, “A systematic mapping study of
software product lines testing,” Information and Software Technology,
vol. 53, pp. 407–423, May 2011.
J. Weiss, A. Schill, I. Richter, and P. Mandl, “Literature review of
empirical research studies within the domain of acceptance testing,”
in 2016 42th Euromicro Conference on Software Engineering and
Advanced Applications (SEAA), pp. 181–188, 2016.
E. C. D. Santos, P. Vilain, and D. H. Longo, “A systematic literature
review to support the selection of user acceptance testing techniques,”
in Proceedings of the 40th International Conference on Software
Engineering: Companion Proceeedings, ICSE ’18, (New York, NY,
USA), p. 418–419, Association for Computing Machinery, 2018.
E. C. dos Santos, “Aplicação de técnicas de testes de aceitação
automatizados para especificação de software em editais de licitação,”
Master’s thesis, Universidade Federal de Santa Catarina, Florianópolis,
Santa Catarina, Brasil, 2019.
K. R. F. Scannavino, E. Y. Nakagawa, S. C. P. F. Fabbri, and F. C.
Ferrari, Revisão Sistemática da Literatura em Engenharia de Software:
teoria e prática. Elsevier Brasil, 2017.
E. C. dos Santos and P. Vilain, “Automated acceptance tests as software
requirements: An experiment to compare the applicability of fit tables
and gherkin language,” in Lecture Notes in Business Information
Processing, pp. 104–119, Springer International Publishing, 2018.
E. C. D. Santos, P. Vilain, and D. H. Longo, “A systematic literature
review to support the selection of user acceptance testing techniques,”
in Proceedings of the 40th International Conference on Software
Engineering: Companion Proceeedings, ACM, may 2018.
N. Chaveiro, S. B. R. Duarte, A. R. d. Freitas, M. A. Barbosa, C. C.
Porto, and M. P. d. A. Fleck, “Instrumentos em língua brasileira de
sinais para avaliação da qualidade de vida da população surda,” Revista
de Saúde Pública, vol. 47, pp. 616–623, 2013.
N. Chaveiro, C. L. Rodrigues, S. B. Duarte, R. Rodrigues de
Oliveira Garcia, A. Ribeiro de Freitas, L. Rodrigues de Oliveira,
K. Rocha Gomes da Silva, and C. C. Porto, “A tool to evaluate the
quality of life of deaf people using whoqol instruments,” in 2018 IEEE
International Conference on Systems, Man, and Cybernetics (SMC),
pp. 3871–3876, 2018.
J. Fang, M. P. Fleck, A. Green, K. McVilly, Y. Hao, W. Tan, R. Fu, and
M. Power, “The response scale for the intellectual disability module
of the whoqol: 5-point or 3-point?,” Journal of Intellectual Disability
Research, vol. 55, no. 6, pp. 537–549, 2011.
“Web content accessibility guidelines (wcag) overview.” https://www.
w3.org/WAI/standards-guidelines/wcag/. Accessed: 2021-05-10.
J. R. D. Cruz, R. A. Gazmin, A. E. K. Guillen, and M. J. C.
Samonte, “Talkie: An Assistive Web-based Educational Application
Using Audio Files and Speech Technology for the Visually Impaired,” in
Proceedings of the 2020 The 6th International Conference on Frontiers
of Educational Technologies, (Tokyo Japan), pp. 39–42, ACM, June
N. J. Y. Bergantin, A. L. Buenaventura, K. A. M. Cordobin,
M. J. C. Samonte, A. K. D. Balan, and G. B. Resuello, “eALEAP: Online Reviewer System for Alternative Learning Equivalency
and Accreditation Program,” in Proceedings of the 2020 The 6th
International Conference on Frontiers of Educational Technologies,
(Tokyo Japan), pp. 1–5, ACM, June 2020.
M. J. C. Samonte, A. R. I. Garcia, B. J. D. Valencia, and M. J. S.
Ocampo, “Using Online Handwritten Character Recognition in Assistive
Tool for Students with Hearing and Speech Impairment,” in Proceedings
of the 2020 11th International Conference on E-Education, E-Business,
E-Management, and E-Learning, (Osaka Japan), pp. 189–194, ACM,
Jan. 2020.
M. Co, J. K. Balan, J. D. Dolendo, J. C. De Goma, and M. J. C. Samonte,
“TEXT2FSL: A Filipino Sign Language Phrase Translator Tool for Deaf
and Mute,” in Proceedings of the 2020 The 6th International Conference
on Frontiers of Educational Technologies, (Tokyo Japan), pp. 82–86,
ACM, June 2020.
A. K. Rai and A. K. Malviya, “Testing MapReduce program using
Induction Method,” in 2020 IEEE International Students’ Conference
on Electrical,Electronics and Computer Science (SCEECS), (Bhopal,
India), pp. 1–5, IEEE, Feb. 2020.
T. Yesmin and M. W. Carter, “Evaluation framework for automatic
privacy auditing tools for hospital data breach detections: A case study,”
International Journal of Medical Informatics, vol. 138, p. 104123, June
B. Losada, J.-M. López-Gil, and M. Urretavizcaya, “Improving Agile
Software Development Methods by means of User Objectives: An
End User Guided Acceptance Test-Driven Development Proposal,” in
Proceedings of the XX International Conference on Human Computer
Interaction - Interacción ’19, (Donostia, Gipuzkoa, Spain), pp. 1–4,
ACM Press, 2019.
C. Kuchta, R. D. Budiardja, and V. G. V. Larrea, “Harmony: A Harness
Monitoring System for the Oak Ridge Leadership Computing Facility,”
in Proceedings of the Practice and Experience in Advanced Research
Computing on Rise of the Machines (learning), (Chicago IL USA),
pp. 1–6, ACM, July 2019.
T. N. Thanh, M. Morgan, M. Butler, and K. Marriott, “Perfect Match:
Facilitating Study Partner Matching,” in Proceedings of the 50th ACM
Technical Symposium on Computer Science Education, (Minneapolis
MN USA), pp. 1102–1108, ACM, Feb. 2019.
R. Rwemalika, M. Kintis, M. Papadakis, Y. Le Traon, and P. Lorrach,
“Ukwikora: continuous inspection for keyword-driven testing,” in
Proceedings of the 28th ACM SIGSOFT International Symposium on
Software Testing and Analysis - ISSTA 2019, (Beijing, China), pp. 402–
, ACM Press, 2019.
T. Tuglular and S. Sensulun, “SPL-AT Gherkin: A Gherkin Extension for
Feature Oriented Testing of Software Product Lines,” in 2019 IEEE 43rd
Annual Computer Software and Applications Conference (COMPSAC),
(Milwaukee, WI, USA), pp. 344–349, IEEE, July 2019.
A. N. Rachman, R. N. Shofa, E. N. F. Dewi, and A. Hidayat,
“Application of Collision Detection Algorithm and Scoring Health
Point in Fighting Games with Android-Based Augmented Reality
Technology,” in 2019 2nd International Conference of Computer and
Informatics Engineering (IC2IE), (Banyuwangi, Indonesia), pp. 216–
, IEEE, Sept. 2019.
M. Nass, E. Alegroth, and R. Feldt, “Augmented Testing: Industry
Feedback To Shape a New Testing Technology,” in 2019 IEEE
International Conference on Software Testing, Verification and
Validation Workshops (ICSTW), (Xi’an, China), pp. 176–183, IEEE, Apr.
M. J. C. Samonte, R. A. Gazmin, J. D. S. Soriano, and M. N. O. Valencia,
“BridgeApp: An Assistive Mobile Communication Application for the
Deaf and Mute,” in 2019 International Conference on Information and
Communication Technology Convergence (ICTC), (Jeju Island, Korea
(South)), pp. 1310–1315, IEEE, Oct. 2019.
R. A. Supangan, L. A. S. Acosta, J. L. S. Amarado, E. B. Blancaflor, and
M. J. C. Samonte, “A gamified learning app for children with ADHD,” in
Proceedings of the 2nd International Conference on Image and Graphics
Processing - ICIGP ’19, (Singapore, Singapore), pp. 47–51, ACM Press,
S. Nurjanah, H. B. Santoso, and Z. A. Hasibuan, “The User Acceptance
Test of An "ICT Adoption for Education" Framework,” in Proceedings
of the 10th International Conference on Computer Modeling and
Simulation - ICCMS 2018, (Sydney, Australia), pp. 129–133, ACM
Press, 2018.
Y. Xiao, L. Xiao, and Z. Li, “Framework Research on the
Implementation of Automated Test User Requirements,” in Proceedings
of the 2018 2nd International Conference on Computer Science and
Artificial Intelligence - CSAI ’18, (Shenzhen, China), pp. 190–194, ACM
Press, 2018.
E. K. Gebre-Amanuel, F. G. Taddesse, and A. T. Assalif, “Web based
expert system for diagnosis of cattle disease,” in Proceedings of the
th International Conference on Management of Digital EcoSystems -
MEDES ’18, (Tokyo, Japan), pp. 66–73, ACM Press, 2018.
D. A. Martillano, A. F. D. Chowdhury, J. C. M. Dellosa, A. A.
Murcia, and R. J. P. Mangoma, “PINDOTS: An Assistive Six-dot
Braille Cell Keying Device on Basic Notation Writing for Visually
Impaired Students with IoT Technology,” in Proceedings of the 2018
nd International Conference on Education and E-Learning - ICEEL
, (Bali, Indonesia), pp. 41–47, ACM Press, 2018.
M. J. C. Samonte, R. C. D. Mullen, J. M. P. Bañaga, W. Y. Cortes, and
J. A. V. D. Calzada, “In-Patient Medication Delivery in Mobile App and
Out-Patient Online Lab Results for Hospitals,” in Proceedings of the 2nd
International Conference on E-Society, E-Education and E-Technology
- ICSET 2018, (Taipei, Taiwan), pp. 155–159, ACM Press, 2018.
S. Tachmammedov, Y. K. Hooi, and K. S. Kalid, “Automated Multifactor Analytics for Cheat-Proofing Attendance-taking,” in Proceedings of the 2018 7th International Conference on Software and Computer Applications - ICSCA 2018, (Kuantan, Malaysia), pp. 183–188, ACM
Press, 2018
A. Hidayati and R. Nabila, “E-commerce development using AngularJS
framework and RESTful API,” IOP Conference Series: Materials
Science and Engineering, vol. 403, p. 012063, Oct. 2018.
S. Hotomski and M. Glinz, “GuideGen: a tool for keeping requirements
and acceptance tests aligned,” in Proceedings of the 40th International
Conference on Software Engineering: Companion Proceeedings,
(Gothenburg Sweden), pp. 49–52, ACM, May 2018.
R. C. Morco, F. B. Calanda, J. A. Bonilla, M. J. S. Corpuz, J. E.
Avestro, and J. M. Angeles, “e-RICE: An Expert System using RuleBased Algorithm to Detect, Diagnose, and Prescribe Control Options
for Rice Plant Diseases in the Philippines,” in Proceedings of the
International Conference on Computer Science and Artificial
Intelligence - CSAI 2017, (Jakarta, Indonesia), pp. 49–54, ACM Press,
G. J. P. Rosales and A. C. Lagman, “Development of Industry Academe
Linkage Alumni and Placement Portal,” in Proceedings of the 2017
International Conference on Information Technology - ICIT 2017,
(Singapore, Singapore), pp. 184–189, ACM Press, 2017.
M. R. L. Perez, A. C. Lagman, and R. T. Adao, “Event Management
Solution Using Web Application Platform,” in Proceedings of the
International Conference on Information Technology - ICIT 2017,
(Singapore, Singapore), pp. 206–211, ACM Press, 2017.
S. Hotomski, E. B. Charrada, and M. Glinz, “Aligning Requirements and
Acceptance Tests via Automatically Generated Guidance,” in 2017 IEEE
th International Requirements Engineering Conference Workshops
(REW), (Lisbon, Portugal), pp. 339–342, IEEE, Sept. 2017.
P. Cruz and H. Astudillo, “Deriving high-priority acceptance test cases
using utility trees: A case study,” in 2017 36th International Conference
of the Chilean Computer Science Society (SCCC), (Arica), pp. 1–8,
IEEE, Oct. 2017.
L. Almeida, E. Cirilo, and E. A. Barbosa, “SS-BDD: Automated
Acceptance Testing for Spreadsheets,” in Proceedings of the 1st
Brazilian Symposium on Systematic and Automated Software Testing -
SAST, (Maringa, Parana, Brazil), pp. 1–10, ACM Press, 2016.
A. Koehl and H. Wang, “SERF: optimization of socially sourced
images using psychovisual enhancements,” in Proceedings of the
th International Conference on Multimedia Systems - MMSys ’16,
(Klagenfurt, Austria), pp. 1–12, ACM Press, 2016.
D. H. Longo, P. Vilain, L. P. da Silva, and R. d. S. Mello, “A web
framework for test automation: user scenarios through user interaction
diagrams,” in Proceedings of the 18th International Conference on
Information Integration and Web-based Applications and Services -
iiWAS ’16, (Singapore, Singapore), pp. 458–467, ACM Press, 2016.
D. Clerissi, M. Leotta, G. Reggio, and F. Ricca, “Test Driven
Development of Web Applications: A Lightweight Approach,” in 2016
th International Conference on the Quality of Information and
Communications Technology (QUATIC), (Lisbon, Portugal), pp. 25–34,
IEEE, Sept. 2016.
L. Neubeck, G. Coorey, D. Peiris, J. Mulley, E. Heeley, F. Hersch, and
J. Redfern, “Development of an integrated e-health tool for people with,
or at high risk of, cardiovascular disease: The Consumer Navigation
of Electronic Cardiovascular Tools (CONNECT) web application,”
International Journal of Medical Informatics, vol. 96, pp. 24–37, Dec.
M. El-Attar and H. A. Abdul-Ghani, “Using security robustness
analysis for early-stage validation of functional security requirements,”
Requirements Engineering, vol. 21, pp. 1–27, Mar. 2016.
G. Blichmann, C. Radeck, S. Hahn, and K. Meißner, “Componentbased workspace awareness support for composite web applications,”
in Proceedings of the 17th International Conference on Information
Integration and Web-based Applications &Services - iiWAS ’15,
(Brussels, Belgium), pp. 1–10, ACM Press, 2015.
D. H. Longo and P. Vilain, “Creating user scenarios through user
interaction diagrams by non-technical customers,” in Proceedings of the
th International Conference on Software Engineering and Knowledge
Engineering, KSI Research Inc. and Knowledge Systems Institute
Graduate School, jul 2015.
T. M. King, G. Nunez, D. Santiago, A. Cando, and C. Mack, “Legend:
an agile DSL toolset for web acceptance testing,” in Proceedings of the
International Symposium on Software Testing and Analysis - ISSTA
, (San Jose, CA, USA), pp. 409–412, ACM Press, 2014.
V. Horner and A. Coleman, “Evaluation of a guidelines-based e-health
decision support system for primary health care in South Africa,” in
Proceedings of the Southern African Institute for Computer Scientist and
Information Technologists Annual Conference 2014 on SAICSIT 2014 Empowered by Technology - SAICSIT ’14, (Centurion, South Africa),
pp. 142–147, ACM Press, 2014
T. Straszak and M. Smiałek, “Automating Acceptance Testing with tool
support,” pp. 1569–1574, Sept. 2014.
R. Ahmad, W. A. Chyi, A. Sarlan, and R. Kasbon, “Guiding novice
database developers in database schema creation,” in 2014 IEEE
Conference on e-Learning, e-Management and e-Services (IC3e),
(Hawthorn, Australia), pp. 64–69, IEEE, Dec. 2014.
A. Marques, F. Ramalho, and W. L. Andrade, “Comparing ModelBased Testing with Traditional Testing Strategies: An Empirical Study,”
in 2014 IEEE Seventh International Conference on Software Testing,
Verification and Validation Workshops, (OH, USA), pp. 264–273, IEEE,
Mar. 2014.
S. Y. Bani Hashem, N. A. Mat Zin, N. F. Mohd Yatim, and
N. Mohamed Ibrahim, “Improving Mouse Controlling and Movement for
People with Parkinson’s Disease and Involuntary Tremor Using Adaptive
Path Smoothing Technique via B-Spline,” Assistive Technology, vol. 26,
pp. 96–104, Apr. 2014.
N. Nomura, Y. Kikushima, and M. Aoyama, “Business-Driven
Acceptance Testing Methodology and Its Practice for E-Government
Software Systems,” in 2013 20th Asia-Pacific Software Engineering
Conference (APSEC), (Bangkok), pp. 99–104, IEEE, Dec. 2013.
E. Alegroth, M. Nass, and H. H. Olsson, “JAutomate: A Tool
for System- and Acceptance-test Automation,” in 2013 IEEE Sixth
International Conference on Software Testing, Verification and
Validation, (Luxembourg, Luxembourg), pp. 439–446, IEEE, Mar. 2013.
S. Kamalakar, S. H. Edwards, and T. M. Dao, “Automatically Generating
Tests from Natural Language Descriptions of Software Behavior:,” in
Proceedings of the 8th International Conference on Evaluation of Novel
Approaches to Software Engineering, (Angers, France), pp. 238–245,
SciTePress - Science and and Technology Publications, 2013.
M. Druk and M. Kropp, “ReFit: A Fit test maintenance plug-in for
the Eclipse refactoring plug-in,” in 2013 3rd International Workshop
on Developing Tools as Plug-Ins (TOPI), (San Francisco, CA, USA),
pp. 7–12, IEEE, May 2013.
C.-Y. Hsieh, C.-H. Tsai, and Y. C. Cheng, “Test-Duo: A framework for
generating and executing automated acceptance tests from use cases,” in
8th International Workshop on Automation of Software Test (AST),
(San Francisco, CA, USA), pp. 89–92, IEEE, May 2013.
A. Shafie, W. F. Wan Ahmad, N. Mohd., J. J. Barnachea, M. F. Taha, and
R. L. Yusuff, ““SynMax”: A Mathematics Application Tool for Down
Syndrome Children,” in Advances in Visual Informatics (D. Hutchison,
T. Kanade, J. Kittler, J. M. Kleinberg, F. Mattern, J. C. Mitchell,
M. Naor, O. Nierstrasz, C. Pandu Rangan, B. Steffen, M. Sudan,
D. Terzopoulos, D. Tygar, M. Y. Vardi, G. Weikum, H. B. Zaman,
P. Robinson, P. Olivier, T. K. Shih, and S. Velastin, eds.), vol. 8237,
pp. 615–626, Cham: Springer International Publishing, 2013. Series
Title: Lecture Notes in Computer Science.
W. M. Watanabe, R. P. M. Fortes, and A. L. Dias, “Using acceptance
tests to validate accessibility requirements in RIA,” in Proceedings of
the International Cross-Disciplinary Conference on Web Accessibility -
W4A ’12, (Lyon, France), p. 1, ACM Press, 2012.
Rafidah Mohd Arif and O. O. Khalifa, “Online tutoring system
in college: Case study in private education,” in 2012 International
Conference on Computer and Communication Engineering (ICCCE),
(Kuala Lumpur, Malaysia), pp. 608–611, IEEE, July 2012.
O. Liskin, C. Herrmann, E. Knauss, T. Kurpick, B. Rumpe,
and K. Schneider, “Supporting Acceptance Testing in Distributed
Software Projects with Integrated Feedback Systems: Experiences and
Requirements,” in 2012 IEEE Seventh International Conference on
Global Software Engineering, (Porto Alegre, Rio Grande do Sul, Brazil),
pp. 84–93, IEEE, Aug. 2012.
R. Hirschfeld, M. Perscheid, and M. Haupt, “Explicit use-case
representation in object-oriented programming languages,” p. 10.
R. Borg and M. Kropp, “Automated acceptance test refactoring,” in
Proceeding of the 4th workshop on Refactoring tools - WRT ’11,
(Waikiki, Honolulu, HI, USA), p. 15, ACM Press, 2011.
Z. M. Jiang, A. Avritzer, E. Shihab, A. E. Hassan, and P. Flora, “An
Industrial Case Study on Speeding Up User Acceptance Testing by
Mining Execution Logs,” in 2010 Fourth International Conference on
Secure Software Integration and Reliability Improvement, (Singapore,
Singapore), pp. 131–140, IEEE, 2010.
W. F. W. Ahmad, E. A. P. Akhir, and S. Azmee, “Game-based learning
courseware for children with learning disabilities,” in 2010 International
Symposium on Information Technology, (Kuala Lumpur, Malaysia),
pp. 1–4, IEEE, June 2010.
M. El-Attar and J. Miller, “Developing comprehensive acceptance tests
from use cases and robustness diagrams,” Requirements Engineering,
vol. 15, pp. 285–306, Sept. 2010.
B. Güldali, S. Sauer, P. Winkelhane, H. Funke, and M. Jahnich, “Patternbased generation of test plans for open distributed processing systems,”
in Proceedings of the 5th Workshop on Automation of Software Test -
AST ’10, (Cape Town, South Africa), pp. 119–126, ACM Press, 2010.
C. Wohlin, P. Runeson, M. Höst, M. C. Ohlsson, and B. Regnell,
Experimentation in Software Engineering. Springer, 2012.
C. Wohlin, “Guidelines for snowballing in systematic literature studies
and a replication in software engineering,” in Proceedings of the 18th
International Conference on Evaluation and Assessment in Software
Engineering, EASE ’14, (New York, NY, USA), Association for
Computing Machinery, 2014.
M. Bown and A. Sutton, “Quality control in systematic reviews
and meta-analyses,” European Journal of Vascular and Endovascular
Surgery, vol. 40, no. 5, pp. 669–677, 2010.
A. C. F. Silva, C. L. Rodrigues, N. Chaveiro, R. R. de Oliveira Garcia,
S. B. Duarte, M. A. R. Araújo, V. B. Santos, K. R. G. da Silva,
P. M. S. Rodrigues, L. M. Oliveira, and C. C. S. Sousa, “Lessons
learned about oral-auditory and visual-spatial communication in
requirements engineering with deaf stakeholders,” in SAC ’20: The 35th
ACM/SIGAPP Symposium on Applied Computing, online event, [Brno,
Czech Republic], March 30 - April 3, 2020 (C. Hung, T. Cerný, D. Shin,
and A. Bechini, eds.), pp. 1379–1386, ACM, 2020.
R. W. Miller, C. T. Collins, S. Developer, S. Software, and Developer,
“Acceptance testing,” Tech. Rep. practically, 2001.
