What is the Impact of Bad Layout in the Understandability of Social Goal Models?

Mafalda Santos, Catarina Gralha, Miguel Goulão, João Araújo, Ana Moreira, Joao Cambeiro

Research output: Chapter in Book/Report/Conference proceedingConference contribution

9 Citations (Scopus)

Abstract

The i∗ community has published guidelines, including model layout guidelines, for the construction of models. Our goal is to evaluate the effect of the layout guidelines on the i∗ novice stakeholders' ability to understand and review i∗ models. We performed a quasi-experiment where participants were given two understanding and two reviewing tasks. Both tasks involved a model with a bad layout and another model following the i∗ layout guidelines. We evaluated the impact of layouts by combining the success level in those tasks and the required effort to accomplish them. Effort was assessed using time, perceived complexity (with NASA TLX), and eye-tracking data. Participants were more successful in understanding than in reviewing tasks. However, we found no statistically significant difference in the success, time taken, or perceived complexity, between tasks conducted with models with a bad layout and models with a good layout. Most participants had little to no prior knowledge in i∗, making them more representative of stakeholders with no requirements engineering expertise. They were able to understand the models fairly well after a short tutorial, but struggled when reviewing models. In the end, adherence to the existing i∗ layout guidelines did not significantly impact i∗ model understanding and reviewing performance.
Original languageEnglish
Title of host publication24th IEEE International Requirements Engineering Conference
Place of PublicationBeijing
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages206-215
Number of pages10
ISBN (Electronic)978-150904121-3
DOIs
Publication statusPublished - 2 Dec 2016
Event24th IEEE International Requirements Engineering Conference Workshops, REW 2016 - Beijing, China
Duration: 12 Sep 201616 Sep 2016

Conference

Conference24th IEEE International Requirements Engineering Conference Workshops, REW 2016
CountryChina
CityBeijing
Period12/09/1616/09/16

Fingerprint

Requirements engineering
NASA
Experiments

Keywords

  • NASA
  • Complexity
  • Construction of models
  • Diagram layout
  • Eye-tracking
  • Model understanding
  • Social goals
  • Statistically significant difference
  • Understandability

Cite this

Santos, M., Gralha, C., Goulão, M., Araújo, J., Moreira, A., & Cambeiro, J. (2016). What is the Impact of Bad Layout in the Understandability of Social Goal Models? In 24th IEEE International Requirements Engineering Conference (pp. 206-215). [7765526] Beijing: Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/RE.2016.51
Santos, Mafalda ; Gralha, Catarina ; Goulão, Miguel ; Araújo, João ; Moreira, Ana ; Cambeiro, Joao. / What is the Impact of Bad Layout in the Understandability of Social Goal Models?. 24th IEEE International Requirements Engineering Conference. Beijing : Institute of Electrical and Electronics Engineers Inc., 2016. pp. 206-215
@inproceedings{ef687f4aab434acf9d2ed9f9f58be051,
title = "What is the Impact of Bad Layout in the Understandability of Social Goal Models?",
abstract = "The i∗ community has published guidelines, including model layout guidelines, for the construction of models. Our goal is to evaluate the effect of the layout guidelines on the i∗ novice stakeholders' ability to understand and review i∗ models. We performed a quasi-experiment where participants were given two understanding and two reviewing tasks. Both tasks involved a model with a bad layout and another model following the i∗ layout guidelines. We evaluated the impact of layouts by combining the success level in those tasks and the required effort to accomplish them. Effort was assessed using time, perceived complexity (with NASA TLX), and eye-tracking data. Participants were more successful in understanding than in reviewing tasks. However, we found no statistically significant difference in the success, time taken, or perceived complexity, between tasks conducted with models with a bad layout and models with a good layout. Most participants had little to no prior knowledge in i∗, making them more representative of stakeholders with no requirements engineering expertise. They were able to understand the models fairly well after a short tutorial, but struggled when reviewing models. In the end, adherence to the existing i∗ layout guidelines did not significantly impact i∗ model understanding and reviewing performance.",
keywords = "NASA, Complexity, Construction of models, Diagram layout, Eye-tracking, Model understanding, Social goals, Statistically significant difference, Understandability",
author = "Mafalda Santos and Catarina Gralha and Miguel Goul{\~a}o and Jo{\~a}o Ara{\'u}jo and Ana Moreira and Joao Cambeiro",
note = "Sem PDF conforme despacho. Nurses Organization of Veterans Affairs",
year = "2016",
month = "12",
day = "2",
doi = "10.1109/RE.2016.51",
language = "English",
pages = "206--215",
booktitle = "24th IEEE International Requirements Engineering Conference",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

Santos, M, Gralha, C, Goulão, M, Araújo, J, Moreira, A & Cambeiro, J 2016, What is the Impact of Bad Layout in the Understandability of Social Goal Models? in 24th IEEE International Requirements Engineering Conference., 7765526, Institute of Electrical and Electronics Engineers Inc., Beijing, pp. 206-215, 24th IEEE International Requirements Engineering Conference Workshops, REW 2016, Beijing, China, 12/09/16. https://doi.org/10.1109/RE.2016.51

What is the Impact of Bad Layout in the Understandability of Social Goal Models? / Santos, Mafalda; Gralha, Catarina; Goulão, Miguel; Araújo, João; Moreira, Ana; Cambeiro, Joao.

24th IEEE International Requirements Engineering Conference. Beijing : Institute of Electrical and Electronics Engineers Inc., 2016. p. 206-215 7765526.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - What is the Impact of Bad Layout in the Understandability of Social Goal Models?

AU - Santos, Mafalda

AU - Gralha, Catarina

AU - Goulão, Miguel

AU - Araújo, João

AU - Moreira, Ana

AU - Cambeiro, Joao

N1 - Sem PDF conforme despacho. Nurses Organization of Veterans Affairs

PY - 2016/12/2

Y1 - 2016/12/2

N2 - The i∗ community has published guidelines, including model layout guidelines, for the construction of models. Our goal is to evaluate the effect of the layout guidelines on the i∗ novice stakeholders' ability to understand and review i∗ models. We performed a quasi-experiment where participants were given two understanding and two reviewing tasks. Both tasks involved a model with a bad layout and another model following the i∗ layout guidelines. We evaluated the impact of layouts by combining the success level in those tasks and the required effort to accomplish them. Effort was assessed using time, perceived complexity (with NASA TLX), and eye-tracking data. Participants were more successful in understanding than in reviewing tasks. However, we found no statistically significant difference in the success, time taken, or perceived complexity, between tasks conducted with models with a bad layout and models with a good layout. Most participants had little to no prior knowledge in i∗, making them more representative of stakeholders with no requirements engineering expertise. They were able to understand the models fairly well after a short tutorial, but struggled when reviewing models. In the end, adherence to the existing i∗ layout guidelines did not significantly impact i∗ model understanding and reviewing performance.

AB - The i∗ community has published guidelines, including model layout guidelines, for the construction of models. Our goal is to evaluate the effect of the layout guidelines on the i∗ novice stakeholders' ability to understand and review i∗ models. We performed a quasi-experiment where participants were given two understanding and two reviewing tasks. Both tasks involved a model with a bad layout and another model following the i∗ layout guidelines. We evaluated the impact of layouts by combining the success level in those tasks and the required effort to accomplish them. Effort was assessed using time, perceived complexity (with NASA TLX), and eye-tracking data. Participants were more successful in understanding than in reviewing tasks. However, we found no statistically significant difference in the success, time taken, or perceived complexity, between tasks conducted with models with a bad layout and models with a good layout. Most participants had little to no prior knowledge in i∗, making them more representative of stakeholders with no requirements engineering expertise. They were able to understand the models fairly well after a short tutorial, but struggled when reviewing models. In the end, adherence to the existing i∗ layout guidelines did not significantly impact i∗ model understanding and reviewing performance.

KW - NASA

KW - Complexity

KW - Construction of models

KW - Diagram layout

KW - Eye-tracking

KW - Model understanding

KW - Social goals

KW - Statistically significant difference

KW - Understandability

U2 - 10.1109/RE.2016.51

DO - 10.1109/RE.2016.51

M3 - Conference contribution

SP - 206

EP - 215

BT - 24th IEEE International Requirements Engineering Conference

PB - Institute of Electrical and Electronics Engineers Inc.

CY - Beijing

ER -

Santos M, Gralha C, Goulão M, Araújo J, Moreira A, Cambeiro J. What is the Impact of Bad Layout in the Understandability of Social Goal Models? In 24th IEEE International Requirements Engineering Conference. Beijing: Institute of Electrical and Electronics Engineers Inc. 2016. p. 206-215. 7765526 https://doi.org/10.1109/RE.2016.51