On the impact of semantic transparency on understanding and reviewing social goal models

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

Context: i∗ is one of the most influential languages in the Requirements Engineering research community. Perhaps due to its complexity and low adoption in industry, it became a natural candidate for studies aiming at improving its concrete syntax and the stakeholders' ability to correctly interpret i∗ models. Objectives: We evaluate the impact of semantic transparency on understanding and reviewing i∗ models, in the presence of a language key. Methods: We performed a quasi-experiment comparing the standard i∗ concrete syntax with an alternative that has an increased semantic transparency. We asked 57 novice participants to perform understanding and reviewing tasks on i∗ models, and measured their accuracy, speed and ease, using metrics of task success, time and effort, collected with eye-tracking and participants' feedback. Results: We found no evidence of improved accuracy or speed attributable to the alternative concrete syntax. Although participants' perceived ease was similar, they devoted significantly less visual effort to the model and the provided language key, when using the alternative concrete syntax. Conclusions: The context provided by the model and language key may mitigate the i∗ symbol recognition deficit reported in previous works. However, the alternative concrete syntax required a significantly lower visual effort.

Original languageEnglish
Title of host publicationProceedings - 2018 IEEE 26th International Requirements Engineering Conference, RE 2018
EditorsDaniel Amyot, Walid Maalej, Guenther Ruhe
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages228-239
Number of pages12
ISBN (Electronic)9781538674185
DOIs
Publication statusPublished - 12 Oct 2018
Event26th IEEE International Requirements Engineering Conference, RE 2018 - Banff, Canada
Duration: 20 Aug 201824 Aug 2018

Conference

Conference26th IEEE International Requirements Engineering Conference, RE 2018
CountryCanada
CityBanff
Period20/08/1824/08/18

Fingerprint

Transparency
Semantics
Concretes
Requirements engineering
Engineering research
Feedback
Industry
Experiments

Keywords

  • Eye tracking
  • IStar
  • Physics of notations
  • Social goal models

Cite this

Santos, M., Gralha, C., Goulão, M., Araújo, J., & Moreira, A. (2018). On the impact of semantic transparency on understanding and reviewing social goal models. In D. Amyot, W. Maalej, & G. Ruhe (Eds.), Proceedings - 2018 IEEE 26th International Requirements Engineering Conference, RE 2018 (pp. 228-239). [8491138] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/RE.2018.00031
Santos, Mafalda ; Gralha, Catarina ; Goulão, Miguel ; Araújo, João ; Moreira, Ana. / On the impact of semantic transparency on understanding and reviewing social goal models. Proceedings - 2018 IEEE 26th International Requirements Engineering Conference, RE 2018. editor / Daniel Amyot ; Walid Maalej ; Guenther Ruhe. Institute of Electrical and Electronics Engineers Inc., 2018. pp. 228-239
@inproceedings{1bac686ec9414ac99fae556519efc034,
title = "On the impact of semantic transparency on understanding and reviewing social goal models",
abstract = "Context: i∗ is one of the most influential languages in the Requirements Engineering research community. Perhaps due to its complexity and low adoption in industry, it became a natural candidate for studies aiming at improving its concrete syntax and the stakeholders' ability to correctly interpret i∗ models. Objectives: We evaluate the impact of semantic transparency on understanding and reviewing i∗ models, in the presence of a language key. Methods: We performed a quasi-experiment comparing the standard i∗ concrete syntax with an alternative that has an increased semantic transparency. We asked 57 novice participants to perform understanding and reviewing tasks on i∗ models, and measured their accuracy, speed and ease, using metrics of task success, time and effort, collected with eye-tracking and participants' feedback. Results: We found no evidence of improved accuracy or speed attributable to the alternative concrete syntax. Although participants' perceived ease was similar, they devoted significantly less visual effort to the model and the provided language key, when using the alternative concrete syntax. Conclusions: The context provided by the model and language key may mitigate the i∗ symbol recognition deficit reported in previous works. However, the alternative concrete syntax required a significantly lower visual effort.",
keywords = "Eye tracking, IStar, Physics of notations, Social goal models",
author = "Mafalda Santos and Catarina Gralha and Miguel Goul{\~a}o and Jo{\~a}o Ara{\'u}jo and Ana Moreira",
note = "info:eu-repo/grantAgreement/FCT/5876/147279/PT# SFRH/BD/108492/2015",
year = "2018",
month = "10",
day = "12",
doi = "10.1109/RE.2018.00031",
language = "English",
pages = "228--239",
editor = "Daniel Amyot and Walid Maalej and Guenther Ruhe",
booktitle = "Proceedings - 2018 IEEE 26th International Requirements Engineering Conference, RE 2018",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
address = "United States",

}

Santos, M, Gralha, C, Goulão, M, Araújo, J & Moreira, A 2018, On the impact of semantic transparency on understanding and reviewing social goal models. in D Amyot, W Maalej & G Ruhe (eds), Proceedings - 2018 IEEE 26th International Requirements Engineering Conference, RE 2018., 8491138, Institute of Electrical and Electronics Engineers Inc., pp. 228-239, 26th IEEE International Requirements Engineering Conference, RE 2018, Banff, Canada, 20/08/18. https://doi.org/10.1109/RE.2018.00031

On the impact of semantic transparency on understanding and reviewing social goal models. / Santos, Mafalda; Gralha, Catarina; Goulão, Miguel; Araújo, João; Moreira, Ana.

Proceedings - 2018 IEEE 26th International Requirements Engineering Conference, RE 2018. ed. / Daniel Amyot; Walid Maalej; Guenther Ruhe. Institute of Electrical and Electronics Engineers Inc., 2018. p. 228-239 8491138.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - On the impact of semantic transparency on understanding and reviewing social goal models

AU - Santos, Mafalda

AU - Gralha, Catarina

AU - Goulão, Miguel

AU - Araújo, João

AU - Moreira, Ana

N1 - info:eu-repo/grantAgreement/FCT/5876/147279/PT# SFRH/BD/108492/2015

PY - 2018/10/12

Y1 - 2018/10/12

N2 - Context: i∗ is one of the most influential languages in the Requirements Engineering research community. Perhaps due to its complexity and low adoption in industry, it became a natural candidate for studies aiming at improving its concrete syntax and the stakeholders' ability to correctly interpret i∗ models. Objectives: We evaluate the impact of semantic transparency on understanding and reviewing i∗ models, in the presence of a language key. Methods: We performed a quasi-experiment comparing the standard i∗ concrete syntax with an alternative that has an increased semantic transparency. We asked 57 novice participants to perform understanding and reviewing tasks on i∗ models, and measured their accuracy, speed and ease, using metrics of task success, time and effort, collected with eye-tracking and participants' feedback. Results: We found no evidence of improved accuracy or speed attributable to the alternative concrete syntax. Although participants' perceived ease was similar, they devoted significantly less visual effort to the model and the provided language key, when using the alternative concrete syntax. Conclusions: The context provided by the model and language key may mitigate the i∗ symbol recognition deficit reported in previous works. However, the alternative concrete syntax required a significantly lower visual effort.

AB - Context: i∗ is one of the most influential languages in the Requirements Engineering research community. Perhaps due to its complexity and low adoption in industry, it became a natural candidate for studies aiming at improving its concrete syntax and the stakeholders' ability to correctly interpret i∗ models. Objectives: We evaluate the impact of semantic transparency on understanding and reviewing i∗ models, in the presence of a language key. Methods: We performed a quasi-experiment comparing the standard i∗ concrete syntax with an alternative that has an increased semantic transparency. We asked 57 novice participants to perform understanding and reviewing tasks on i∗ models, and measured their accuracy, speed and ease, using metrics of task success, time and effort, collected with eye-tracking and participants' feedback. Results: We found no evidence of improved accuracy or speed attributable to the alternative concrete syntax. Although participants' perceived ease was similar, they devoted significantly less visual effort to the model and the provided language key, when using the alternative concrete syntax. Conclusions: The context provided by the model and language key may mitigate the i∗ symbol recognition deficit reported in previous works. However, the alternative concrete syntax required a significantly lower visual effort.

KW - Eye tracking

KW - IStar

KW - Physics of notations

KW - Social goal models

UR - http://www.scopus.com/inward/record.url?scp=85056807494&partnerID=8YFLogxK

U2 - 10.1109/RE.2018.00031

DO - 10.1109/RE.2018.00031

M3 - Conference contribution

SP - 228

EP - 239

BT - Proceedings - 2018 IEEE 26th International Requirements Engineering Conference, RE 2018

A2 - Amyot, Daniel

A2 - Maalej, Walid

A2 - Ruhe, Guenther

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Santos M, Gralha C, Goulão M, Araújo J, Moreira A. On the impact of semantic transparency on understanding and reviewing social goal models. In Amyot D, Maalej W, Ruhe G, editors, Proceedings - 2018 IEEE 26th International Requirements Engineering Conference, RE 2018. Institute of Electrical and Electronics Engineers Inc. 2018. p. 228-239. 8491138 https://doi.org/10.1109/RE.2018.00031