TY - JOUR
T1 - AgentDSM-Eval
T2 - A framework for the evaluation of domain-specific modeling languages for multi-agent systems
AU - Alaca, Omer Faruk
AU - Tezel, Baris Tekin
AU - Challenger, Moharram
AU - Goulão, Miguel
AU - Amaral, Vasco
AU - Kardas, Geylani
N1 - info:eu-repo/grantAgreement/FCT/3599-PPCDT/TUBITAK%2F0008%2F2014/PT#
info:eu-repo/grantAgreement/FCT/6817 - DCRRNI ID/UID%2FCEC%2F04516%2F2013/PT#
info:eu-repo/grantAgreement/FCT/6817 - DCRRNI ID/UIDB%2F04516%2F2020/PT#
PY - 2021/6
Y1 - 2021/6
N2 - Software development required for constructing multi-agent systems (MAS) usually becomes challenging and time-consuming due to the properties of autonomy, distributedness, and openness of these systems in addition to the complicated nature of internal agent behaviors and agent interactions. To facilitate MAS development, the researchers propose various domain-specific modeling languages (DSMLs) by enriching MAS metamodels with a defined syntax and semantics. Although the descriptions of these languages are given in the related studies with the examples of their use, unfortunately, many are not evaluated in terms of either the usability (being hard to learn, understand and use) or the quality of the generated artifacts. Hence, in this paper, we introduce an evaluation framework, called AgentDSM-Eval, with its supporting tool which can be used to evaluate MAS DSMLs systematically according to various quantitative and qualitative aspects of agent software development. The empirical evaluation, presented by the AgentDSM-Eval framework, was successfully applied for one of the well-known MAS DSMLs. The assessment showed that both MAS domain coverage of DSMLs and the agent developers’ adoption of modeling elements can be determined with this framework. Moreover, the tool's quantitative results can assess MAS DSML's performance on the development time and throughput. AgentDSM-Eval also enables the qualitative assessment of MAS DSML features according to novel quality characteristics and measures, which it defines specifically for the MAS domain.
AB - Software development required for constructing multi-agent systems (MAS) usually becomes challenging and time-consuming due to the properties of autonomy, distributedness, and openness of these systems in addition to the complicated nature of internal agent behaviors and agent interactions. To facilitate MAS development, the researchers propose various domain-specific modeling languages (DSMLs) by enriching MAS metamodels with a defined syntax and semantics. Although the descriptions of these languages are given in the related studies with the examples of their use, unfortunately, many are not evaluated in terms of either the usability (being hard to learn, understand and use) or the quality of the generated artifacts. Hence, in this paper, we introduce an evaluation framework, called AgentDSM-Eval, with its supporting tool which can be used to evaluate MAS DSMLs systematically according to various quantitative and qualitative aspects of agent software development. The empirical evaluation, presented by the AgentDSM-Eval framework, was successfully applied for one of the well-known MAS DSMLs. The assessment showed that both MAS domain coverage of DSMLs and the agent developers’ adoption of modeling elements can be determined with this framework. Moreover, the tool's quantitative results can assess MAS DSML's performance on the development time and throughput. AgentDSM-Eval also enables the qualitative assessment of MAS DSML features according to novel quality characteristics and measures, which it defines specifically for the MAS domain.
KW - Agent-oriented software engineering
KW - AgentDSM-Eval
KW - Domain-specific modeling language
KW - Evaluation framework
KW - Human factors in Modelling
KW - Multi-agent system
KW - Usability evaluation
UR - http://www.scopus.com/inward/record.url?scp=85100216898&partnerID=8YFLogxK
U2 - 10.1016/j.csi.2021.103513
DO - 10.1016/j.csi.2021.103513
M3 - Article
AN - SCOPUS:85100216898
SN - 0920-5489
VL - 76
JO - Computer Standards and Interfaces
JF - Computer Standards and Interfaces
M1 - 103513
ER -