Assessing user stories: the influence of template differences and gender-related problem-solving styles

User stories are often used for elicitation, specification, and prioritisation of requirements. There are several user story templates, varying in their structure (particularly, in the order in which they present their elements), and also in the elements they use (namely, some user story templates include personas, while others do not). The potential benefits and shortcomings of choosing one template over another need to be understood, so that practitioners can make informed choices on the extent to which a particular user story template can affect user stories’ quality, leading to ambiguity, lack of completeness, or accidental complexity. Our goal was to analyse the differences between 4 alternative user story templates when creating and understanding user stories. In addition, we used the GenderMag framework to assess the effects of different problem-solving styles, usually associated with gender, while performing the tasks. We conducted a quasi-experiment. We asked 41 participants to perform creation and understanding tasks with the different user story templates. We measured their effectiveness using metrics of task success; their speed, with task duration; their visual effort, collected with an eye-tracker; and their perceived effort, with the NASA-TLX questionnaire. We characterised the participants’ problem-solving styles with a GenderMag questionnaire. Regarding the impact of the different templates in creating user stories, we observed statistically significant differences in some of the metrics for effectiveness, speed, and visual effort. We observed small differences in the participants’ visual effort while understanding user stories specified with different templates. Concerning the impact of different problem-solving styles, in the creation tasks, we found differences in time, visual effort, and perceived effort. Regarding understanding tasks, we observed differences in effectiveness, time, and visual effort, but not in their perceived effort. Although some templates outperformed others in a few metrics, no template obtained the best overall result. As such, we found no compelling evidence that one template is “better” than the others. This suggests the recommended template may depend on the goal we want to achieve, e.g., effectiveness or speed. The differences associated with the different problem-solving styles also suggest no overall superiority of any of those styles, hinting at a gender-neutral effect of the different templates. These preliminary conclusions should be further consolidated via the conduction of replications.