SaPeer and ReverseSaPeer Approaches for Training Students in Requirements Elicitation Interviews---Educational Material

This training package includes the educational material needed to practice the SaPeer approach for training students in requirements elicitation interviews. With SaPeer, students receive an initial lecture, followed by a role-playing interview experience with a fictional customer. Then, they receive a second lecture in which the typical mistakes of student analysts are listed, together with recommendations to avoid them. Based on the lecture, they are asked to listen to their own interview recording, and perform self-assessment by evaluating the mistakes committed. Then, they are also required to peer-review for mistakes the interview of another student. After this activity, they perform a second interview, which can be also self-assessed and peer-reviewed. At the end of the training, the students are required to reflect on their experience through a feedback questionnaire.  

The training package includes the following files.

1. Preliminary Training.pdf: slides of the preliminary training
2. Product Description 1st Interview.pdf: description of the product to be used for role playing in the 1st interview
3. Mistake-based Training.pdf: slides of the training on typical students' mistakes
4. Self-assessment Questionnaire.pdf: self-assessment questionnaire
5. Peer-review Questionnaire.pdf: peer-review questionnaire
6. Product Description 2nd Interview.pdf: description of the product to be used for role playing in the 2nd interview
7. Feedback Questionnaire.pdf: final questionnaire to get feedback from the students

The package includes also the .zip file titled AnonymizedReflections.zip, which includes student reflections given as feedback to ReverseSaPeer, which is a variant of the SaPeer approach with reverse role-playing. 

More information is available in the publication:

Ferrari, A., Spoletini, P., Bano, M., & Zowghi, D. (2019, September). Learning requirements elicitation interviews with role-playing, self-assessment and peer-review. In 2019 IEEE 27th international requirements engineering conference (RE) (pp. 28-39). IEEE. DOI: https://doi.org/10.1109/RE.2019.00015 

Video lectures associated to the slides in Preliminary Training.pdf, and Mistake-based Training.pdf are also available online:

- Preliminary Training: https://www.youtube.com/watch?v=kjaOuOYsdUM&feature=youtu.be

- Mistake-based Training: https://www.youtube.com/watch?v=p3z8eozPzeA&feature=youtu.be

Learning Outcomes and Context

The activities included in the presented learning package are appropriate for a module to be integrated in a Requirements Engineering (RE) course, in which the topics of requirements elicitation, analysis, specification, validation, and maintenance are covered. 

We are assuming a 2 to 5 credit course (1 credit = 45 min to 1 hour lecture), either graduate or undergraduate, that can be delivered face to face, online or with a hybrid modality, with the goal of covering the main RE activities.

A module based on the training package contributes to the ability of elicit requirements. The specific learning outcomes of the module are: 

1. The ability to conduct requirements elicitation interviews with relevant stakeholders.
2. The ability to analyze the execution and the content of requirements elicitation interviews.

The different parts of the module contribute together to these outcomes. Both executing and reviewing interviews contribute to creating and improving the students skills to conduct interviews (1). Reviewing and reflecting on the interviews support students in learning how to listen to and learn from interviews (2).

Possible additional learning outcomes for an RE course that integrates the module could be: 

1. The ability to analyse requirements by means of model-based RE approaches (goal-oriented, UML-based).
2. The ability of develop requirement specifications following the ISO/IEC/IEEE 29148:2011 Standard.
3. The ability to specify requirements in the form of user stories, or other templates (e.g., EARS, Chris Rupp's Template).
4. The ability to evaluate requirements quality by means of manual or semi-automated reviews.  
5. The ability to validate requirements with relevant stakeholders.

Assessment

The final assessment of the students should be based on the evaluation of the information that they are able to document after the requirements elicitation activities. 

To this end, at the end of the SaPeer process students should be asked to submit an initial list of requirements in the form of user stories for each of the products: 

As a <type of user>, I want <some goal> so that <some reason> 

Since listening to interviews and writing the requirements should also help students to realize what they have missed (i.e., things that they should have asked and did not, incomplete information, etc.), students should be also asked to list this information in the submission, as a witness of their ability to reflect on their practice, and to listen to their peers. 
Therefore, the evaluation should depend not only on the list of requirements, but also on these considerations provided by the students.  

In particular, the assessment should be based on the following dimensions:

• completeness of the list of requirements (with respect to the elicited information and not the original problem) - 40% of the grade 
• redundancy (i.e., replicated information in the requirements list) - 10% of the grade
• format of the requirements (i.e., compliance with the user story format) - 10% of the grade
• completeness and depth of the interview analysis (the includes: missed topics, missed questions and follow up, misunderstandings, ...) - 40% of the grade 

This assessment approach is the one used to evaluate the students involved in the first implementation of SaPeer. 

Guidelines to Apply the Approach

In an RE course, the module can be used in the classes dedicated to elicitation. The module can be used as a homework/project or as substitutive materials (in lieu of in class lectures). Students should be given around 3 weeks to work on all the activities of the module. 

A suggested timeline to apply the approach is the following:

• [Days 1-2]: Students watch the "Preliminary Training" video on interviews
• [Days 3-7]: Schedule and execute the 1st interview
• [Days 8-10]: Students watch the "Mistake-based Training" video and review the 1st interview (self- and peer-review)
• [Days 11-15]: Schedule and execute the 2nd interview
• [Days 16-18]: Review of 2nd interview (self- and peer-review)
• [Days 19-21]: Production and submission of the requirements in the form of user stories

The interviews can be conducted via Skype or in a classroom environment. 

The approach requires one or more tutors to play the role of customer in the interviews. The tutor is also in charge of recording and distributing the interviews to be reviewed. Therefore, the role of the tutor and their responsibility is crucial.   

Tutors playing the role of customers need to study the material presented in Product Description 1st Interview.pdf and in Product Description 2nd Interview.pdf beforehand, in order to better perform their role.  

Guidelines for Tailoring the Approach

According to the presented study, the most useful and effective activities are the role-playing interviews. Therefore, in case there is insufficient time to apply the whole approach, this can be tailored by removing one or more of the self- or peer-review activities. 

In case more time is available, one can iterate further with additional interviews. The same products can be considered, with tutors playing the role of different stakeholders. This requires more resources also in terms of tutors.  

Depending on the number of students, multiple tutors may be required. If tutors are not available, students can play the role of customers (so called role-reversal approach). This can be also didactically effective. However, as the ability of the customer is crucial to have a realistic interview, a trained tutor should be preferred.