Can a large language model, not specializing in anything “agile,” pass a screening interview for a Scrum Master position? In this ChatGPT job interview, I put OpenAI’s latest generative AI to the test. I took several questions from the 73 Scrum Master Interview Questions guide, see below, covering the whole spectrum from the broad picture to more specific questions, only answerable with hands-on experience on the interviewee’s side.
So, read on and learn whether Scrum Masters will soon be replaced with a chatbot.
🗞 Shall I notify you about articles like this one? Awesome! You can sign up here for the ‘Food for Agile Thought’ newsletter and join 36,000-plus other subscribers.
🎓 Join Stefan in one of his upcoming Professional Scrum training classes!
🖥 💯 🇺🇸 Professional Scrum Product Owner w/ PSPO I Certificate — January 23-26, 2023.
📈 Join 100-plus peers: Hands-on Agile 49: ChatGPT.
The task for this ChatGPT job interview session was to create a screening interview of about 30 minutes to decide whether to proceed with a fictitious application for a Scrum Master position by ChatGPT.
All questions are from the 73 Scrum Master Interview Questions guide, including guidance on suitable answers. (For example, “Q 03” refers to the third question of the guide.)
I ran the ChatGPT interview session on two days: On 2022-12-18, from 4:00-4:40 pm CET, and on 2023-01-08, from 6:10-6:45 am CET. I did not edit ChatGPT’s answers. I used follow-up questions to explore the model when ChatGPT did not provide obvious answers.
Question: Thank you for your application as a Scrum Master! Why would you be a good fit for the position?
ChatGPT’s answer:
Thank you for considering me for the Scrum Master position! I believe I would be a good fit for the role because I have a strong understanding of Scrum and the Agile framework, and I am able to communicate and facilitate effectively with team members and stakeholders. I am also able to handle conflicts and facilitate discussions to find solutions, and I am skilled at helping teams adhere to Scrum values and practices. In addition, I am able to adapt to new situations and environments, and I am committed to continuous learning and improvement.
Comment: Thumbs up; we just ignore the lack of working experience.
Follow-up question: Why is it beneficial that the team adheres to the values and practices of Scrum?
ChatGPT’s answer:
Adhering to the values and practices of Scrum can provide a number of benefits to a development team, including:
Overall, adhering to the values and practices of Scrum can help a development team work more effectively and efficiently, and deliver better results.
Comment: I am starting to like this!
Cannot see the form? Please click here.
Question: Should a Scrum Master remove problems on behalf of the Scrum Team?
Guidance: A Scrum Master should not be concerned with removing problems that the Scrum Team can solve themselves, no matter how often this requirement is mentioned in job advertisements. If a Scrum Master acts like a ‘Scrum parent’, their team will never become self-organizing.
A Scrum Team must learn to make its own decisions. Typically, this inevitably results in failures, dead-ends, and other unplanned excursions when the team is learning something new. Consequently, in the beginning, a team will need more guidance than usual from the Scrum Master. Such advice should not, however, become an exercise in protective “parenting” — a team must be allowed to learn from their failures.
That being said, there is one area where the Scrum Master is indeed removing problems on behalf of the team. This Scrum Master stance applies when the Scrum Team cannot solve the problem themselves, for example, because the issue is an organizational problem. Now we are talking about “impediments.” In this situation, the Scrum Master becomes the impediment remover of the Scrum Team.
ChatGPT’s answer:
It is not the responsibility of the Scrum Master to remove problems on behalf of the Scrum team. The Scrum Master's primary role is to serve the Scrum team and protect it from external interruptions and distractions. However, the Scrum Master can assist the team in finding ways to resolve problems and remove barriers to progress.
The Scrum team is responsible for solving their own problems and finding ways to improve their processes and work. The Scrum Master can help by coaching the team, facilitating discussions, and providing support and guidance as needed. However, it is important for the Scrum team to take ownership of their own problems and work together to find solutions, rather than relying on the Scrum Master to remove problems on their behalf.
Comments: ChatGPT’s answer is a reasonable interpretation of a Scrum Master’s accountabilities, stressing the importance of helping the team to support themselves. A team must embrace ownership of its problem and solution space and its way of working.
Question: You’ve already provided your product’s stakeholders with training in Scrum. However, after the initial phase of applying the concepts, when the first obstacles are encountered, some of these stakeholders begin to resist continued adoption. What is your strategy for and experience in handling these situations?
Guidance: This question is meant to encourage an exchange of ideas and lessons learned when overcoming resistance to Scrum within an organization. In addition, familiarity with agile failure patterns common to many organizations will demonstrate your candidate’s experience.
Your candidate should also be familiar with the challenge middle managers face in any transition to agile practices. For example, moving from a command-and-control style (i. e., managing people and telling them what to do) to a servant-leadership kind — thus abandoning Taylor’s principles — is not for everyone.
ChatGPT’s answer:
As a language model, I don't have personal experience in handling situations where stakeholders resist the continued adoption of Scrum.
However, here are some strategies that a Scrum Master might consider when faced with this challenge:
Regenerated Response:
There are a few strategies that a Scrum Master might use when stakeholders resist the continued adoption of Scrum:
As a language model, I do not have any personal experience with handling these types of situations. However, I can provide general guidance based on my understanding of Scrum and the principles of effective communication and conflict resolution.
Comment: This answer is an exciting moment in the ChatGPT job interview! ChatGPT admits having no first-hand experience in the situation. (Which would be surprising, to say the least.) However, ChatGPT also manages to overcome the awkward “I do not know” situation by moving to the metalevel and applying general communication and conflict resolution principles to a Scrum team’s stakeholder relations.
Question: What factors should a Scrum Team consider at the Sprint Planning to determine a feasible Sprint Goal?
Guidance: Typical criteria for a Scrum team to consider are, for example:
As any Scrum team plays an infinite game, there are no winners. Most likely, Stakeholders will regard a Scrum team as successful when it creates value for customers and the organization every single Sprint.
Therefore, from a team perspective, building rapport and trust with stakeholders requires more Wallstreet-like expectation management: stakeholders value a reliable delivery more than a sporadic outburst of productivity. This understanding should guide the Scrum team in determining feasible Sprint Goals.
ChatGPT’s answer:
There are a number of factors that a Scrum team should consider when determining a feasible Sprint Goal during the Sprint Planning event. These factors include:
By considering these factors, the team can set a feasible Sprint Goal that is challenging but achievable, and that will help the team make progress towards the long-term goals of the project.
Follow-up question: What about technical debt and available skills?
ChatGPT’s answer:
Technical debt and available skills are also important factors to consider when determining a feasible Sprint Goal.
Technical debt refers to the accumulation of technical challenges and problems that a team has not yet addressed. These challenges can slow down the team's progress and make it harder to achieve the Sprint Goal. When determining a feasible Sprint Goal, the team should consider the amount of technical debt that needs to be addressed and whether they have the capacity to address it within the sprint.
Available skills are another important factor to consider. The team should consider the skills and expertise of each team member and how they can be leveraged to achieve the Sprint Goal. If the team lacks certain skills that are needed to complete the work, they may need to adjust the goal or find ways to acquire the necessary skills.
By considering technical debt and available skills, the team can set a feasible Sprint Goal that takes into account the challenges and constraints that may impact their ability to achieve it. This can help the team work more effectively and efficiently, and ultimately deliver better outcomes for the project.
Comment: Okay, this part of the ChatGPT job interview needed a follow-up question to also identify existing technical debt and the team’s skill sets as determining factors for the Sprint Goal determination. Otherwise, identifying five out of eight factors straight away is not bad either.
Question: Is it okay to use a ‘Definition of Ready’ as a Scrum Team?
Guidance: Using a ‘Definition of Ready’ depends on the Scrum team’s situation. For example, suppose it is a junior team still struggling with the mechanics of Scrum. In that case, it might be a temporarily helpful way of taking some of the pressure off the team during Product Backlog management, refinement, and Sprint Planning. On the other hand, a senior Scrum team will have found its mojo and won’t need to rely on training wheels.
However, suppose the ‘Definition of Ready’ is used dogmatically as a checklist, rejecting all work items during Sprint Planning that are not 100 percent covered by this new standard. In that case, you are reintroducing waterfall through the backdoor; only the Developers are doing that this time.
Even worse would be the organization’s use of a ‘Definition of Ready’ as a team metric or indicator for a Scrum team’s “fluency” in agile product development.
ChatGPT’s answer:
Yes, it is generally acceptable for a Scrum team to use a "Definition of Ready" (DoR) to guide their work. A DoR is a set of criteria that a team defines to determine when a product backlog item is ready to be included in a sprint.
The purpose of a DoR is to ensure that the team has a clear understanding of the work that needs to be done and is fully prepared to complete it. By having a DoR, the team can avoid starting work on items that are not fully understood or that are not ready to be completed, which can help them work more efficiently and effectively.
A DoR should be specific to the needs of the team and the project, and may include criteria such as:
Using a DoR can help a Scrum team ensure that they are working on the right items at the right time and that they have the necessary information and resources to complete the work. However, it's important to note that the DoR should be a living document that the team continually updates and adjusts as needed to meet the changing needs of the project.
Comment: While I am not generally supportive of applying a DoR as these kinds of training wheels can quickly turn into something malicious, ChatGPT delivered the positive aspects of having one. In a ChatGPT job interview, though, I would appreciate a hint that the DoR concept extends to Scrum.
Question: What can a Scrum team do to get stakeholders to attend the Sprint Review?
Guidance: In my experience, you need to “sell” the Sprint Review within the organization, at least at the beginning of using Scrum:
Tackle the problem by addressing it at the next Retrospective and ask for the support of the whole Scrum team to remedy the issue; it is not just something between the Product Owner and the Scrum Master.
ChatGPT’s answer:
There are a few things that a Scrum team can do to encourage stakeholders to attend the Sprint Review:
Overall, it's important for the team to communicate the value of the Sprint Review to stakeholders and make the meeting as engaging and interactive as possible to encourage attendance.
Comment: Another acceptable ChatGPT job interview answer. I would give bonus points for pointing to the necessity to educate stakeholders on how Scrum as a framework is supposed to work and how stakeholders can contribute to a Scrum team’s success; for example, by attending the Sprint Review and providing feedback.
Question: Your Scrum Team consistently fails to meet forecasts and Sprint Goals, and their velocity is volatile. What are the probable reasons for this problem, and how would you address it with the team?
Guidance: Your candidate should discuss the notoriety of ‘velocity’ as the industry’s most prevalent metric for measuring a team’s progress. They should further be able to explain why velocity is altogether a doubtful agile metric and point out that quantitative metrics are not ideally suited to measuring a team’s progress in mastering Scrum.
Many factors contribute to a Scrum Team’s velocity volatile:
Another common cause for a Scrum Team to consistently fail in meeting their forecasts is that the team’s Product Backlog items are poorly prepared, thus making the work items difficult for the team to understand. Conversely, the projects given to the team might suffer from poorly documented legacy code, excessive technical debt, or too much buggy and badly written code — all of which make estimation a gamble.
Finally, your candidate should not align themselves with the fallacy that a team’s adoption of Scrum is working only because a Scrum Team’s forecasts and velocity are aligned. Cooking the agile books is easy to do!
ChatGPT’s answer:
There are a number of reasons why a Scrum team may consistently fail to meet forecasts and Sprint Goals, and their velocity may be volatile.
Some possible reasons for this problem include:
To address these problems with the team, the Scrum Master could:
By addressing these issues and helping the team develop the necessary skills and processes, the Scrum Master can help the team make progress towards their goals and improve their velocity.
Comment: ChatGPT mentioned a few general things that help with predictability. However, insisting on an “accurate[ly] forecast” is problematic, as there is no such thing in a complex environment. Also, there is no mention of the importance of Product Backlog refinement and the early involvement of team members in the general product discovery progress, all of which help to improve forecasting. The ideas of how a Scrum Master could support their team are merely a repetition of the first list and thus not helpful with the ChatGPT job interview in general.
So far, this is the weakest answer of the ChatGPT job interview for a Scrum Master position.
Question: How would you create the first Scrum Team in your organization?
Guidance: When an organization is transitioning to Scrum and at the same time dealing with significant organizational, business, and technical problems, the founding members of its Scrum Teams should be volunteers who fully understand the challenge ahead of them rather than people pressed into service. The best volunteers are those eager to prove that becoming agile is the most effective way to reach an objective.
Candidates for the role of Scrum Master should be astute enough to suggest inviting every member of the product delivery team, as well as the C-level executives sponsoring the transition, to a kickoff meeting. The objective of a transition kickoff meeting is to support the members of the engineering and product teams in how they choose to self-organize into the first cross-functional Scrum Teams. Transition kickoff meetings can last a few hours or several days, depending on the circumstances of a particular organization.
Despite the importance of the kickoff meeting to a Scrum transition, going much deeper into its structure will take too much time from the interview. So instead, it’s more important that your candidates embrace the idea of team self-selection and present a brief roadmap of what should happen next for the newly formed Scrum Teams.
Although somewhat dependent upon the existing skills, experience, and training of the members of an organization’s new Scrum Teams, your candidates should anticipate having to teach the very basics of Scrum following a kickoff meeting. They might propose doing this through a series of workshops or on-the-job training with exercises in Product Backlog refinement, writing user stories, estimating, creating boards, and setting up collaboration software.
ChatGPT’s answer:
To create the first Scrum team in an organization, there are a few steps that can be followed:
By following these steps, the team can effectively establish themselves as a Scrum team and begin working using the Scrum framework.
Regenerated response:
To create the first Scrum team in an organization, the following steps could be taken:
By following these steps, the organization can create a strong and effective Scrum team that is well-equipped to work effectively and deliver value to the organization.
Comment: This ChatGPT job interview answer is mainly unrelated to Scrum but is general project management best practice to get started. It completely ignores the idea of self-management and Scrum Values, which ChatGPT previously put on a pedestal. Regenerating the reply did not help either.
Follow-up question: Shouldn’t members of a Scrum team self-select? After all, Scrum teams are self-managing.
ChatGPT’s answer:
In Scrum, the team is responsible for self-organizing and self-managing its work. As such, it is generally best practice for team members to self-select onto the team, rather than being assigned to the team by someone outside the team. This is because self-selection allows team members to choose to work on the team based on their skills, interests, and availability, and can help create a sense of ownership and commitment to the team's success.
However, there may be cases where it is necessary for someone outside the team to play a role in selecting team members. For example, if the team is working on a project that requires specialized skills or expertise that are not available within the team, it may be necessary for someone outside the team to help identify and bring in team members with those skills. In these cases, it is important to involve the existing team members in the selection process and ensure that their input is considered.
Overall, it is generally best practice for Scrum teams to be composed of team members who self-select onto the team, but there may be cases where outside input is necessary to ensure that the team has the necessary skills and expertise to be successful.
Follow-up question: Why can’t Scrum team members identify what skills they miss and a suitable expert to compensate for that lack of expertise?
ChatGPT’s answer:
Scrum team members are responsible for self-organizing and self-managing their work, and as such, they are generally expected to identify any skills or expertise that they may be lacking and find ways to compensate for those gaps. This may involve seeking guidance or support from subject matter experts, or acquiring the necessary skills through training or other means.
There may be cases where the team is unable to identify the specific skills or expertise that they are lacking, or where they are unable to find a suitable expert to compensate for those gaps. In these cases, it may be necessary for someone outside the team, such as the Scrum Master or a manager, to help identify the skills or expertise that are needed and to assist in finding a suitable expert to fill those gaps.
Overall, it is important for Scrum teams to be proactive in identifying and addressing any skills or expertise that they may be lacking, and to seek out the support and resources they need to be successful.
Comment: ChatGPT believes in the benefits of “guidance from the management” in setting up Scrum teams or from the Scrum Master, as the Scrum team may be unable to identify what skills it is lacking. While our language model is still “convinced” of the benefits of managerial caste, it is my experience that the most skilled Scrum teams very well understand what they do not know (yet) and where they need support from outside experts. In no case, they required their managers to tell them.
The ChatGPT job interview is a mixed bag. The more specific the questions get, the further away from the Scrum Guide and Scrum’s first principles they move ChatGPT increasingly starts to fail. This failure should not be a surprise as we do not know on what content this LLM was trained.
So, based on these few example questions, I would not have taken the next step in the Scrum Master interview process, inviting ChatGPT to a full-size interview with several Scrum team members. Not yet.
However, this ChatGPT job interview also allows a glimpse into the future: What if we could train ChatGPT 4.0 ourselves? Of course, I am not referring to the whole model, but the agile and Scrum specifics parts, respectively. How much training effort does it take to get ChatGPT to a level that would make us interviewers engage in the next step with the model?
Moreover, what documents would you choose to train ChatGPT 4.0 in Agile and Scrum? Please share your suggestions with us in the comments.
Agility and Scrum According to OpenAI’s ChatGPT — Be Surprised!
How to use ChatGPT in product design: 8 practical examples
Free Ebook: 73 Scrum Master Interview Questions to Identify Suitable Candidates.
Delve into understanding the ChatGPT job interview challenges with our Scrum training classes, workshops, and events. You can secure your seat directly by following the corresponding link in the table below:
See all upcoming classes here.
You can book your seat for the training directly by following the corresponding links to the ticket shop. If the procurement process of your organization requires a different purchasing process, please contact Berlin Product People GmbH directly.
I invite you to join the “Hands-on Agile” Slack Community and enjoy the benefits of a fast-growing, vibrant community of agile practitioners from around the world.
If you like to join all you have to do now is provide your credentials via this Google form, and I will sign you up. By the way, it’s free.
Support your team’s efforts for the ChatGPT job interview by pointing to the free Scrum Anti-Patterns Guide:
TL; DR: Scrum Master Tasks: Let's Bust Some Myths! Rumor says that a great Scrum…
TL; DR: Scrum Master Interview Questions on Creating Value with Scrum If you are looking…
TL; DR: The Product Operating Model — An Interview with Marty Cagan Let’s explore Marty…