Agile and Scrum

“Perfection is not attainable, but if we chase perfection we can catch excellence.”
-Vince Lombardi

Agile

Before Agile exists, whenever a software is developed, there may be a time lag between the formulation of business requirements and the delivery of the technology that answered those requirements. This actually led to a cancellation of many projects due to the fact that business requirements may change over time and the final product not meeting those requirements. The Agile principle is created to solve this problem. Rather than developing software from start to finish in a specific time-frame, agile allows an iterative approach to developing software by dividing work into several consumable increments. This way, whenever a change needs to occur, it can be implemented in the next increment directly.

Agile has four values and twelve principles. The four values is written in its manifesto. These are:

1. Individuals and interactions over processes and tools

It’s easy to overlook people when developing applications because we as developers mainly focus on the processes and tools that is necessary to make them. However, the people are the ones that respond to change and drive development processes. Therefore, valuing the people should be the logical thing to do.

In our experience, we had to change the way we think about the software development process. Whenever we are conversing with the client, we tend to want to go back into the details of the frameworks and the code that is written. We have been constantly reminded to not talk about that and instead communicate about what the clients actually understand and whether or not the features are implemented as desired.

2. Working software over comprehensive documentation

In the past, more time is used to create documentation for the final product release. This resulted in delays that should not have existed. However, this does not mean that documentation is not important. The agile methodology does not eliminate documentation, but made it so that developers only write what is necessary for them at the time and not more.

In the work that we have done, the documentation is done last, when the actual feature has been confirmed to work. Even then, the documentation is only done as needed. The things we are planning to put on the documentation are: how to deploy the software, what features are available on it and how to use them and the basic architecture and how each component communicates with each other.

3. Customer collaboration over contract negotiation

Previously, Waterfall was the main development model for many developers. It is the earliest process model to be introduced.

The work in waterfall is divided into several phases, which are: Requirements Analysis Phase, System Design Phase, Introduction Phase, Implementation Phase, Testing Phase, Deployment Phase and lastly, the Maintenance Phase. Each phase must be completed before the next phase can begin and there is no overlapping between phases. With the waterfall model, there is a stage where the customer is involved in the negotiation of features that needs to be implemented and the time-frame that needs to be followed before the project began. However, the customer involvement ends there. The Agile manifesto defines that a customer should always be involved from the start until the end of development to make sure all business requirements are being implemented correctly and whenever there are modifications to the requirements, the developers can be informed.

The fact that the customer is involved with the process from start to the end, is very useful. In my experience, there is a feature that is added mid-way into the development, which is the storage of auto-generated excel files in the cloud. This feature is not easy to implement as it involves adding another component to the architecture. In short, we have to find out a service that can store the files for as little as possible, and make sure that it is compatible with the existing application that is already developed and easy to use as well. We would also need the customer to agree upon this design change in the middle of the development process. If we were to use the waterfall model, we would have scrape and re-do the software from scratch, which includes re-writing the specifications all over again. If there are many changes which is equal or higher in complexity to what I described above, it can lead to the software never getting deployed.

4. Responding to change over following a plan

The problems of traditional software development requires developers to create an elaborate software development plan and executing it until the program is finished. With agile, whenever there is a change in business requirements, the development team can reorganize the priorities of the tasks that they do and add tasks as needed. This allows change to be implemented relatively quicker when compared to traditional software development.

When we are asked to implement the cloud storage feature that was described previously, and another product was added to our product backlog, which is the display of the excel files, it completely screws up our plan as we originally plan on doing one PBI per sprint. This forces us to work on more than one PBI per sprint as we only have 5 sprints for more than 6 PBIs.

Below are the twelve principles that the four principles were based on. To help you make sense of how this principles are implemented in a software developing project, I will also mention my experiences that are related to the principles below.

• Customer satisfaction through early and continuous software delivery: In each sprint review we try to make sure that all the features are implemented correctly and as desired by the client.
• Accommodate changing requirements throughout the development process: PBI items may be added or removed depending on the Product Owner’s decisions
• Frequent delivery of working software: In the first sprint review we present a software that has some features working. Gradual increase in features is presented every sprint review.
• Collaboration between the business stakeholders and developers throughout the project: We have a representative of the client that is fully involved in the process. The representative is contacted every time there are queries that needs to be addressed to as soon as possible.
• Support, trust, and motivate the people involved: When we have a developer that left early in the software development process, I try to motivate the remaining team members so that they are more inclined to get the work done in a timely manner.
• Enable face-to-face interactions: Face-to-face interactions are done every Daily Scrum Meeting, where we talk about what we have done and what we are going to do so that everybody knows.
• Working software is the primary measure of progress: In each sprint review, if the client has examined the features and accepted it, that means we are one step closer to the finished product.
• Agile processes to support a consistent development pace: We are heavily encouraged to work according to the work hours (and possibly more since progress is measured according to the working software that we have developed) and the high frequency of reviews made by fellow developers and also product owners and clients makes it so that maintaining a consistent development pace needs to be done in order for us to be successful.
• Attention to technical detail and design enhances agility: If a feature is not up to what the client wants, it may be rejected. Therefore we try to make sure that each feature is implemented according to the standards defined in the PBI document.
• Simplicity: Each PBI is separated into several tasks. Each task should be as little as possible to ease development.
• Self-organizing teams encourage great architectures, requirements, and designs: We are a self-organizing team in the sense that the choice of framework, architecture construction and design is made by us. The requirements though, is based on the needs of the client, which can change over time.
• Regular reflections on how to become more effective: This is done in the daily scrum meeting and also the sprint retrospective. The third sprint retrospective that we have done really emphasizes on this as we have failed our sprint.

Scrum

In our class we were told that the software development will involve the scrum framework. Therefore I seek to explain what that is and how it is implemented in our class routine here. But before that, I would like to explain a little bit about what scrum is.

According to the Scrum Guide, scrum is “A framework within which people can address complex adaptive problems, while productively and creatively delivering products of the highest possible value”.

It’s a framework that has been used quite a while, actually (since the early 1990s). However, it is still relevant today, and I believe that is the case due to the fact that it’s adaptable. Scrum does not specify the techniques that you need to use, and the processes that you need to employ. You can implement any technique and process that you need to deliver a product, in addition to using the scrum framework. You cannot, however, remove any elements from the framework, and still call it scrum.

The Scrum framework, though, requires you to form a team (called a scrum team) that is not huge (probably less than twelve), and each member of the team needs to have specific roles, some of which are probably unique to the scrum framework. The roles are as follows:

1. The Product Owner

The product owner manages the Product Backlog. In our class this would either be one of the Teaching Assistants, or in the case where a company wants us to develop a software for them, the product owner will be a representative that is directly sent from that particular company. There usually will only one product owner (at least, in our case, only one exists).

2. Scrum Master

The scrum master helps guide the whole team so that the team follows the scrum guidelines defined in the scrum guide. In our class this would be one of the Teaching Assistants.

3. Development Team

The development team is the one doing the actual development of the product and must deliver a releasable increment of “Done” (Defined according to the definition of done) for the product at the end of each sprint. They are given relative autonomy as to how the product is built and delivered.

Once a scrum team has been formed, it’s time for them to encounter events that are exclusively there due to using the Scrum Framework. These events are called ‘Scrum Events’ and these include:

Sprint Planning

This event is done before the sprint and is created by the collaboration of all the members in the team. In this event two main things are being answered:

• What can be delivered in the Increment resulting from the upcoming Sprint?
• How will the work needed to deliver the Increment be achieved?

This is one of the most difficult things to do, and if it fails, there will be problems later on in the actual sprint.

(The actual) Sprint

This event is when the development of the program occurs. Based on the planning done on the Sprint Planning Scrum Event, the development team develops software according to what is agreed upon.

Daily Scrum

This is a fifteen-minute time-boxed event in which the development team discusses about what happened in the previous day, which includes what we did, what we have learned, the problems that we have encountered (external and internal) and discusses plans for the next twenty-four hours of work. In our class, one day of scrum lasts for around three days (since we cannot afford to do full time without disregarding other courses) and daily scrums happen on Monday and Wednesday. In addition to those things, we added how many hours have we worked on before the current daily scrum and after the previous daily scrum to one of the things that we told our teammates so that we can make sure that the minimum hours of work that we should do is used up properly. Teammates can also comment on things that can be improved directly after a member has spoken.

Sprint Review

Sprint Review is the time where we meet up with the whole scrum team and all stakeholders (our partner in our case), and these actions are usually the ones that will be performed:

1. The product owner will explain what Product Backlog Items has been done and also explain the ones that are not done. To inspect whether or not the product backlog is done or not, the product owner will refer to the definition of done and the acceptance criteria for each product backlog item.
2. The development team will discuss what went well, what problems they encounter and if they have solved it, how they solved it.
3. The development team will demonstrate the results of the sprint by presenting the software in its current form after development in the sprint. Then for each item in the sprint backlog, the product owner will inspect whether or not they have been completed by referring to the definition of done for that particular item.
4. What will be done next will be discussed across all the members of the meeting. This will help quicken the sprint planning process. This may also affect the product backlog.
5. The product backlog will be inspected and revised if needed. PBIs can be added or subtracted as needed, and values could be changed. The result would be a revised product backlog.

What we did was slightly different, in that the client will be the ones that determine whether or not the product is accepted or rejected according to the acceptance criteria, whereas the definition of done, which is more technical (evident due to the presence of the 100% code coverage requirement, which non-tech people will usually not understand), will be inspected by the product owner.

Sprint Retrospective

This is where the team does an evaluation of how the last sprint went and what could be improved upon so that there will be less problems in the next sprint. It is usually done immediately after the sprint review. The participants are the Scrum Masters, Product Owners, and the whole development team. This is usually done within an hour. However, special cases may require more.

The things that are discussed here would usually be guided with these three questions:

1. What does the team need to start doing?
2. What does the team need to stop doing?
3. What does the team need to continue doing?

Of course, depending on the implementation, the guiding questions could be different. It could be more, it could be less. However, as long as the team found ways to improve upon the last sprint, the goal of the sprint retrospective will be achieved.

Sprint Cancellation

If an abrupt situation occurs, a sprint may be cancelled. It is very rare that this occurs. It is not cancelled when the scrum team discovers that it cannot meet it, because it can be poured on to the next sprint. A sprint review determines whether or not it will be poured as is, whether it will be split into multiple PBIs, and to receive feedback on what they should do so that it can be met on the next sprint. The sprint retrospective is important also in this aspect to help developers reflect on why it occured and how they can improve.

If a scrum goal is met early or is projected to be met early there are a variety of things that can be done. These namely include: Renegotiate to take on new tasks, giving free time to the developers as a reward for finishing the tasks, work on improvements such as refactoring or even investigating future sprints. This of course depends on the rules of the organization but typically, new tasks are renegotiated to be taken as long as it does not impact the current sprint goals.

Scrum Artifacts

1. Product Backlog

The product backlog is a list of all features, functions, requirements, enhancements, and fixes that needs to be implemented into the product. Split into PBIs which contains description, order, estimate and value. This document will be maintained by the Product Owner, and the way the value is determined is according to the business logic that needs to be done first.

2. Sprint Backlog

The sprint backlog is a list of the features that have been chosen from the backlog during the sprint planning meeting to be done on a particular sprint.

3. Increment

List of all completed product backlog information from all the sprints.

Artifact Transparency

The concept of artifact transparency is simple. Artifacts must be completely visible and known to the whole team, and the stakeholders. To ensure this, the Scrum Master must work together with the Product Owner, Development Team, and all other parties that are involved with the project to understand if the artifacts are completely transparent. In the presence of incomplete transparency, the Scrum Master must ensure that the artifacts are as transparent as possible, and Scrum Masters can detect their presence by looking at the differences between expected and real results.

The Sprint Goal

According to Visual Paradigm, a Sprint goal shows the desired outcome of an iteration that provides a shared goal to the team, which goal has to be defined before the team starts the Sprint in order to focus to get this goal. It is worked on by the Product Owner, along with the development team, and it is comprised one or two sentences which is elaborated by specific Product Backlog Items that is chosen to be done for a particular sprint.

The User Story

The user story is one of the parts of the product backlog that describes what should be implemented for that product backlog. It consists of three elements:

1. As A: The type of user that is going to be the subject of the User Story
2. I want to: The actions that a user wants to do
3. So that: The direct benefits of the action that the user wants to do

Bundled with it is the acceptance criteria (or acceptance criterias). Acceptance criterias are objective components which a user story’s functionality is judged. When a developer team has taken a PBI to be worked on, the results are tested according to this, and the definition of done. The main difference between an acceptance criteria and the definition of done is that the definition of done is applied for all user stories, whereas each PBI has its own acceptance criteria.

The Definition of Done

The Definition of Done is a living list of requirements that is agreed upon and understood by the scrum team. When all the items on a list is checked for a product increment, the product increment is ready to be released. It is required to be made to ensure transparency in terms of requirements.

When we talk about the contents of The Definition of Done, we should be able to divide it into three main components:

Business or Functional Requirements

Standard Business or Functional Requirements are categorized here. While this may contain other elements, the bare minimum of content that is categorized here is for the acceptance criteria for each PBI to be accepted.

Quality

Coding Standards, Test-Driven Development, Unit Test Coverage, Maintainability Index, Maximum Number of Defects, Maximum Technical Debt, Design Principles are put here.

Non-Functional Requirements

Availability, Maintainability, Performance, Reliability, Scalability, Security, Usability, Compliance/Regulatory and Legal are put here.

As I said before, the definition of done is a living list of requirements. This means that items can be added or subtracted from the list as the time goes on. Depending on how much items are in the list and when it is made, it can be categorized as Initial DoD, Mature DoD and Stringent DoD.

How Agile and Scrum is implemented in our case

Here is a list on how it is implemented to develop our app:

1. We keep an excel document that we use to track the requirements of the web application. The contents of the document is the same with any other Product Backlog document. It contains the Product Backlog Items, the name of the Product Owner, the project title and the timestamp of the last edit. This makes it clear as to what needs to be implemented, and what does not need to be implemented. This also makes it easier to split the functionalities to be done in multiple sprints, and gives us target features to be delivered on each sprint. Other than that, it makes it easier to know when the document is last edited so that we know why there are discrepancies when there are discrepancies between what we know and what is actually written.
2. Communication to the client is done through one of our TAs. This is especially important when we want to implement the algorithm that creates the schedules since it needs to be similar to how the client creates the schedule manually (May I remind you that our app is one that automatically generates test schedules?). The TAs help word the questions so that the client can easily understand them and can give us the answers that we need.
3. The Sprint Planning Meeting is done before a sprint starts. It is usually done on Thursday, which is the first day that a sprint starts, and the day when the sprint ends with a sprint review. The sprint planning for the subsequent sprints is done after the sprint retrospective. This is when we pick a Product Backlog Item to do, and divide the Product Backlog Item into small tasks, then assign them to separate people to be worked on. We divided the tasks to be so small, so that we can accurately portray the work that needs to be done, and so that the people that are doing the tasks aren’t overwhelmed by the complexity of the tasks that are needed to be done. Here we also decide which task to take first and plans to take other tasks later on if we have done the tasks that we have taken.
4. The development work that we do is based on what we have planned
5. The Daily Scrum Meeting is done on Mondays and Wednesdays. It should be done everyday but, again, we have to split one day of work into 2-3 days so that we can fit it inside our busy schedule (we are not doing this full time). The last Daily Scrum before the sprint review, in addition to doing the things that is usually done, is when we test the app, if possible, to verify whether or not it works as intended so that there are no hitches when the app is presented to the client on the day of the sprint review.
6. The Sprint Review Meeting is done on Thursday. In this meeting we present the current iteration of software and for the Product Backlog Items that we have taken, verify whether or not the acceptance criteria has been fulfilled, and whether or not our work satisfies the definition of done. If for a certain Product Backlog Item, both the acceptance criteria and the definition of done is fulfilled, then the product backlog item will be accepted. If one or both is not fulfilled, then the product backlog item will be denied. The one that determines this is our partner.
7. The Sprint Retrospective Meeting is done on Thursday as well, a few hours after we did our Sprint Review. The delay is due to the fact that we have classes after the sprint review, so it cannot be done immediately. This is done to help us reflect and improve ourselves so that the next sprint could be better. In the case where we fail, we brainstorm plans so that the next sprint will be successful.

Reference: