Practical DevOps Use Cases for Enterprise IT

Minoru KATO

2018/05/24

"itSMF Japan 65th Seminar" sponsored by itSMF Japan [1] which focused on “DevOps”, was held on March 2, 2018. His presentation was about NTT COMWARE‘s efforts for DevOps, including challenges facing DevOps, and how to resolve them during installation and implementation.

DevOps was born to adapt the market changes

In today's IT system development, Systems of Engagement (SoE) focusing on customer connection has attracted a lot of attention and also it emphasizes ongoing relationships with customers. Therefore development techniques that can quickly capture feedbacks, this is what is called “Agile development”, becomes necessary. However, as you release functions sequentially in steps like “Agile development”, certain problems will occur. One is trouble in handover from development team to operation team.

With the release (delivery) of the developed IT system, development teams hand over their documents of the systems and operations, and operation tests, etc. to operation teams. Furthermore, if new functions and operations are frequently provided, it will become difficult to handle in both teams.

The idea that developers are able to deploy applications 10 or more times by taking charge of operational works partially was born to solve the tradeoff problem. Even if the operators prohibit the developers to touch the system (because of the role of operators is stably operate the application for their customers), both teams adapt to the rapidly changing market by trusting and respecting each other. This is the beginning of DevOps.

Modern DevOps is not limited to cooperate between dev and ops

Why do we want to deploy software many times a day?
Is it because there are several bugs to fix software programs? The answer is wrong. We release a website to make its flow better as soon as possible. Furthermore, we release the website as soon as requested by customers, release it as soon as pageview increases, and release as soon as rating goes up. In this way, we maximize the value that customers gain from using our IT systems and constantly develop. Daily deployment increases as this sequence repeats.

DevOps build feedback loops that quickly incorporate the customer experience of “business” requirements like Agile which can quickly incorporate the customer experience of “functional” requirements.

Therefore, in addition to the attempt that responding the rapidly changing market by building trustful relationship between development and operation teams, we will continue to develop business by maximizing the value provided by our IT systems.
This is the contemporary DevOps, we think.

The real goal of DevOps is expanding business

Many divisions such as development, operation and planning will build one circulatory cycle to try to incorporate feedback quickly of customer experiences. When a lot of groups with different goals and cultures come to have complex contacts that cannot be represented by systematic business flows, various issues will appear. These matters can be said to be negative effects of the silo type organization, which is the optimal type formed for the old business.

NTT COMWARE CORPORATION [2] recognizes that the solution to these problems is the essence of DevOps. Moreover we aim DevOps trying to find out the business expansion by returning customer experience to the value generated by IT systems.
This way of thinking can also be applied to the digital transformation era that is coming.

The practice of DevOps is to solve problems related to organization, people, information and technology for customers.

Next, I will talk about problems when starting with DevOps development based on my own experience.

Understanding the significance of practicing DevOps

The first challenge is to put yourself in the position of others.
Until now, we were instructed "Do not proceed with work by individual thought, and follow the standard!”. Suddenly one day, we are told "The future is SoE, work in consideration of customer’s feelings and relationships". With this inconsistent argument, it is difficult to switch our way of thinking quickly. Therefore we start with understanding why we work with SoE and DevOps at first.

Next step is transformation of action characteristics.
In order to adopting DevOps, everyone, regardless of developers, operators, or planners, needs to feel the customer experience. In an extreme case, there is a possibility that a member who does not feel it will hinder DevOps sooner or later. For that reason, we always pay attention to pairing and scheduling so that the members in DevOps team have equal opportunities to contact with customers.

In general, members of the backstage who have few opportunities to contact with customers, and will not face the brunt of customers even if a trouble occurs. However, understanding about the same opinion of customers changes greatly depending on whether they meet with customers at least once. When you are acquainted with customers and understand their way of thinking, you will inspire yourself like "What they are saying is not wrong. Let’s do it.” In addition, if there are more opportunities to meet with customers, you will be able to think about themselves as potential customers' needs.

I am not the only one who thinks that such transformations are more important for DevOps than any automation tools.

Define the essential KPI of what you want to gain

Let's move on to the next topic.
“Kickstart DevOps anyway without complaining!” Have you ever heard to do so from your boss or instructed a subordinate like this? In such a situation, DevOps will probably fail.

As well as knowing what DevOps is, it is important how we achieved DevOps. Let’s think where our project is now and where we are going. This is very important to set the growth of DevOps and achievement condition (KPI) beforehand.

However, the KPI with useless numbers will not work for the future business. Projects should set the essential KPIs that what will we gain from the experience, and also these KPIs should be agreed by all team members. In addition it is important that all members commit to the agreement.

A portfolio is more significant at long term and large-scale development

In agile development of large scaled enterprise systems, the project will fail by just build up sprints without planning. Thus portfolio management for developers is necessary.

In order to successfully control SoE developments in a company that SoR developments have taken root, it is necessary to think from developmental milestones and business events, which are the necessary plans and procedures for companies, from a short-term to medium- to long-term perspective. For this purpose, it is necessary to set timing (delimiting, milestone) to verify each task execution units, product release units, and business plan units.

For example, in my project, I defined the framework as follows:
- Sprint: 3 weeks (task execution period, shortest releasable period)
- Season: 3 months (standard release cycle, quality evaluation unit, capitalization unit)
- Vision: 12 months (investment plan, development plan, medium-term business strategy)

These were redefined for my development project based on Microsoft’s way of thinking. (However, it depends on your environment). In my project, I decided that these settings of the framework had the highest affinity between development milestones and business events.

Resolve operational problems in continuous delivery

Resolving the continuous release process by both developers and operators is a big issue involved with fundamentals of DevOps. In Agile development which has no continuous release, it will be chosen either release the whole system by one time or deploy continuously to the staging environment. None of them is wrong, and there are several cases for specific purposes (such as Blue-Green Deployment).

The essence of continuous delivery is to give top priory to customers’ needs. If our customers say "We are not able to win in the market unless we release services as soon as possible.", we have to release services promptly.

If it is a product that can release important functions quickly, it may not be necessary to release for each sprint. Moreover except for emergency product releases, we may be able to release the whole product for each season. If so, we may be able to reduce liability on developers and operators, even with the current handover.
“That’s not DevOps!” Such a voice may be heard. However if it is possible to achieve the goal for expanding customers’ business, no need to argue about what DevOps is.

Organization cannot be changed easily

Although the organizational silos are the core problem, we are not able to drastically change, easily. There is no guarantee you will succeed, thus you don’t want to do if need not do it. It is exactly such a problem.

At first, my team was composed of four departments such as the planning department, the infrastructure operation department, the system operation and customer support department, and the development department. Each department is a typical silo type organization with a project manager. The planning department was responsible for everything, and then they got help from the development department because of lack of skills and manpower. Subsequently, the manpower was not enough in the development department in a row, and we decided to get manpower from infrastructure operation department. We surely knew that shortage of manpower in the system operation department next time.

In this way, it was clear that there was a limit even if we were survived by short term measures. What should we do? A team for DevOps should be organized by one department at the beginning is one way. However, it is a selfish conclusion ignored the environment of the company.

The reason that many large companies are difficult at working with DevOps is that they are not composed of a single business. After a system development, the system will be in ?“off-season” for the development project. Therefore, the members in the project will migrate to another development project of a system that will become a new source of income. It would be fine if your company is able to put manpower as many as the DevOps team, but usually it is not a good idea for developers to stay on the same system permanently for operation.

In other words, the key to success of large companies is that able to establish effective teams for DevOps without changing organization and organizational functions.

Add optimize rules for SoE to your company

So far everything in the companies has been optimized for SoR. If you try to work on SoE in such a situation, the biggest issue is conformity to your company’s rules. However, it is difficult to change all the rules at once. SoE is a minority and the governance of the majority SoR remains.

In such a case, it is more reasonable to make rules of SoE, which is a minority, than to change the basic rules of the whole company that the majority is adapting. At NTT COMWARE, we have tried to discuss about the alternative idea for compliance and governance that were equivalent to SoR in SoE development and then we made DevOps rules. We are now transforming projects to SoE sequentially.

“Do you want to make “unregulated areas” in your company?” You may be misunderstood by others if you try to work as described above. Please persuade them again and again, because you cannot build deep relationships with customers if there are conflicts in the workplace.

Power to change the culture of IT companies

There were several movements focusing on improving development efficiency, improving quality and reducing cost. DevOps started as a movement in the IT industry that had the potential for changing the culture of the company from bottom up. NTT COMWARE considers the DevOps as the winning method for customers and us by adopting it to customers’ businesses. There are people who think DevOps is a method for short term cost reduction certainly by their companies. However, don’t you think the DevOps is more exciting that keep you to win with customers?