Non-permanent processes in Software Development

An obvious solution to software development problems is to create buffer capacities for non-permanent processes. Some companies figured this out a long time ago. In 3M for several decades, developers have been involved in 85% of their capacity. At Google, engineers are allowed 20% of the time (day of the week) to do whatever they want, which means they always have extra time available if a project is delayed. But such a solution is quite difficult to organize. Few will escape the temptation to keep employees 100%. Managers reflexively issue new tasks when they see downtime.

But there are other solutions as well.

Change control system

In the case of a pharmaceutical company, this may mean aligning the goals of the animal testing department with the development department. For example, you can reward the testing department for quickly responding to requests rather than utilizing resources.

Selectively increase resources

By adding additional resources to areas where the utilization rate exceeds 70%, you can greatly reduce the waiting time. If a pharmaceutical company did this in the animal testing department, then responses to requests for new chemical compounds would come faster. When testing is done with computer modeling and simulations, the increase in resources costs relatively little, as it usually means the purchase of additional hardware and software licenses.

Limit the number of active projects

If you can’t increase resources in the animal testing department, you can reduce resource utilization by reducing the number of active projects. Discipline to hard limit the product development pipeline often leads to better focus and clearer priorities.

Improve the visibility of tasks that are being worked on

One of the methods is visual control boards. They may look different, but the key point is to have some kind of physical indicators, like notes for notes, representing the work. The board should display all the work and the status of each part of the project. It should be at the center of the management process. Daily 15-minute meetings can be held at these boards to coordinate efforts and move the work forward.

image

Typical view of a development control board

The first column is new tasks, approximately similar in complexity. The second is work in progress. The number indicates the maximum number of simultaneously solved tasks. A check mark next to a task indicates a problem. The third is the parts ready for testing. A turned leaf means the manager needs to intervene. Fourth – test tasks. The number indicates the maximum number of simultaneously solved tasks in total in both columns. Fifth – tasks that have passed testing.

Misconception 2: Processing tasks in large batches improves the economics of the development process

The second reason for queues in product development is batch size. Let’s say a new product consists of 200 components. You can decide to design and build all 200 before you start testing them. But if 20 components are developed instead, the batch size will be 90% smaller. This will greatly reduce the waiting time, since on average the queue is proportional to the batch size.

Reducing batches is a basic principle of flexible manufacturing. It allows you to reduce work, speed up feedback, improve production cycles, quality and efficiency. Small batches in product development are even more impo

rtant, but few people understand this.

One reason is the nature of their workflow. Since the information being processed does not have a physical representation, neither do the batch sizes. The second is that the developers have some kind of penchant for large parties. Perhaps they mistakenly believe that this allows economies of scale.

In a well-managed process, batch size will balance transactions and storage costs. It’s like buying eggs at the grocery store – if you buy eggs a year in advance, the cost of transportation will be low, but most of the eggs will spoil, which will increase the cost of storage. If you buy eggs for the day, they will not spoil, but the cost of transportation will be high.

image

The optimal batch size is the one with the lowest total cost (transactions and storage). Small deviations from this point do not bring big problems. For example, if you deviate from the ideal lot size by 20%, the change in the cost of production will be only 3%.

Having figured this out, companies are starting to reduce batch sizes, with surprising results. Some are moving from testing large batches once every 90 days to testing small batches several times a day. One computer peripheral manufacturer reduced software testing cycle time by 95% (from 48 months to 2.5), increased efficiency by 220%, and reduced defects by 33%. The savings amounted to twice what the company expected.

Misconception 3: We have great development plans, we just need to strictly adhere to them

We have never seen a project whose requirements have remained constant throughout the development. But many organizations continue to firmly believe in their plans. Any deviations are chalked up to poor management and execution, and they track every step to reduce them. For production processes with frequent repetition of steps, this is a good approach – but it is not suitable for development innovation. New insights can come daily, and conditions are constantly changing.

The classic study of technical problem solving shows the fluid nature of a developer’s job. It shows how the engineers who created the aviation subsystem developed options and chose the best one. As they worked, their preferences changed frequently, and they tested and refined solutions. This is typical of innovative projects – testing and experimentation reveals what works and what does not, and preliminary estimates of cost and value can change.

Software development from scratch

The other day I had a rather interesting idea for an article based on the following premise: HabrĂ© has never talked about organizing enterprise development “from and to”. In terms of completely from scratch and at least to a comfortable minimum. As a starting point, I will take a situation of complete chaos, when some code exists on the machine of a single developer, there is no version control system, there are no test environments, the database object code exists only inside the reference productive database, there are no build and installation processes, no quality control code and so on. Perhaps the reader will ask the question “Does this happen in 2020? Does anyone else develop like this?” and you will be only partly right. I propose to discuss the details under the cut.

I came up with at least three possible scenarios:

your company has outsourced all development and regularly orders improvements to the vendor, and he only gives back binary distributions of the next version of the system. Both the code and the intellectual property of the software seem to belong to you, but you absolutely do not know how to work with it. And then one day you decide to take the development process into your own hands. And if the contract is in foreign currency and the exchange rate has jumped again, then you may want to leave the vendor several times stronger;

you were parachuted as a developer / leader / manager in such a project. At the interview, we discussed the most important things, agreed on the expected reward, checked your skills, but somehow kept silent about the current status of the organization of processes. And on the first working day, the mentor offers to copy the project code from the network folder. Whoops;

you have been working in such a place for a long time and ignored all modern trends, did not read information resources on development, and suddenly suddenly began to see the light, realizing the inefficiency and irrelevance of processes and tools. You want to develop with comfort, in a team with well-established processes and supporting software. This is a script a la modern javascript for dinosaurs.

Note: In my 10+ years of experience in development, I have come across each of the scenarios (in past jobs).

That is, we will now try to outline a plan for turning some enterprise development with near-zero process organization and tool support into a fairly comfortable and modern one. To a lesser extent, the article will concern startups and game development studios, since they have their own rules and features. Although there are exceptions, I will mention startups in the chapter on distributions.

Getting started – git

So, suppose you are in one of the three hypothetical situations I listed in the introduction. Where to begin? Of course with git. This is the industry standard, the most common version control system, the best development tool of the basic and necessary. Just install it on your machine, go to the project folder and type these commands in sequence:

git init

git add .

git commit -m “Initial commit”

From now on, you will now have a reckoning. From now on, the answer to the question “when and what changes were made in the code” will always be extremely unambiguous. Now you can make point commits of edits that are separate in meaning, you can roll them back and return to any point in history, you can experiment in a separate branch, and so on. Even on a single machine without any synchronization, git already opens up a lot of possibilities.

Teamwork – GitLab

What if there is more than one developer in the team? It’s okay, git was created for teamwork as well. And if 5-7 years ago I would recommend installing and configuring git on the server next, now there is GitLab. GitLab is an excellent example of an OpenSource project that has become ubiquitous and accepted throughout the industry.

Wikipedia says:

GitLab is an open source DevOps lifecycle web tool that provides a code repository management system for git with its own wiki, bug tracker, CI/CD pipeline, and more.

You can just take a GitLab distribution and deploy it inside a corporate private network – you get your own little “pocket” GitHub. GitLab has paid enterprise versions, you can check the list of available features here. From pleasant – paid features gradually flow into free versions.

So, GitLab can become the cornerstone of the development organization. It allows you to operate with the concept of “project”. The project has a separate git repository, a list of issues / defects (issues), an agile board, a wiki, etc. In other words, out of the box and for free, you get almost all the tools you need for efficient and distributed development. But first things first.

Code storage

First and foremost, GitLab allows you to efficiently and conveniently work on code changes. It fully supports the Github Flow paradigm. This is a fairly lightweight process, where each new change is implemented in a separate branch, and there are so-called merges to the main codebase to merge into the main codebase. merge requests or merge requests (colloquial mergerequest). The sequence of steps is approximately the following: a new branch is created from the task by pressing one button, and at the same time a merge request is created. Then the developer switches to this branch, implements new functionality or fixes a bug, commits changes locally, pushes to the server. All his work will be displayed as conveniently as possible on the merge request tabs: description, list of commits, line-by-line diff. At the time of completion of work on the task, the developer removes the “work in progress” (WIP) flag. For a teammate, this can be a signal to conduct a code review. Separate comments can be made directly from the diff, there you can also propose your changes, and there you can immediately accept them. This is a very convenient and efficient way to conduct a code review. You can set up a rule: you cannot merge changes until all discussions are closed. In the latest versions, you can also add the necessary reviewers without assigning a task.

Task tracking

The task is further understood as some entity with a description of the problem / new functionality / idea, within which changes can be made. GitLab allows you to work with issues in two ways: