Speed Versus Structure In Information Technology

Since the explosion of Information Technology, it has become a necessity for companies of all types and sizes. From a small start-up to a mega-corporation, practically everybody wants to move fast and develop solutions with minimal red tape. There is a constant struggle to balance speed versus structure in most IT departments. I have worked on both ends of the spectrum. On one hand, I've entered companies who believed that they had a traditional, Waterfall methodology when what they actually had was chaos. On the other hand, I have done consulting with the most strict Agile you can find where the client, if dissatisfied with the work completed during the most recent 2-week sprint, would not pay for those two weeks. What I have discovered over the last 20+ years, though, is that there is no silver bullet. There is no single approach that will work for every situation. However, I can guarantee that there is a specific level of structure in every situation that will optimize the team and what they produce. You just need to analyze the current status and introduce structure as-appropriate while not being afraid to change and iterate until you find that sweet spot. I will provide a few of my own scenarios below in the hopes that they can help you figure out what is best for your scenario.

Scenario 1:  Hyper-strict Agile

The hyper-strict Agile scenario I noted above actually worked incredibly well. The teams I oversaw pumped out an impressive amount of very high quality code in those 2-week sprints, and our clients were very satisfied. There were risks, though.

1. It took a very special person with a somewhat unique personality and work ethic to excel under those conditions.
2. Colleagues who did not fit dropped out very quickly, but those on the borderline who maybe could have excelled if managed better had a high level of stress-induced burn-out.

I found that there were 2 keys to success in this scenario.

1. Treat all colleagues fairly, honestly, and openly while ensuring that they play a pivotal role in selecting the work to be completed each sprint. Certainly, part of the leader's job is to set high expectations, but having a team fully committed to the work before them instead of just trying to meet expectations allows for great things.
2. Lead from the front, not from the back. If you have a team you expect to work hard and put in long hours, then you must be willing to do the same. In my situation, I intentionally did not self-assign tasks in each sprint. However, I made it my mission to provide an extreme level of assistance to any colleagues falling behind and, in the second week of each sprint, to pick up every dropped task and complete them myself or in collaboration with the assigned developer. This had several positive ramifications. The first was engendering a real team spirit when they saw that the person leading them was also in the trenches with them. Equally as important, though, it gave me the opportunity to truly understand each colleague on the team, their capabilities and interests, where they excelled, and where they had opportunity for improvement. This allowed us to tweak assignments each sprint to move even faster while still maintaining quality and keeping the team happy and learning new things.

Scenario 2:  Waterfall

In both start-up companies with which I have been involved, we generally followed a Waterfall methodology with what I would call a touch of Agile. Primarily, the Agile portion was that we had one meeting a day to discuss progress, roadblocks, and plans. These scenarios were a bit unique, though, and the keys to success might not be achievable in other situations.

1. We were a small group of people all extremely committed to a single goal.
2. All team members were highly self-motivated and entrepreneurial. It was ingrained into their personalities that they were going to work hard and produce the best output they could. Plus, it didn't hurt that every task they performed essentially fed directly into their own personal success or failure due to their tight alignment with and ownership stake in the company.
3. Any addition to the team would quickly fit in per the previous 2 bullets or be replaced as we didn't worry about things like HR policies.

This has been my personal experience with highly-successful Waterfall, but I'm sure there are other situations where it can work. The key is that a lot of companies seem to refer to anything that isn't Agile as Waterfall. Per the next scenario, what they actually often have is chaos.

Scenario 3:  Chaos

Just the word chaos generally has negative connotations, and I would certainly argue that chaos is a strong negative when it comes to Information Technology. I walked into a new job a few years back and instantly recognized the telltale signs of chaos. Prior managers who had since left used to micro-manage and dole out tasks to individual team members while dissuading collaboration. Daily processes were constantly running long and/or failing. Only through the extreme efforts of key individuals on the team was everything just barely holding together. I interviewed every member of my new, large team and realized that they all had a decent idea of their own roles and the work they were doing, but had almost no idea what was going on with the rest of the team. They also all confirmed that there was clearly no process for work intake, prioritization, testing, releasing, and teamwork. In this scenario, we very quickly started to introduce some order into the chaos through structure. Certainly their were doubters both within and external to the team, but we took the time-tested approach of clearly stating our goals and developing a methodology to measure and report on our progress towards them. It is difficult for doubters in such a situation to argue with hard metrics. It did not happen overnight, be we were able to, without expanding the team, increase throughput 3-fold, improve our rate of achieving SLAs by 500%, and shrink our weekly pending tasks list (essentially bugs and non-project items) by 95% into the single digits. It took 5 months to achieve these numbers, but the initial benefits showed up very quickly. A tremendous side-effect was the overall happiness and satisfaction of the team. People even started doing their week-long on-call rotations without a single incident so they could sleep through the night which was better for everybody. All of that was made possible by finding the right amount of structure for the situation. We continued tweaking and adjusting that structure for a year through both our team organization, our procedures, and our tools (e.g. Jira) until we felt like it was just about as good as it could get.

I hope that maybe some of the information I have shared above can help you improve your own scenarios. Continuous improvement and a desire to implement change is neither an admission of failure nor an assignment of blame. It is simply a way to always be moving forward and making progress.