Twelve Steps to Recover Rusted Out Information Technology Departments

Canada, we have a problem

Twelve Steps to Recover Rusted Out Information Technology Departments

Jordan Press, of The Canadian Press, published an article titled “Federal IT systems at risk of ‘critical failure,’ Trudeau warned in memo.” In this article, he reveals the situation is so dire that even the Prime Minister of Canada, Justin Trudeau, has been briefed.

This article really struck a nerve with me, and I had several reactions upon reading it. They started with anger and disappointment. Then morphed into acceptance with phrases like Duh, No shit Sherlock, and Thank you Captain Obvious. The final reaction was some thoughts that resulted in this post.

Is It Actually A Problem?


Mission-critical IT systems that are “rusting out and at risk of failure” present a bevy of risks and problems.

At best, it may not be possible to add new services without first spending large sums of money and significant time to update existing systems. For governments, this may mean increasing taxes or just not offering additional services. For enterprises, this may manifest itself in missed opportunities and losing customers to the competition.

At worst, system failures could result in loss of access to critical services. Imagine what would happen if major financial systems failed for an extended time. What about healthcare system failures? Chaos will inevitably ensue if technology failures cause extended outages in the delivery of water, power, and gas.

Outdated and unpatched systems are also rife with security vulnerabilities. Fraud, identity theft, ransomware attacks, and denial of service attacks are just a few events that could turn a person’s life upside-down, bring down a government, or bankrupt an organization.

How Common Is This Problem?

I have been an information technology professional long enough, and have talked to enough IT professionals to know that the problem of IT being in trouble is widespread. It is pervasive in all levels of government (i.e., federal, provincial, and municipal). It is present in enterprises across all verticals (i.e., manufacturing, healthcare, utility, financial, pharmaceutical, retail, etc.). It shows up in Internet service providers of all sizes. And it spans the globe. It seems that the ideal IT shop is as elusive as Bigfoot.

Why should the Canadian federal IT system be any better than that of any other organization? It should surprise no one that they have problems too. Although I cannot imagine what kind of system would still be operational after sixty years. I really hope that was a typo. To add some perspective, the Apollo 11 moon landing occurred just over fifty years ago.

Who Is To Blame?

“Off with their heads!” - Queen of Hearts

The politically correct answer would be that there is no room for blame; instead, we should look for solutions. However, it is essential to understand a problem before effective solutions can be designed.

Taking a step back and looking at the situation as a whole reveals that the problems are industry agnostic and have been occurring for decades. Therefore, it is clear that no single entity is at fault. No one government, corporation, vendor, or individual is responsible for the current situation. We are then facing systemic problems. Here are some key factors that accelerate the rusting process.

Improper funding: Unless it can be proven that a system can help generate revenue, or drive down costs for an organization, it is common to underfund those systems.

Out of sight, out of mind: Since no one knows IT exists until something is broken, spending does not translate directly into votes. MRIs, roads, and schools garner more votes than patching critical infrastructure. The same holds true for a CEO who wants to leave a legacy behind. Erecting a new manufacturing plant does that way more effectively than approving the spending to update a payroll system.

Time estimates: Inflexible timelines are constructed with the assumption that everything will go smoothly. When problems occur, features are removed, workarounds are installed, and corners are cut to stay on schedule. Technical debt, anyone?

Choices: Technical decisions are often made by the wrong people. Many decisions default to the highest-paid person in the room. Others are made by the person who has been around the longest. Frequently there is someone more qualified to pilot each ship in the armada.

Toxic environments: It is hard to manage under-performers and toxic people. They often consume a disproportionate amount of a manager’s time, leaving less time for them to coach other staff members. Less still to help optimize the environment. Not to mention the toll this takes on everyone.

Procurement policies: Everyone likes to save money by getting a good deal. However, in many cases, you get what you pay for. Insisting that all IT spending goes to the lowest bidder is a race to the bottom. This is as true for hardware and software as it is for people.

Rinse and repeat: Some lessons are never learned, and mistakes are repeated in subsequent projects, all the while telling ourselves it will be better this time. Lessons learned is the most common oxymoron in IT. The lessons are neither taught nor learned.

The Recovery Program

“We cannot solve our problems with the same level of thinking that created them.” - Albert Einstein

Moving on from blaming and shaming, let’s get on the road to recovery. The following steps are geared to the three levels of people who have direct influence within an organization. I could include my thoughts on some vendor practises, but I’ll save that for another time.

Government and Corporate Leaders

  1. Your Information Technology departments have to be seen as a business and service enabler rather than a cost centre and then resourced appropriately.
  2. It is not a question of if you need IT, it is a question of how much IT do you need. Leverage Information Technology to solve more of your business problems by making sure they have a seat at the table.
  3. Building a new hospital or a new manufacturing plant is a great photo opportunity. Holding shovel-in-the-ground and ribbon-cutting ceremonies for a new application can provide those same opportunities.
  4. Anything you can do to emphasize the importance of IT will help improve its service offerings.

IT Managers

  1. Communication is key. Celebrate your employees’ successes and report these successes to your managers. Relay business goals and expectations to your employees.
  2. Change is hard for some people, so expect some of your staff to put your coaching skills to the test. It is crucial to vaccinate the environment against the fear, uncertainty, and doubt (FUD) that those resistant to change will spread either consciously or subconsciously.
  3. Change may also be hard for you. The longer you are a manager, the more your technical skills and acumen starts to wane. Delegate technical decision making to the people that are best qualified to make those decisions.
  4. Protect your team’s time.

IT Staff

  1. Knowledge is everywhere. Find qualified mentors who can help you attain your goals. Be a mentor to those with a thirst for more knowledge.
  2. Communicate ways to save time, save money, or improve customer experience to your peers, to your managers, and to other decision-makers.
  3. Embrace the change that is coming and give it a chance. Improving your systems should prove to be a fun and rewarding ride.
  4. Stay current. Some professional development will have to occur on personal time. This is common in professions ranging from athletes to musicians and everything in between. No one cares about your professional growth as much as you do. Take ownership of it.

Bonus step that applies to everyone: Show up to work and do your best every day. Your best will fluctuate from day to day, but the level of engagement and effort should be consistently high.

Final Thoughts

This twelve-step program (there’s thirteen, but who’s counting) is a simple plan to create a healthy IT department. Exercising regularly and eating healthy is also a simple plan for achieving a healthy body, but it is not always easy to follow. I will be the first to admit that these twelve steps will also not be easy to follow.

At the end of the day, we can only control ourselves. We manage our levels of engagement, effort and caring. Maintaining all three at a high standard and combining them with sound decision making, thoughtfulness and heart is a great way to keep yourself between the ditches. Show leadership by staying consistent, and you’ll probably pick up a few hitchhikers on the road to recovery.