‘Do you want tech reliability and availability with that?’: Ensuring you are managing (and minimising) the risks of critical technology outages (Part 1 of 2)

‘It is a truth universally acknowledged that cyber security is a fundamental priority of all Australian businesses’ (apologies to Jane Austen). Of course, it should be for all businesses with any connection to the internet or a telecommunications network, to adequately protect themselves from the scourge of cyber incidents which are increasing in volume and severity every year.

However, recent outages of a major global fast-food outlet in Australia (and in a number of other regions), a telco and similar recent occurrences across a range of significant Australian and global businesses over the last 3 years remind us that malicious cyber incidents should not be the only concern of businesses that use technology in their day‑to‑day business. These tech ‘outages’ (and the hundreds of non‑publicised outages) over this period were not due to a malicious cyber incident but, rather, friendly fire – an often undisclosed tech ‘glitch’ or failure. 

These outages are a reminder of the potential significant impacts that any tech outage or failure can have on BAU, no matter how caused. Arguably, with no malicious actor to blame, a non-cyber incident tech ‘outage’ is more embarrassing and damaging to one’s reputation and the trust of customers. The good news is that any work done in relation to better managing such outages (and ensuring your IT resilience) will greatly assist in your recovery from all outages, including those caused by a cyber incident.

Introduction

Just like building cyber incident preparedness/readiness, following the lead of the ‘Critical Infrastructure Risk Management Program’ for critical infrastructure, there are a number of steps, measures, processes and controls that can be implemented and provisions that can be included in your agreements with third party providers (collectively measures) to manage and substantially reduce the risks and impacts of a failure of any critical technology used by your business. ‘Critical technology’ being any element of tech used (whether internally developed, third party provided or outsourced) which is fundamental to the BAU operation of a business. In particular, the key element(s) of any technology that, if it fails, means 'closing the doors' for any amount of time or which requires significant effort and cost to keep the doors open and the business to keep operating.

Failures of critical tech do not usually materialise out of ‘nowhere’. That is, ongoing interaction, assessment/audit and dialogue between the business and the third-party provider (or, as relevant, the internal tech team) will usually raise warning signs as to this possibility/the probability of a tech issue before it becomes a failure/outage. By having the appropriate measures in place, most potential failures/outages of critical tech can be identified and addressed well before they result in the business being unable to operate for a period of time (i.e. until such is fixed or the tech replaced).

To use the analogy of a car, the engine ‘warning light’ usually appears well before the total failure of the engine, allowing those who understand what the warning light means time to address the issue (e.g. add oil) and prevent the engine from ceasing to work altogether. But, first, you need to check the ‘warning light’ is actually working and understand what it means when it appears. Business must first ensure that appropriate measures are in place and that they understand what the ‘warning signs’ mean for your critical tech. Therefore, first, below are detailed the steps you can take for existing critical tech and for new acquisitions/implementation of critical tech to determine if or ensure that (as the case may be) the appropriate measures are in place. Then, by way of example, in Part 2 (to follow) we examine what those key appropriate measures are for the acquisition/implementation of critical software/software development and implementation in a business. That is, when following the steps below, what the appropriate measures you should be seeking to implement in the case of a critical software/software development and implementation.

Your existing critical tech arrangements

For existing critical tech arrangements, the first of our recommended steps (to ensure the appropriate measures are in place) is to undertake an assessment of what critical tech you actually use. Undertake a thorough assessment of what key elements of all the current technology used in the business (including any internally developed, third party provided and outsourced tech) to determine what is critical to the BAU operation of your business. That is, realistically what impact would there be on BAU if each specific element of the technology (or in combination with any other specific tech elements) went down or was unavailable for a period.  Would this significantly impact the operations of your business/BAU? What is the ability of your business to continue to operate in this circumstance in a similar manner to pre‑outage BAU, without significant additional costs or effort?

Once this assessment has been done then, for each identified ‘critical’ tech or element of your critical technology, determine the process(es) that was/were undertaken to ‘acquire’ and implement that critical tech element (or the tech it is included in). What due diligence was done on the provider and the tech itself and what are the terms of the agreement (specifically, relevant provisions regarding continuity, availability and levels of performance of the tech)? You should also determine what ‘audit’ and testing, meetings, improvement and/or independent certification rights/obligations there are (usually in the contract) for each critical tech element (or the tech they are included in).

An important aspect of this determination is what due diligence was done ‘pre‑acquisition’ of the tech and what relevant terms are in the agreement. Also, determine what has actually happened in practice since the acquisition/implementation in terms of the ongoing review and assessment of and meetings in relation to the performance of the tech and the provider (and the success of such in addressing any concerns and fixing any identified issues as and when they have occurred).

Once the 'assessment' and 'determine' steps are complete, for each existing critical tech the business should then measure the process(es) that actually occurred in (and are included in the contract) acquiring the tech and those that are currently occurring (i.e. as found in the determine step) against a pre‑determined objective set of better practices (or the goals/desired practices the business wishes to implement for future tech acquisitions) to determine any ‘gaps’ in the current measures for that existing critical tech.

Once the gaps are identified, the business should determine how/what is needed to address those gaps (or missing measures) as best it can for all existing critical tech. Determine what needs to (and can) be done to address the identified gaps and uplift the measures for your existing critical tech. However, the best option may not always possible (although, in our experience, often it is) where changes to an agreement are required, for example, and, if not, the business should determine what other risk mitigation measures can be implemented until such time as the agreement can be changed.

The business should then implement the changes determined at the ‘address’ step necessary to uplift the existing measures in place to achieve the relevant better practices/measures and to address all identified gaps. As well as uplifting obligations and mechanisms for accountability, such as audit and regular meetings between the relevant teams of the business and the third party (or the internal team), this also requires an uplift of (and doing of) the business’ processes/measures not done in respect of the original acquisition and implementation of the critical technology. For example, if not previously done, undertake a due diligence of the provider and the tech itself to ‘flush out’ any key risks with either the provider or the tech (including any interdependencies) that will need to be managed.

Your future critical tech acquisitions

All of the steps noted above for existing critical tech arrangements are also relevant (and should be adapted as necessary and implemented) for and as part of any new critical tech acquisition/implementation process. In particular, businesses must consider how all new tech (whether critical or not) will impact both BAU and the other tech with which it will interoperate. Could an outage in this tech, while not itself necessarily catastrophic for ongoing business operations, possibly cause a ‘chain reaction’ impacting other tech (or other parts of the business) which, together, will significantly impact the ability of the business to operate BAU?

Perhaps the most important of the steps in the acquisition/implementation of new tech (especially for likely critical tech) is to undertake a comprehensive ‘due diligence’ on both the provider (e.g. the third party) and the technology in question. Just what this due diligence should cover will be detailed in Part 2.

Where the critical tech development (or tailoring of the tech) is to be done by an internal team, a similar type of comprehensive ‘due diligence’ should be done, focussed on their technical capabilities, bandwidth and available resources to deliver the relevant tech to specification within the allocated time. Where such tech will be crucial to BAU, is a critical aspect of improving the competitiveness of the business and/or where there is a tight timeframe for delivery and implementation, an independent assessment should be undertaken where possible.

Conclusions - what you can do now

In Part 2 we will address the better practices/measures you should implement as regards critical software/software development and implementation, as an example for critical tech where an outage may significantly impact BAU. For now, however, our recommendations are:

• for your existing technology, undertake the 'assessment' and 'determine' steps in readiness for the 'measure', 'address' and 'implement' steps, once you have determined the appropriate better practice measures for your existing critical tech;
   
• adapt as necessary the above steps and implement them as part of your process for the acquisition/implementation of all new tech (especially critical tech proposed to be acquired/implemented) and, if critical software/software development and implementation, prepare your benchmark measures including our comments in Part 2; and
   
• for both your existing critical tech and all future acquisition/implementation of tech (especially critical tech) undertake (retrospectively for existing tech) a comprehensive due diligence of the provider (including if the work is to be done by an internal team) and the tech itself. Further details of what this due diligence involves will be included in Part 2.

Part 2 of this examination of tech resilience and our guide to managing and minimising risks arising from outages of your critical technology will be published by early May. In the meantime, please don't hesitate to reach out should you wish to discuss any of the matters raised above or if we can assist you to implement any of the steps noted.

Clyde & Co’s Technology & Media Team has unparalleled and specialised expertise across the privacy, cyber and broader technology and media practice areas. It also houses the largest dedicated and market leading privacy and cyber incident response practice across Australia and New Zealand. Our team is also highly regarded for their expertise and experience in managing all forms of disputes across sectors and international borders, including advising on some of the most high-profile disputes and class actions commenced in Australia.

The firm's tech, cyber, privacy and media practice provides an end-to-end risk solution for clients. From advice, strategy, transactions, innovations, cyber and privacy pre-incident readiness, incident response and post-incident remediation through to regulatory investigations, dispute resolution, litigated proceedings (plaintiff and defendant), recoveries and third-party claims (including class action litigation), we assist our clients, inclusive of corporate clients, insurers, insureds and brokers, across the full spectrum of legal services within these core practice areas.

For more information, please contact Alec Christie, John Moran, Reece Corbett- Wilkins, Richard Berkahn or Stefanie Luhrs