Telstra has acknowledged that a nationwide network outage, which significantly disrupted services for millions of Australians and prevented some from accessing emergency Triple Zero calls, could have been prevented. The telecommunications giant stated that a critical software update had not been applied to a device and that crucial documentation regarding a design change was missing, leading to the widespread failure.
Root Cause of the Outage Identified
In a submission to a parliamentary inquiry investigating the incident, Telstra detailed the sequence of events that led to the extensive disruption. The core issue stemmed from a GPS card within a device that was undergoing scheduled maintenance. When power was restored to the network after the maintenance, the GPS card did not function as anticipated. This malfunction had a cascading effect, impacting various services across the country.
Telstra explained that the GPS card’s unexpected behavior was a consequence of an ‘intentional design change’ previously implemented to rectify an earlier fault. However, this modification was not adequately documented. Consequently, the maintenance team was unaware of the potential ramifications when restarting the device, leading to the unforeseen outage.
Missed Software Update and Documentation Failures
The company admitted that a software update, which possessed the capability to avert the outage, had not been applied to the affected device. This dual failure – the absence of a necessary software patch and the lack of proper documentation for a design alteration – created a vulnerability that ultimately led to the network failure.
Telstra expressed deep regret for the impact of the outage, stating, ‘We have let our customers and Australians down, and for that we are deeply sorry.’ The company acknowledged the severity of the situation, with its submission noting, ‘That is clearly unacceptable. If maintenance work can trigger this kind of outage, it suggests our controls were not good enough.’
Accountability and Future Controls
Telstra has taken accountability for the failures, pledging a thorough investigation into why the design change was not documented and why the software update was not completed. The company committed to examining its control mechanisms to ensure that known risks are identified, prioritized, and addressed before they can affect customers.
‘We are accountable for that, and our investigation will address why that design change was not documented, why the software update was not completed, and what needs to change in our controls so known risks are captured, prioritised and closed before they can affect customers,’ the submission stated.
Broader Impact of the Network Failure
The outage had far-reaching consequences, affecting millions of Australians. Mobile phone users were unable to make calls, including critical calls to emergency services like Triple Zero. Train services in Victoria and New South Wales experienced severe disruptions, leading to cancellations and significant inconvenience for commuters.
Furthermore, an EFTPOS payment system used by approximately 80,000 businesses was impacted, leaving many unable to process transactions. This had a direct effect on small businesses and their customers.
Parliamentary Scrutiny and Customer Compensation
Telstra’s Chief Executive, Vicki Brady, was scheduled to face questioning from a Senate inquiry regarding the outage. Ahead of the hearing, Greens Senator Sarah Hanson-Young criticized the company, suggesting that profits had been prioritized over customer safety and service reliability. She called for stronger regulatory measures to protect consumers.
‘Millions and millions of Australians have been impacted by Telstra’s bad performance, from Telstra’s incompetence, and we’ll be asking questions of the CEO and the executives about that today,’ Senator Hanson-Young commented.
In response to the widespread impact, Telstra has opened a process for affected customers and small businesses to submit compensation claims, provided they can offer supporting evidence of their losses.
Welfare Checks and Apologies
The company also conducted over 630 welfare checks on customers who were unable to connect to Triple Zero due to the outage. Chief Executive Vicki Brady issued a personal apology, stating, ‘Last week we let you down, and I am sorry for that.’ She reiterated the company’s commitment to accountability, transparency, and swift issue resolution.
‘When something goes wrong, we’re committed to taking accountability, giving you clear information, and fixing issues as quickly as possible,’ Brady said. ‘We understand what caused the issue and will complete a full investigation and take what we learn to make our network stronger. You have my commitment on that.’
Representatives from the Australian Communications and Media Authority and the Department of Communications were also expected to provide evidence at the inquiry, highlighting the multi-agency attention on the incident.
Conclusion: Lessons Learned and Network Resilience
Telstra’s admission of an avoidable error underscores the critical importance of rigorous maintenance protocols, comprehensive documentation, and timely software updates in maintaining network stability. The incident serves as a stark reminder of the potential consequences when these fundamental operational aspects are overlooked. The company’s commitment to a full investigation and strengthening its controls indicates a move towards enhanced network resilience and improved customer service in the future. The ongoing parliamentary inquiry aims to ensure accountability and to implement measures that prevent similar disruptions from occurring again.
