Why a network upgrade pushed millions in Canada offline

Rogers building

In early July, a massive network outage at Rogers, a Canadian telecoms giant, forced more than 10 million customers – over a quarter of the country’s population – off their internet or wireless services. The firm is now under intense pressure from regulators to explain what happened.

The consequences of the 19-hour nationwide outage ranged from the potentially dangerous, to the frustrating, to the ironic.

Police services reported that 911 services were inaccessible on many mobile phones. Hospitals reported communications problems, and one Ontario hospital had to redirect cancer patients when emergency radiation treatments were affected by the outage.

Banking services were disrupted, and many businesses could not accept debit payments or could only take cash.

Fans of pop superstar The Weeknd – who hails from Toronto – were turned away after he was forced to cancel a hometown tour stop at the Rogers Centre (yes, the same Rogers) stadium.

The Canadian Radio-television and Telecommunications Commission (CRTC) – the regulatory body that oversees Rogers and other Canadian telecoms – couldn’t receive calls.

The company, one of Canada’s main telecoms firms, is now facing intense scrutiny from the federal government and the CRTC, which has ordered Rogers to explain in detail what caused the “unacceptable” shutdown.

Rogers CEO Tony Staffieri blamed the system failure on a maintenance update and has apologised to customers, offering a five days service credit as compensation.

But questions remain about how a seemingly routine process left millions without crucial access to online services – and on Friday, a Parliamentary committee said it would be studying the outage in meetings this month.

Long before the outage, expressing frustration with Canada’s telecoms industry has been described as a national pastime, uniting the country much like ice hockey and the ubiquitous Tim Hortons coffee chain.

“Rogers, Bell Canada and Telus are known as the companies that you love to hate,” said Richard Leblanc, a professor of law, governance and ethics at York University in Toronto.

The firms control 90% of the country’s telecoms market.

“We’re essentially beholden to three companies,” said Mr Leblanc. “They’ve got so much authority and control.”

Experts say it’s a symptom of Canada’s strict foreign ownership rules for the industry.

The government has, for years, supported industries like the airlines and the telecoms out of fear that foreign countries might take over, said Ben Klass, a PhD candidate at Carleton University’s School of Journalism and Communication.

“As the neighbour of the United States, Canada has got a bit of a complex. We want to ensure we don’t just become a branch plant of the US.”

The oligopoly isn’t cheap – Canadians also pay some of the highest mobile phone and roaming rates in the world, according to multiple studies.

Industry analysts say Canada’s rather non-competitive telecoms market has allowed its Big Three – Rogers, Bell and Telus – to face few consequences for sometimes shoddy service.

In a statement, Mr Staffieri vowed Rogers “will make every change and investment needed to help ensure that [a similar outage] will not happen again” and would work with the industry to ensure certain essential services would not be interrupted by technical issues.

Bell’s top executive, Mirko Bibic, said last week the company takes its role in Canadians’ communications networks “very seriously” and was investing billions in upgrades for a “robust and resilient” infrastructure.

People using internet at a Starbucks
Image caption,Toronto residents flocked to a local Starbucks to use the internet amid the shutdown

And telecoms firms have argued that Canada’s large land mass and sparse population make it expensive for telecoms to provide service.

But Rogers in particular has had a rough time in recent months.

July’s massive outage was the second in two years.

In April 2021, the company’s wireless and cable networks crashed in a similar fashion. At the time, Rogers blamed the failure on a software update by one of its equipment suppliers.

Months later, the company was hit with a public relations crisis as the family-owned business suffered a showdown between siblings, reminiscent of HBO television show Succession.

Edward Rogers – the namesake and only son of the late family patriarch – sought to unilaterally remove then-chief executive Joe Natale, blaming him for the firm’s underperformance.

But Mr Rogers was stymied by his mother, Loretta, and two sisters, who closed ranks around Mr Natale, setting the stage for a very public family feud.

The dispute eventually made its way to court, where a judge sided with Mr Rogers. He went on to hand-pick Mr Staffieri, who is now the current CEO, saying he was among “the most highly regarded and seasoned” executives in the industry.

For Rogers, the timing of this outage blunder was particularly bad, as the company is working to close a proposed C$20bn ($15bn; £13bn) takeover of competitor Shaw Communications.

Mr Staffieri said this week that Rogers remains committed to the merger.

But Rogers still needs the approval of the Competition Bureau Canada – the national antitrust authority – and the federal government.

And the massive outage raised doubts that the relevant authorities will opt to put more power in the hands of a single company.

“It certainly won’t help their cause,” said Mr Klass.

“Just the fact that this outage occurred on such a wide scale and it would have been worse had Rogers already taken Shaw over”, with an even larger customer base knocked offline.

As Canadians saw, the country’s outsized reliance on just three telecoms firms make it susceptible to sweeping network crashes.

But experts say this reliance on a few firms also makes Canada vulnerable to cyber-attacks.

“We’ve put all our eggs in one basket, and that’s now a target for threat actors – Russia, China, Iran, North Korea,” said York University’s Mr Leblanc.

“You don’t have to attack financial services, or healthcare, or emergency services, you just have to attack the telco because all these industries are contingent on these two or three players,” he said.

If a system failure could shut down the system for hours, hostile actors could do much more damage.

Rogers has until Friday to comply with the CRTC’s demand for a detailed account of last week’s failure.

Many industry experts have gone one step further, calling for a public probe into how this most recent outage occurred.

“This cannot happen behind closed doors,” said Mr Leblanc. “It needs to be a public inquiry, where the telcos are hauled in front of elected politicians. There needs to be accountability.”

For now Mr Leblanc, a long-time Rogers customer, has done the only thing he can do: call the competition.

“Someone is coming over on Saturday to install my internet,” he said. “I went with Bell.”