'Cyber war': Days of Waikato DHB IT systems down as experts work to oust hackers
It struck in the early hours of Tuesday morning.
An email attachment had been opened – that’s the working theory, anyway – launching a cyber attack on Waikato DHB’s systems.
The result was basically an IT blackout across five hospitals in the region and days of consequences.
That’s what cyber war looks like, a security expert says.
* Day 3 of Waikato DHB cyber attack: what patients need to know before heading to the hospitals
* Cyber attack sends Waikato DHB cancer patients to Auckland for radiation therapy
* Waikato doctors go old school while teams face 'demanding' task of recovering IT systems following cyber attack
* 'No ransom will be paid' – Waikato hospitals reeling after cyber attack
If it was done across the board it could cripple a country, US-based cybersecurity professor Hossein Sarrafzadeh said.
As in war, details are being closely guarded.
“The only people who know are the hackers and the victim. And they both don’t reveal more information than necessary to protect themselves, to not reveal their hand so that the other side doesn’t take advantage,” he said.
As teams worked on a fix, patients were sent around the country, staff worked old-school style, and employees got nasty shocks when they saw their bank balances on payday.
Day one: IT blackout, ‘absolute chaos’, DHB rules out a ransom
The first chief executive Kevin Snee knew was from a text in the early hours of Tuesday morning.
The director responsible for the DHB’s IT area was alerting him to a cyber security issue which was being worked on overnight.
The severity was clear.
“The systems were down in the hospital, so it was apparent from the start there was a major incident happening,” Snee said.
By 7am, he and other key players were in the first of many major incident meetings through the week, as the DHB started up the response system often used in natural disasters.
Meanwhile, phones and computers were down across the region’s five hospitals.
Email was the only IT service not blocked.
It was “absolute chaos”, one health worker said, and the most stressful day she’d had.
“We have patients turning up, and we don't know who they are there to see,” she told Stuff on Tuesday.
“We can’t go and use [Microsoft] Word and save it because it doesn’t save to anywhere.”
A message to the DHB pointed to a ransomware attack, but Snee said early on that “no ransom will be paid”.
The communications needed to be verified, and cybersecurity experts and police were investigating, he said.
Despite the turmoil, 95 of 101 planned elective surgeries went ahead at Waikato.
‘I truly hope that those criminals don’t get away with this’
If you walked through ED on Tuesday night, you would have seen “whiteboards and clipboards and people writing stuff down with pens, stuff we haven’t done much of for the last while”, a clinician said.
It was like working in ED 20 years ago, but busier than back then, said the clinician, who Stuff agreed not to name.
“It was manageable, but challenging.
“I truly hope that those criminals don’t get away with this and haven’t managed to steal significant amounts of data that we haven’t managed to back up,” the clinician said.
“[Among staff] it’s a mood of frustration with the criminals that would do this, and slightly resigned – ‘We really didn’t need this right now.’ But, again, blaming the criminals.”
Reception computers weren’t turned on, so clerical staff like receptionists deserved a shout-out for their extra work checking patient details manually, the clinician said.
One charge receptionist came in at 4am and was still around at 6pm.
Clinicians were basically thinking ‘let’s do this’, and had been kept well-informed by the DHB.
Patients understand, the clinician said, and no IT is needed to talk to them, examine them, and make judgment calls.
Many things still functioned – resuscitation rooms, CT scanners and cath labs – things were just slower, they said.
The attack hit just after a crazy Monday, always the busiest day of the week, and it could be tough to manage the weekend if disruptions continued.
Lab results were coming back on pieces of paper in an internal vacuum suction tube system normally used to send samples to the lab, and the machine that normally dispenses medication wasn’t working.
Putting the attack under the microscope would be important for the future protection of Waikato and other DHBs, the clinician said.
“If it’s one person who clicked on a link, don’t blame the person. We all could have done that.
“You and I know that [people behind this type of attack] are evil but smart.”
MARK TAYLOR / STUFF
With all systems down at Waikato Hospital on Tuesday, Samantha Catterall couldn't be seen in ED and was forced to go elsewhere for treatment. This video was first published on May 18, 2021.
Day two: An 0800 number, a key cancer service down, and patient transfers
Wednesday dawned with more challenges for Waikato DHB.
Patients wanted to know if clinics and operations were still happening, comments were building on the DHB Facebook page as people wondered whether to make what were, in some cases, lengthy trips.
By 2pm, they had an 0800 number to call.
The cyber attack left Waikato’s equipment “completely down” so cancer patients due for radiation therapy were sent to Auckland, Snee said.
That could affect up to 70 patients a day, he told Stuff.
Radiation therapy is driven by computer software and involves directing X-rays at just the right spot and dosage to zap cancer.
Other DHBs picked up patients from Waikato to help out in the wake of the attack, including Wellington, Tauranga, and Whakatāne.
Day three: Pay woes, phone line reactivated
Bank balance shocks abounded as Waikato DHB employees checked their accounts after the latest pay run.
As he walked the wards on Thursday, Snee soon heard from staff that Wednesday’s pay run hadn’t gone right.
A woman who contacted Stuff had a child who didn’t get paid at all, and another who missed out on about $400, as did her partner.
The one who wasn’t paid had just moved overseas and was expecting her final pay and down to her last $5, the woman said.
The health board hadn’t anticipated the pay problem, Snee said, but it made extra payments once alerted and was looking into Prezzy Cards for urgent needs.
“[Staff] are all working in difficult circumstances and we don’t want to undermine that by not paying people.”
One more step towards normality came on Thursday evening, when the main DHB phone line (07 839 8899) was reactivated.
“This is running at a limited capacity, so please be patient while we deal with the high volume of calls,” a DHB statement said.
The health board also reissued a warning about keeping its hospitals’ emergency departments for life-threatening conditions.
‘This is what cyber war looks like’
Waikato DHB’s systems have been crippled, cybersecurity professor Hossein Sarrafzadeh said.
If that was done across the board, it could cripple a country
“This is what cyber war looks like,” said Sarrafzadeh, who heads a cybersecurity centre of excellence at North Carolina Agricultural and Technical State University.
He sees it as likely that hackers copied Waikato DHB data, and also that they will release it if the DHB doesn't pay.
It would be released slowly, not all at once, to put more pressure on the DHB, he said.
“Anyone could have been hacked, so don’t blame the Waikato DHB,” he said. “They have been attacked by an enemy. Everyone needs to support them. That's my position.”
It appears ransomware attacks are skyrocketing, as the amount paid to hackers in 2020 was triple what it was the year before, Sarrafzadeh said.
“This year, we’ve seen it go crazy.”
Outpatient appointments and patients coming for a non-urgent surgery have been affected.
New Zealand could definitely do more in the cybersecurity zone, said Sarrafzadeh, who previously worked at Unitec Institute of Technology, and still has family here.
Concerned companies – especially in the health sector – should check their backup systems, invest in staff training, and look into anti-ransomware protection.
If Sarrafzadeh could, he’d make it compulsory for all health organisations to have a dedicated team, and to invest more in the area.
Training is essential, he said, because even a phone call could help hackers get in.
A few years back, hackers got into a complex US bank system after a caller pretending to be a colleague from another branch got some vital information, he said.
Sarrafzadeh is trying to help raise awareness and will co-chair the International Conference on Privacy, Security and Trust, scheduled for Auckland in December.
Provided borders are open, it will bring international experts for two days of academic conference and one day of industry symposium, he said.
As for how long before Waikato DHB gets running again, he wouldn’t be surprised if it took another week, but it’s hard to say.
“The only people who know are the hackers and the victim. And they both don’t reveal more information than necessary to protect themselves, to not reveal their hand so that the other side doesn't take advantage,” he said.
What the health board boss says
In light of all this, what are Snee’s feelings towards those who created this turmoil with the cyber attack?
“I’m not giving that any thought, really.”
All his energy and focus is on patients, and managing the situation as best the health board can, he said.
“Let other people worry about what’s in the minds of people who perpetrate this kind of crime.”
Nor would he comment on whether the perpetrators had contacted Waikato DHB, because it’s part of an investigation.
It’s been a strange week for Snee.
“I’ve been doing a lot of things I wouldn’t normally be doing. A lot of interviews for television.”
Phone interviews with Stuff are often limited to 10-minute windows, in Thursday’s case, just before a 4pm briefing on the cyber situation.
He attends three or four a day now, he said, but there were more in the early stages.
Health hacks around the globe
Ransomware attacks on health providers have recently made headlines in several spots around the world.
Ireland’s health system shut down its entire IT system in response to a ransomware attack on May 14, AP News reported.
Authorities said it could take weeks to get the public health system back to normal after what Irish Prime Minister Micheal Martin called a “heinous” attack.
Thousands of appointments, clinics and surgeries had been cancelled or delayed, AP reported.
In the United States, a May 1 attack on San Diego-based Scripps Health left patients asking if the hackers got hold of private medical or financial information, The San Diego Union-Tribune reported.
Scripps, the region’s second-largest healthcare system, refused to answer, saying the investigation hadn’t finished.
Closer to home, a major Melbourne health network had weeks of recovery from a ransomware attack on March 16.
Almost a fortnight later, staff at Eastern Health – which operates four hospitals – couldn’t access patient histories or access internal emails and IT systems, The Age reported.
On April 27, Eastern Health said in a statement that patient services were no longer affected by IT issues.
Past glitches at Waikato DHB
And Waikato DHB has come through past health hacks and IT glitches.
An apparently Algerian-based group, DZ Informatics Mafia, got into the website in May 2013, leaving a black screen saying “hacker inside” and, later, “Forbidden: Access denied”.
That prompted a full web security review.
Hundreds of online job applications disappeared in December 2011, but what was first thought to be the work of a hacker proved to be due to a flood of spamwhich overwhelmed the website’s server.
In December 2009, a memory stick put into a computer in a carpark booth at Waikato Hospital let the Conficker virus into the system, prompting a 48-hour network shutdown.
Audit New Zealand looked into the attack and Waikato DHB planned to spend an extra $1.276 million upgrading its system.
Data vanished in a “sentinel event’’ in October 2008, whisking away emails and personal work files of 690 Waikato DHB staff, many of which US experts couldn’t get back.
And an October 2004 attack left patient management systems and payroll down for 34 hours.