What the University of Toronto’s AI Worm Reveals About Future Cybersecurity Risks

News Analysis

Cybersecurity experts have long warned that artificial intelligence could make malware more dangerous, and a recent proof-of-concept developed by University of Toronto researchers offers a glimpse of what that danger might look like.

The researchers built and tested an AI-powered computer worm in a controlled environment to better understand how future cybercriminals could exploit advances in artificial intelligence and how their targets might prepare for those risks.

The experiment showed how an AI-powered worm could move across a network, exploit known vulnerabilities, coordinate activity among multiple copies of itself, and adapt its behaviour as conditions changed.

More important than any single capability displayed by the worm is what the research suggests about the future of cybersecurity. The experiment raises questions about how effective today’s cybersecurity tools would be against malware that can adapt and make decisions without direct human guidance.

Computer worms have been around for decades. In 1988, the Morris Worm infected up to 10 percent of all internet computers. These malicious codes copy themselves from one device to another across a network without any guidance from the creator.

Unlike viruses, which must attach themselves to an existing file or program to spread, a worm travels independently, exploiting unpatched software or misconfigured security settings as it moves.

Traditional worms operate on a fixed script, targeting one specific vulnerability and exploiting it the same way on every machine. Encountering a system that falls outside its programming stalls it. That predictability is also their limitation, and that’s what security patches and antivirus tools are designed to counter.

On June 2, a team of researchers at the University of Toronto announced they had developed a new class of malware worm. Their intent was to be better “positioned to develop the countermeasures needed to detect and defend against” this new type.

“Every online device is a potential target,” they cautioned. “And current cyber defences are not yet ready for it.”

The researchers developed the program in a “secure, closed system, taking extensive precautions” and then released a proof-of-concept—a pre-print, not actual code that can be abused by hackers.

The researchers argue that sharing their findings will help defenders prepare, but the work also raises a question familiar to cybersecurity experts: when a new capability is demonstrated, who else may be watching?

The Cost of Attacks

The most prohibitive factor for cybercriminals using AI has been the cost of the computing infrastructure needed to power large language models. This barrier has forced criminals to rely on large organizations for support.

But the research demonstrates a potential solution to the criminals’ problem: a worm that could draw on the victim’s own hardware. If such techniques become practical in real-world attacks, they could represent a boon for cybercriminals.

And that’s just one of the capabilities highlighted by the experiment.

The worm “does not require the capability to discover novel zero-days, only an AI model capable enough to operationalize known vulnerabilities,” the researchers wrote.

In other words, it uses publicly disclosed but unpatched flaws, weak passwords, and misconfigurations—which are common—to penetrate systems.

Companies and security researchers routinely disclose software vulnerabilities as a security precaution. A worm that could read those disclosures could use the information much faster than IT teams could patch the flaws.

Swarming Capabilities

When multiple copies of this worm are active simultaneously, they form a coordinated “swarm,” sharing information and dividing tasks in real time. One copy claims a device through a shared coordination system so that other copies could bypass it. When one password is broken, it immediately becomes available to every other copy in the swarm.

Computing loads are also distributed. Copies lacking processing power could shift work to more powerful machines elsewhere in the network to avoid stalling. If one machine goes offline, the workload shifts automatically.

Shutting down an infected machine, or even a group of them, would not stop the worm. Remaining copies could continue operating and redistributing their computing load among themselves. Machines that have already been cleaned could become targets again if other copies remained active on the network.

For a defender, removing the worm from one device, or 10, accomplishes nothing if copies remain anywhere else on the network.

Evolutionary Code

Most coverage of the experiment identifies the “adaptive” nature of the worm in the experiment, but few convey the degree of its autonomy.

The preprint details several instances in which the agent engaged in novel strategies it wasn’t designed to perform.

Its adaptability also extended to its own code.

“Because the worm can modify its own source code after compromising a host, logs emitted by the agent cannot be treated as authoritative evidence of propagation,” the researchers wrote.

In other words, defenders couldn’t necessarily trust the worm’s own logs. This shift in malware from pre-programmed tools to agents that rewrite their own code could pose new challenges for cybersecurity professionals.

In fact, the researchers note that “any protection mechanism within the agent’s domain of control — a software kill-switch, a firewall installed within the virtual machine, a monitoring daemon — can be disabled by the agent.”

That includes its own kill-switch.

A safety check was built into the agent’s code to shut down if it perceived itself to be in the wrong place, such as outside the lab. But the researchers observed the parent copy deleting that check from new replicas when it wasn’t receiving the desired response, then successfully launching the replica.

The researchers, however, note that this kill switch was not the main containment mechanism. Actual containment operated outside the worm’s reach.

Whether AI-powered worms ever become a widespread threat remains to be seen.

But the research raises a pressing question: How quickly could similar capabilities be adopted by malicious actors now that they have been demonstrated publicly?