Government-Controlled AI Could Enable Surveillance, Impact Free Expression: Report

Greater government control over artificial intelligence in Canada could allow increased surveillance of users and discourage people from speaking freely in private interactions with AI systems, according to a Canadian rights advocacy group.

Increased state involvement in AI systems could also lead to routine government access to private conversations and data, according to a May 5 report by the Justice Centre for Constitutional Freedoms (JCCF).

“Canadians need spaces to think, explore, and communicate freely, secure in the knowledge that their Charter rights remain intact,” report author Nigel Hannaford said in a release. “Public safety must be pursued without sacrificing the fundamental freedoms that underpin a free and democratic society.”

The report warns that stronger government control over AI, including expanded lawful access powers and metadata retention requirements under Bill C-22, also known as the Lawful Access Act, could weaken privacy and free expression by making it easier for authorities to access user data. It recommends rejecting proposals to nationalize AI systems, maintaining strong judicial oversight for access to private information, and limiting AI regulation to only specific public safety measures.

The report comes amid increased debate over artificial intelligence guardrails spurred by the Feb. 10 school shooting in Tumbler Ridge, B.C., that left nine people dead and 27 injured.

Family members of the shooting victims are suing OpenAI, owner of ChatGPT, arguing that OpenAI had “specific knowledge of the shooter using ChatGPT to plan a mass casualty event.” The lawsuit says the shooter relied on ChatGPT for mental health support and counselling and treated the chatbot as a “mental health counsellor, advisor and/or pseudo-therapist” as well as a “trusted confidante, collaborator, ally, and friend.”

Federal officials have since considered tighter regulation of AI companies, possible changes to escalation policies, and even restricting or banning ChatGPT in Canada. B.C. Premier David Eby has also called for federal legislation requiring clearer reporting obligations for AI companies, arguing that public safety should take precedence over corporate interests.

In its report, JCCF argues that past federal interventions in digital media, including the Online Streaming Act and Online News Act, show how regulatory control can influence what content Canadians see online and could extend to AI systems.

It warns that integrating AI into a heavily regulated or state-directed framework could blur the line between private use and government oversight, with AI interactions subject to monitoring or compelled disclosure to law enforcement, potentially outside traditional warrant processes.

“At the same time, this report does not suggest that AI companies have clean hands,” the report states. “Today’s AI systems can infer sensitive information about users beyond what users themselves disclose, generating highly accurate profiles and insights that users neither anticipate nor fully control. This raises legitimate privacy concerns that may justify carefully tailored government regulation.”