Australia’s government is acknowledging the growing risks associated with artificial intelligence (AI), establishing a dedicated institute to address these concerns. Assistant Minister for Science and Technology, Andrew Charlton, recently highlighted the unpredictable nature of advanced AI models, stating they are “already doing things their creators never intended: cheating, deceiving, going their own way.” In response, the Australian government has launched the AI Safety Institute, a key component of its National AI Plan, intended to serve as a “national testing capability.” However, questions remain about whether this initiative possesses the scope and resources to adequately safeguard Australians against the rapid advancements in this transformative technology.
Current and Emerging AI Harms
The risks posed by AI are not merely theoretical; they are already impacting Australians. Present-day concerns include the proliferation of nudify applications, increasingly sophisticated scams, and the misuse of deepfake technology and voice cloning. Chatbots have also been linked to social isolation among teenagers and the encouragement of harmful behaviors. Furthermore, cybersecurity risks are escalating, prompting significant concern among national security agencies.
Beyond these immediate threats, experts warn of future dangers stemming from rapidly expanding AI capabilities and the development of agentic AI systems for which effective controls are yet to be devised. A recent report by the United Nations’ Independent International Scientific Panel on AI echoed these concerns, drawing attention to the global concentration of power and resources in AI, the potential for increased inequality, and the technology’s profound influence on human cognition, reasoning, and work practices.
The Mandate of the AI Safety Institute
As outlined by Andrew Charlton, the AI Safety Institute has been tasked with three primary objectives:
- Analyzing and Testing AI Models: Evaluating the performance and safety of new AI systems.
- Supporting Regulators: Providing guidance to government bodies to help them navigate the evolving landscape of AI capabilities, risks, and harms.
- Shaping Safe AI Development: Influencing the responsible development, deployment, and international governance of AI in alignment with Australia’s interests.
The institute is already engaged in research concerning multi-agent risks, collaborating with the Gradient Institute. AI agents, capable of performing tasks like booking reservations or planning travel with minimal human oversight, present unique challenges. When these agents interact autonomously, unpredictable and potentially hazardous outcomes can emerge. The institute is also working with the CSIRO on AI alignment – the critical field dedicated to ensuring AI systems operate in accordance with human values and objectives. This focus is crucial given the current inability to fully predict AI behavior, evidenced by instances where AI has deceived users or circumvented safety protocols during testing.
Kate Conroy, the head of the AI Safety Institute, who also holds a position as a philosopher and Royal Australian Air Force reservist, emphasized the institute’s commitment to addressing both immediate harms affecting vulnerable populations and the more profound, long-term risks associated with advanced AI.
Resource Allocation and Future Directions
While the establishment of the AI Safety Institute is a significant step, its allocated budget of A$29.4 million over four years raises questions about its adequacy, especially considering the rapid pace of AI development and the breadth of emerging issues. Comparative international investments highlight potential shortfalls:
- United Kingdom: Approximately A$460 million allocated in its 2025 spending review for its AI Security Institute.
- Singapore: An annual budget of around A$11 million for its AI Safety Institute.
- Canada: Roughly A$50 million allocated over five years for its AI Safety Institute.
These figures stand in stark contrast to the substantial investments made by the industry, with OpenAI alone reportedly spending US$19 billion on research and development in 2025. Proposals for a significantly increased budget for the Australian institute received strong backing at the recent AI Safety Forum.
The UN report’s findings on AI’s potential to exacerbate inequality underscore the need for further action. Two promising next steps include legislating a digital duty of care to mitigate AI-driven online harms and addressing digital exclusion to ensure equitable access to AI’s benefits for all Australians. Assessments by the Tech Policy Design Institute also point to Australia’s need for enhanced computing power for research and public interest activities, crucial for maintaining its competitive edge and preventing the public sector from falling behind.
Broadening the AI Conversation
A more inclusive and extensive dialogue about AI’s societal opportunities and impacts is essential. To address Australia’s low levels of public trust in AI and effectively manage its potential harms, a wider array of actions is required. The recent shift in government approach suggests a potential for broader engagement.
Australia possesses a unique advantage with its highly skilled AI safety researchers and experts in social impacts, coupled with a societal imperative to navigate AI’s risks and opportunities for the benefit of all citizens. By fostering a wider conversation and implementing robust policies, Australia can aim to lead in responsible AI development and deployment.
