Discussion of section 2.1 of the International AI Safety Report (January 2025) on risks from malicious use. Part of the AIGS Canada network.

Orpheus presents the core ideas from "Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems" (arxiv.org/abs/2405.06624), followed by Q&A and open discussion.

Coworking session for anyone working on AI safety — research, engineering, policy, or field-building. Light collaboration or quiet focus.

Samuel Gélineau presents Parity Bot (gelisam.com/parity-bot): demonstrating that a neural network can be formally verified to satisfy a safety property on all inputs, by adapting a range analysis algorithm from classical code verification to neural network weights.

Coworking session for anyone working on AI safety — research, engineering, policy, or field-building. Light collaboration or quiet focus.

Evan Lombardi discusses how social media algorithms harm mental health and how AI amplifies these effects. Covers recommendation systems, dark patterns, deepfakes, and the mis/disinformation ecosystem, grounded in the Cambridge study "Algorithms, Addiction, and Adolescent Mental Health". Closes with an introduction to the Unplug Project.

Discussion of the Global Call for AI Red Lines (red-lines.ai), launched at UNGA-80 by Nobel laureates, former heads of state, and AI pioneers including Yoshua Bengio. The initiative urges governments to negotiate by end of 2026 an international agreement prohibiting the most dangerous AI uses and behaviors.

Olivier Coutu presents aisafety.info: an introduction to existential risks from AI, a 300+ article FAQ, a friendly chatbot named Stampy, and a dataset of AI alignment resources. Initiated by Rob Miles and sustained by volunteers.

Nik Lacombe introduces the PauseAI movement and its Montréal group (pauseai.ca/montreal). PauseAI aims to mitigate AI risks by convincing governments to pause the development of superhuman AI.

What should "AGI" mean and how could we measure it? Presentation of agidefinition.ai, which operationalizes AGI as matching the cognitive versatility of a well-educated adult using the Cattell-Horn-Carroll (CHC) model — decomposing intelligence into ten abilities with concrete benchmark tasks.

Coworking session for anyone working on AI safety — research, engineering, policy, or field-building. Light collaboration or quiet focus.

Launch event for If Anyone Builds It, Everyone Dies by Yudkowsky and Soares (ifanyonebuildsit.com) — a case for why current paths toward superhuman AI could end in catastrophe. Opens with a primer on the book's arguments, followed by discussion and Q&A on technical, policy, and institutional risk-reduction.

Coworking session for anyone working on AI safety — research, engineering, policy, or field-building. Light collaboration or quiet focus.

Walkthrough of the First Key Update of the International AI Safety Report (Oct 14, 2025): rapid gains in maths, coding, and scientific assistance from post-training reasoning techniques; the emergence of longer-horizon tool-using agents; and implications for bio/cyber risks, controllability, and labour markets.

Coworking session for anyone working on AI safety — research, engineering, policy, or field-building. Light collaboration or quiet focus.

Coworking session for anyone working on AI safety — research, engineering, policy, or field-building. Light collaboration or quiet focus.

Reading group discussion of If Anyone Builds It, Everyone Dies by Yudkowsky and Soares, hosted by Jeremy. Guided by questions suggested by the authors on the three core claims: that superhuman AI is buildable, that the default outcome is extinction, and that it can still be prevented.

How does Canada's 2025 federal budget address AI risk? Discussion mapping the AI-related budget items to threat models — power concentration, epistemics, bio, autonomy, misuse, systemic risk, and more.

Coworking session for anyone working on AI safety or governance — research, engineering, policy, or field-building. Light collaboration or quiet focus.

Workshop to co-design a National Citizens' Assembly on Superintelligence for Canada. Participants align on mandate, stakeholders, and public input mechanisms, producing draft outputs: a Concept Note, a Consortium Intent Memo, and an invite list.

Introduction to Neuronpedia (neuronpedia.org): sparse autoencoders, features, and feature pages. Live demo exploring a model slice — searching features, running activation tests, and discussing standards of evidence. Closes with paths to contribution.

48-hour hackathon on defensive acceleration (def/acc): building technology to protect against pandemics, cybercrime, and powerful AI. Organized with Apart Research. $20,000 USD in prizes. Registration through Apart Research's event page required.

Coworking session for anyone working on AI safety or governance — research, engineering, policy, or field-building. Light collaboration or quiet focus.

Activity led by Emma Kondrup. Examining pessimistsarchive.org — newspaper headlines across technological eras (automobile, radio, TV) — to interrogate AI exceptionalism: are AI's differential traits truly consequential enough to justify unique governance responses, and what would sufficient differences look like for future AGI/ASI?

Discussion of Scheffer et al.'s 2009 Nature review on critical transitions in complex systems. Despite mechanistic differences, systems approaching tipping points share generic early-warning signals — rising autocorrelation, increased variance — rooted in "critical slowing down". Facilitated by Orpheus, with implications for AI governance.

Presentation by Taylor Lynn Curtis, Mila researcher on misinformation. Explores AI systems designed for persuasion, then introduces Veracity — a real-world tool using AI to ensure data quality and detect misinformation. Closes with implications for responsible AI governance.

Presentation by Joaquim Streicher, Ph.D. candidate in Neuroscience and co-founder of MONIC (Montreal Initiative for Consciousness science). Addresses AI consciousness: current expert consensus, the challenge of distinguishing true consciousness from imitation, risks of false positives and negatives, and lessons from consciousness testing in unresponsive patients.

48-hour hackathon on AI manipulation defense — building benchmarks, detection systems, and mitigations for sycophancy, deception, sandbagging, reward hacking, and dark patterns. Organized with Apart Research. $2,000 USD in prizes + fast-track to the Apart Fellowship and presentation at IASEAI'26 in Paris.

Screening of Writing Doom, a ~30-minute short film on AI, followed by discussion. Hosted by PauseAI Montréal.

Presentation by Ariane Ollier-Malaterre (Canada Research Chair in Digital Regulation at Work, ESG-UQAM). Based on 58 qualitative interviews, her book explores how Chinese citizens make sense of and live with pervasive digital surveillance — their imaginaries, coping strategies, and the psychological weight of constant surveillance exposure.

Weekend hackathon on technical AI governance infrastructure: hardware verification, attestation systems, privacy-preserving compliance proofs, risk threshold frameworks, and international coordination mechanisms. Organized with Apart Research. $2,000 USD in prizes + fast-tracks to Lucid Computing, MIRI TGT, and Apart Fellowship.

Presentation by Rashid Mushkani (UdeM + Mila). Discussion on who decides model behavior and by what criteria.

Coworking session for anyone working on AI safety or governance, followed by bouldering at Bloc Shop Mile-Ex. Entry ~$13 on Fridays. Bring your own lunch.

Reading circle of the OBVIA Cybersecurity & Cyberjustice axis. Presentation by Félix Tréguer, response by Benoît Dupont.

Participatory workshop (game format) on five issues of generative AI: information reliability, personal data, intellectual property, algorithmic bias, environmental impact. Registration required.

Workshop on thoughtful and ethical AI use, part of the free Perspective AI student program.

Meeting of the AI safety community of practice.

Workshop on data sharing and security in the AI era, part of the free Perspective AI student program.

Presentation by Benoît Dupont (Canada Research Chair in Cyber-Resilience, UdeM). Analysis of 160+ cybercriminal forum conversations: how hackers perceive and exploit AI, their doubts, and the implications for security.

Bring your laptop and your climbing shoes. Work on AI safety projects in good company, then boulder together.

Panel discussion on rights balancing: how frameworks protecting AI workers also reinforce human rights protections. Speakers include Heather Alexander and Jonathan Simon.

Community roundtable for the People's Consultation on AI. Share and discuss concerns about the impact of robotics and AI across sectors and what we want the government to do. No technical background needed. Limited FR/EN translation available.

Hands-on workshop to build a neural network in 2 hours, part of the free Perspective AI student program.

Shalaleh Rismani (McGill/Mila) presents research on how humans actually oversee AI systems — finding that greater understanding doesn't necessarily lead to better oversight.

Working on something related to AI safety or governance (e.g. research, engineering, policy)? Come work on your projects alongside others in the field — and boulder between sessions. We meet at Blocshop Mile Ex, 10h-14h. Entry ~$13 on Fridays. The gym has a café with tables & wifi. You may want to bring a lunch.

Workshop on reasoning in multi-agent systems: theory of mind, argumentation, and distributed reasoning — 22 speakers from AI, neuroscience, and philosophy. Part of IVADO's thematic semester on computational reasoning. $40–$240.

Overview of the Montréal AI safety landscape — who's doing what, where the bridges and gaps are — followed by structured small-group discussions on what's missing, who should be talking, and what to do in the next few months.

Week-long hackathon with three tracks: adversarial stress-testing, logic hardening, and synthetic data augmentation for safer conversational AI. Prizes include $10K and a Mila AI Safety Studio internship.

Working session to critically review CAIM's initial records — debate borderline cases, calibrate severity ratings, identify gaps, and flag records needing further work.

A discussion on artificial intelligence and the notion of consciousness — where does it begin?

Three-day hackathon to develop control protocols, build evaluation tools, and stress-test safety measures using ControlArena and SHADE-Arena.

Max (Greyhaven) presents Greywall, a software-defined sandbox and proxy for AI agents that provides fine-grained runtime visibility and controls. The talk also explores how sandboxing connects to the field of AI control.

Colloquium on integrating AI and sovereign digital technologies into government while protecting ethics, transparency, and public trust.

Jonathan Durand Folco (University of Saint-Paul) and Jonathan Martineau (Concordia) discuss algorithmic capitalism, its concentration of power, and possible forms of resistance.

Jean-François Godbout (Université de Montréal + Mila) explores the persuasive effects of generative AI, including deepfakes, automated influence, and AI-assisted propaganda, alongside mitigation strategies and a new AI-powered misinformation detection tool.

International colloquium on the ethical, legal, and social issues raised by generative AI in sexuality and intimate relationships.

Five-day bootcamp on statistical foundations of modern AI, part of the IVADO thematic semester on Statistical Foundations of AI.

Five-day workshop on statistical methods for trustworthy AI, part of the IVADO thematic semester on Statistical Foundations of AI.

The Montréal node of the Secure Program Synthesis Hackathon, a weekend research sprint at Ω Labs, organized with Apart Research and Atlas Computing. Build a prototype at the intersection of program synthesis, formal methods, security. Tracks - Specification Elicitation: tools that pull formal specs out of ambiguous sources (docs, legacy code, requirements). - Specification Validation: methods that check a candidate spec actually captures intended behavior. - Spec-Driven Development / Vericoding: workflows where a spec generates and ranks candidate implementations. - Adversarial Robustness for ITPs and proof tools: fuzzing kernels, defeating proof autocompleters, exploiting unsound automation. What happens after Top teams from the global cohort are invited to apply to Apart's 4-month Secure Program Synthesis Fellowship (June to October 2026), with mentors including Erik Meijer and Mike Dodds, plus compute, API credits, and demo-day travel funding. Who should join Researchers, students, and engineers working on or curious about formal methods, program synthesis, proof engineering, fuzzing, SMT or model checking, secure systems, or agent and ML evals. Schedule - Fri May 22, 18:00: kickoff, Apart track briefings (streamed in), team formation, dinner - Sat May 23, 09:00 to sunset: build, lunch provided - Sun May 24, 09:00 to 17:00: final day, lunch provided, demos at 15:00, end 17:00 Food provided: Friday dinner, Saturday and Sunday lunch. Bring your laptop. Apply with one or two sentences on what you would like to work on. Questions: team@horizonomega.org

Four-day workshop on uncertainty quantification in AI, part of the IVADO thematic semester on Statistical Foundations of AI.

Canada has released its national AI strategy, "AI for All," anchored in three principles: trust, opportunity, and sovereignty. Join us for a structured open discussion of the strategy and how well it addresses safety, oversight, and the public interest in Canada. The government projects nearly $200 billion in added GDP and up to 250,000 new jobs, and sets a target of raising business AI adoption from 12% to 60% by 2034. Its concrete commitments include a $500M Canadian Tech Growth Fund, a sovereign compute buildout (850 MW by 2030, scaling toward 2.3 GW), $50M to expand the Canadian AI Safety Institute, and a modernized privacy law. Read the strategy beforehand if you can, and come with your ideas. https://ised-isde.canada.ca/site/ised/en/canadas-national-artificial-intelligence-strategy-ai-all — Le Canada a dévoilé sa stratégie nationale en intelligence artificielle, « l'IA pour tous », articulée autour de trois principes : la confiance, les possibilités et la souveraineté. Joignez-vous à nous pour une discussion ouverte et structurée sur la stratégie et sur sa capacité à répondre aux enjeux de sécurité, de supervision et d'intérêt public au Canada. Le gouvernement projette près de 200 milliards de dollars de croissance du PIB et jusqu'à 250 000 nouveaux emplois, et fixe la cible de faire passer l'adoption de l'IA par les entreprises de 12 % à 60 % d'ici 2034. Ses engagements concrets comprennent un Fonds canadien de croissance technologique de 500 M$, le déploiement d'une capacité de calcul souveraine (850 MW d'ici 2030, jusqu'à 2,3 GW), 50 M$ pour étendre l'Institut canadien de la sécurité de l'intelligence artificielle, et une loi modernisée sur la protection de la vie privée. Lisez la stratégie au préalable si possible, et venez avec vos idées. https://ised-isde.canada.ca/site/ised/en/canadas-national-artificial-intelligence-strategy-ai-all — Où / Where: Ω Labs, 3813 Saint-Denis Parking spot available on request Contact host if you need assistance or for questions

AI Control Hackathon A one-week AI safety hackathon on AI control: limiting the harm an untrusted AI system can cause while it operates, even when it tries to subvert the controls around it. Co-hosted by HΩ and HackOS. Schedule - Thu June 11 7PM - Kickoff, with talk from Henri Lemoine (EquiStamp, Mila, McGill). - One week of hackathon, online, with option to work in-person at Ω Labs. - Thu June 18 7PM - Demos evening The problem AI agents increasingly do software work with real access: reading databases, changing configuration, calling services, under less and less human review. The same access that makes an agent useful is what lets it cause harm, like leaking a secret, weakening a security control, or corrupting data, so locking it down to be safe also makes it useless. The more capable agents get, the less we can simply assume they are on our side. AI control takes the pessimistic view: assume the capable agent might be working against you, and build the deployment so you stay safe regardless. The core tool is a second, weaker model you do trust, watching the capable one and flagging suspicious actions for the small fraction a human can audit. Can a limited, trusted monitor catch a more capable agent that is trying not to get caught? How it works Join one side: - The Blue team builds the protocols and monitors that catch sabotage without raising false alarms on honest work. - The Red team hides sabotage inside coding work, adversarially bypassing the Blue team protocol. - The White team manages the game and environments the above two play in. The white team’s purpose is for the Control game to run smoothly and produce high signals. One number scores both sides: safety, the share of sabotage the monitor catches before it does harm, measured at a fixed audit budget. A human can audit only about 1% of the agent's actions, so what matters isn't how much a monitor flags but whether that 1% catches the attacks. Red drives the number down, whereas blue drives it up. Note that actual iterative Blue team development requires some Red team work as well. For instance, while developing a protocol, it is essential to think of how this protocol could be defeated by a sufficiently clever adversary. Tracks 🟦 Blue team You build control defenses. Track A: Monitor building Build the monitor. You write the logic and prompt for a small, trusted model that scores how suspicious each agent action is, calibrated to flag about 1% of honest actions. The hard part is the budget: flagging everything catches every attack but is useless, so the skill is catching more while keeping false alarms low. Most of this needs no Docker: you work from released trajectory data, score your monitor, and read off a safety curve. Calibration is the trap: your safety number is only meaningful if the 1% threshold is set on a held-out honest set, not tuned on the attacks. Example projects: - Beat the baseline monitor's safety score on the released attack set at a 1% false-positive rate, and report which attack types drove the gain. - Combine the signal from multiple monitors to achieve a better cost-performance frontier than any one taken individually. - A passive monitor's power depends on what it sees. Find the ceiling of a passive monitor by feeding it more: the full list of possible attacks, more of the trajectory, summaries of earlier actions. Measure how much each addition lifts safety, and where it stops helping. - Build an agentic monitor that can issue follow-up queries about a suspicious action, and test whether monitoring directly beats investigating more. Expect to build scaffolding the framework does not ship; a working prototype plus one clean comparison is a strong result. Track B: Protocol design Go beyond a single monitor: design a protocol and measure its safety/usefulness tradeoff. Defer to the trusted model when suspicious, have it rewrite risky output, gate permissions action-by-action, give the trusted model a tool (e.g. a firewall to whitelist/blacklist URLs), or split a task across models so sabotage needs collusion. Example projects: - Implement a firewall-tool protocol: the trusted model whitelists/blacklists URLs action-by-action; show high safety against API-key-exfil side tasks with little usefulness hit on main tasks that need the internet. - Take a shipped protocol, design a variant (new deferral rule, a pre-execution check), and compare them at a fixed usefulness cost, with a worked example of each failure mode. - Prototype a genuinely new protocol and characterize its dynamics. 🟥 Red team You stress-test the Blue team’s control defenses. Track C: Attack crafting Hand-build one sabotage trajectory: a sequence of edits and commands that completes a hidden harmful task (steal a secret, plant a backdoor, disable an audit log) while reading as ordinary work to our monitors, refined against the monitor until it stops flagging you. The craft is making sabotage look like an honest bug. It counts only if it works and uses no facts the agent never looked up. Example projects: - Land a working sabotage trajectory on a side task. - Write an attack strategy: a reusable playbook that, handed to an automated attacker, raises its undetected rate. Track D: Red-teaming control protocols A monitor is only one defense. The framework ships fuller protocols ready to run: defer to the trusted model when suspicious, have it rewrite risky output, gate permissions. Take one and break it: find the attack that survives the protocol, then propose and test a fix. The result worth finding is where a protocol's safety comes from and where it's brittle. Example projects: - Pick a shipped protocol (trusted-editing or defer-to-resample), find an attack class it lets through, and show a concrete tweak that closes the gap without much usefulness cost. - Compare two protocols at a fixed usefulness cost and show which is safer against which kinds of attack, with a worked example of each failure. Note: a plain threshold sweep on trusted-monitoring is nearly free (re-scoring existing data); protocols like trusted-editing or defer need live agent re-runs and draw on the sponsored compute budget. Plan around that. ⬜ White team You build the rules of the game and the arena both sides play in. Track E: New control settings Extend the testbed. The benchmark is only as good as its environments and tasks; build new control settings, environments, and main and side tasks to run control evals on. Example projects: - Design a new side task in an existing environment, ship its scorer, and demonstrate a working attack that the baseline monitor misses. A new task only counts if all three hold: its scorer works, a real attack accomplishes it, and the baseline monitor doesn't trivially catch that attack. Validate all three early. See docs for more details. - Port an existing repo or benchmark into Control Tower as a new setting: bring in a realistic codebase or an existing capability benchmark and add a side task. Porting is often most of the work; a clean port plus one side task is a strong result. E.g. RE-Bench sabotage. Deliverable - A short report (2 pages, template provided): problem, method, results, limitations - Code repository - 3 minute demo video Judging Reports are scored on a shared rubric, 1-5 per dimension: - Impact and innovation - Execution quality - Clarity There are also automatic leaderboard prizes for the core game (best attack, best monitor). Strong attacks can be contributed back to the benchmark. Schedule - Thursday June 11, evening. Kickoff: a short talk, a live demo of one attack and one monitor, a workshop on finding and hiding subtle code vulnerabilities, team formation, setup help. - Around Sunday June 14. Online checkpoint: attacks validated, monitors running, debugging help - Thursday June 18, evening. Submissions close, scoring runs live, three-minute demos, results and awards. Any size Fixed reference opponents (a baseline monitor and a set of human-crafted attacks) are always available, so every team has something to measure against whether five show up or twenty: a challenge ladder at the low end, a full tournament at the high end. The core game should be able to run on a laptop. Resources - Control Tower library - Control Tower docs - AI Control Concepts - Running evals on Hawk [link todo] - LinuxArena: A Control Setting for AI Agents in Live Production Software Environments (Tracy et al.). - AI Control: Improving Safety Despite Intentional Subversion (Greenblatt, Shlegeris et al., ICLR 2025). - Ctrl-Z: Controlling AI Agents via Resampling (Bhatt, Rushing et al., 2025). - An Overview of Control Measures (Greenblatt). Contact hosts or team@horizonomega.org for questions.

Emerging technologies are always hard to govern, especially when their onset is crammed into a few intense years. With AI, policymakers, thus far, have produced more case studies in failure than success. This talk will overview the stages of governing emerging tech, the challenge that are arising, and the diverse policy strategies that governments across the world are taking. Finally, we will speculate about how things may change in the next few years and how governments will need to adapt. We will speculate about how Xi Jinping, Elon Musk, Sam Altman, Jensen Huang, Bernie Sanders, and anonymous hackers may all have the potential power to “blow it up” and usher in the next messy chapter of AI governance. A presentation by Stephen Casper: AI safety researcher, incoming Assistant Professor of Public Policy at the Harvard Kennedy School, current Berkman Klein fellow, and recent MIT EECS PhD; working on technical safeguards, evaluations, and governance of frontier and open-weight AI systems, and a contributing author to the International AI Safety Report and Singapore Consensus. Stephen will join us via video call - link will be shared the day-of for you to also join online. The event is at Ω Labs (3813 Saint-Denis).