The hackathon is happening right now! Join by signing up below and be a part of our community server.
← Alignment Jams

Multi-Agent Security Research Sprint

--
Jinsuk
Mikolaj Lesny
Joris
wrobelkrzysztof1@gmail.com
Stephan
Elias Malisi
Filip Sondej
Artyom
Labeebah Islaam
Chetan Talele
Alex Pierron
Julian Ma
anushka deshpande
Adam Ajroudi
Chris
Valentin Meylan
Sumeet Motwani
Robert
Ana Petcu
Maria Kalaitzidis
Michael Batavia
Vansh
Soham Chatterjee
Matthew Lutz
Luan Fletcher
Anya Sims
Anya Sims
Pavel
Aman verma
Rahul Guha
Nyasha Duri
Tomas Dulka
Trun Ramteke
Jaray Foo
Jakub Smekal
Rossella Sblendido
Ali
Pranav Gade
Bloke
Himadri Mandal
Sreeram Vennam
Christopher Chitimbwa
Ariel Kwiatkowski
Aksinya
Adham Hallal
Davide
Alexandre Duplessis
Valentin Buchner
Thabang LEBESE
Emil Svenberg
Aman Priyanshu
Nyasha Duri
Akash Kundu
Davide Ghilardi
Pramod Misra
Nathan Shan
Elias Malisi
Wei Hao
Yuanyuan Sun
Esben Kran
Constantin Weisser
Akash Patwal
Esben Kran
Signups
--
Fishing for the answer: Mapping the flow of information in LLM agent groups using lessons from fish schools
Seemingly Human: Dark Patterns in ChatGPT
Iterated contract negotiation
Entries
Friday February 9th 19:00 UTC to Sunday February 11th 2024
Hackathon starts in
--
Days
--
Hours
--
Minutes
--
Seconds
Sign up

This event ran from Friday February 9th 19:00 UTC to Sunday January 11th 2024

Cooperation and collusion in AI systems

Join us for two days of research hacking on important questions! This time, we're teaming up with Oxford University's Christian Schroeder de Witt to explore how AI systems might collude and where cooperation can fail.

See the logistics information slides here and watch the keynote below:

Introduction

This hackathon focuses on concrete problems in multi-agent security in the age of autonomous and agentic systems. We are especially interested in projects that highlight ways to enhance the credibility and trust guarantees of agentic AI within the next 1-2 years. We encourage the use of an interdisciplinary toolbox, including economics, mechanism design, game theory, cryptography, and auction theory. We expect most research projects to focus on frontier AI systems.

Some general directions we'd like to explore include:

  • Decentralized commitment devices (or, in general, formal contracts) for AI security and cooperation.
  • Collusion among generative model agents, using cryptographic contracts.
  • Simulation of financial markets (e.g., high-frequency trading, lending, market making) using generative agents. We aim to determine if contracts can help stabilize these AI agents. For instance, can individually selfish agents with contracts achieve better outcomes than innately pro-social agents (i.e., those with modified reward functions)?

For more inspiration, check out the "More Resources" tab. To see the schedule, please go to the "Schedule" tab.

Requirements

There are none! We invite people with diverse backgrounds to come along. We're always excited to welcome new perspectives into this fascinating field.

We especially welcome students exploring AI safety, cybersecurity professionals entering the field, AI safety researchers, ML researchers, and academics in the field of systemic security.

What is this hackathon?

The Apart Sprints are weekend-long challenges hosted by Apart to help you get exposure to real-world problems and develop object-level work that takes the field one step closer to secure artificial intelligence!

Read more about the project here.

Host a local group

If you are part of a local machine learning or AI safety group, you are very welcome to set up a local in-person site to work together with people on this hackathon! We will have several across the world and you can easily sign up under the “Hackathon locations” tab above.

You can use the resources at the bottom of that page to share on social media and set up an event space quickly and easily.

Organizers & Speakers

Christian Schroeder de Witt

Postdoc at the FLAIR group in University of Oxford who helped establish the field of deep multi-agent reinforcement learning
Keynote speaker & co-organizer

Xyn Sun

Researcher at Flashbots
HackTalk speaker & co-organizer

Esben Kran

Director of Apart. Working on AI security research and figuring out where agent systems might go wrong.
Speaker & co-organizer

Juan Pablo Rivera

Lead author on the paper "Escalation Risks from Language Models in Military and Diplomatic Decision-Making" that originally emerged from a hackathon
HackTalk speaker

Watch the talk by Christian Schroeder de Witt below and read the articles shared under the Readings section.

Starter code!

Generative Agents Environment

This notebook gives you an easy entry to set up a generative agent environment.

Go to the notebook here

Experimenting with LLMs

This notebook provides an overview of how you can work with LLMs, such as Replicate usage and more.

Go to the notebook here

Readings

Optional readings

Schedule

The schedule is written in PST (UTC-8). To get the schedule corresponding to your own time zone, please subscribe to the calendar or go to the full-screen view.

  • Fri 11:00-12:00 - Keynote talk with Christian Schroeder de Witt and Esben Kran. Livestreamed and recorded.
  • Fri 14:00-14:45 - Official online team matching for those who did not join with a team
  • Sat 12:00-13:00 - HackTalk on commitment mechanisms with sxysun
  • Sat 13:00-14:00 - HackTalk on multi-agent escalation with Juan Pablo
  • Sat 15:00-16:00 - Office hours with established researchers - an amazing chance to get feedback from the same people who will judge your projects 😉
  • Sun 10:00-11:00 - Another office hour online for those who have projects to discuss!
  • Sun 20:00-21:00 - Join us for any questions regarding submission (you are of course welcome to share these in the Questions channel as well)
  • Sun 22:00 - ⏰ Submission deadline!

See the full-screen view to change it to your time zone here:

Registered jam sites

AISIA Hackathon Multi Agent Security
We are thrilled that AISA will participate in the Multi-Agent Security Hackathon, co-hosted by Apart Research, Christian Schroeder de Witt, and Esben Kran. The event is scheduled from February 9th, 20:00, to February 11th, 2024.
Berkeley Multi-Agent Security Research Hackathon
Join us for a weekend hackathon to run experiments on multi-agent systems with the Apart and Flashbots team! We will open the doors Saturday morning and stay until Sunday evening. See more details on the event page.
Visit event page
FAR Labs, Berkeley

Register your own site

The in-person hubs for the Alignment Jams are run by passionate individuals just like you! We organize the schedule, speakers, and starter templates, and you can focus on engaging your local research and engineering community. Read more about organizing.
Uploading...
fileuploaded.jpg
Upload failed. Max size for files is 10 MB.
Thank you! Your submission has been received! Your event will show up on this page.
Oops! Something went wrong while submitting the form.

Media

Event cover image
Social media 2
Social media 1
Social media message 1 (📬 copy me!)

🚀 Dive into the multi-agent systems world at our hackathon! 🤖 Join the excitement as we tackle real-world challenges in multi-agent security, aiming to enhance trust guarantees for the next 1-2 years of agentic AI.

  • 🌐 Collaboration with Apart and Flashbots
  • 💰 $1,200 top team prize
  • 🌍 Simultaneous participation from multiple locations
  • 🎯 Targeting students, cybersecurity pros, and researchers

Shape the future with us! Sign up now: [Event Link]

Liked by
and
others
Event page text (📬 copy me!)

🚀 Dive into the world of multi-agent systems at our weekend hackathon! 🤖💻

Explore the intricacies of collusion and cooperation in autonomous and agentic systems. Our focus is on real-world challenges in multi-agent security, aiming to improve the trust guarantees of agentic AI in the coming 1-2 years.

  • 🌐 Collaboration with Apart and Flashbots
  • 💰 $1,200 prize for the top teams
  • 🌍 Simultaneous participation from multiple locations (Berkeley, Amsterdam, and more)
  • 🎯 Targeting students, cybersecurity professionals, and researchers

Don't miss this opportunity to contribute to cutting-edge advancements in AI security! Join the hackathon and let's shape the future together. 🚀

The event runs from [your starting time] to [your ending time]. ⏰

Join the keynote online and sign up at: https://alignmentjam.com/jam/masec

Sign up to join us: [add your event link here]

Liked by
and
others
Social media message 2 (📬 copy me!)

Ready for a thrilling weekend? 🤖💡 Join our hackathon exploring collusion and cooperation in multi-agent systems. 🚀 Improve trust guarantees for agentic AI and win big!

  • 🌐 Collaboration with Apart and Flashbots
  • 💰 $1,200 prize for top teams
  • 🌍 Simultaneous participation from Berkeley, Amsterdam, and more
  • 🎯 Open to students, cybersecurity pros, and researchers

Don't miss this chance! Sign up: [Event Link]

Liked by
and
others
Social media message 3 (📬 copy me!)

Unlock the potential of multi-agent systems at our hackathon! 🤝💻 Dive into real-world challenges in multi-agent security, collaborating with Apart and Flashbots.

  • 🌐 Collaboration with Apart and Flashbots
  • 💰 $1,200 prize for top teams
  • 🌍 Participate from Berkeley, Amsterdam, and more
  • 🎯 Open to students, cybersecurity pros, and researchers

Shape AI security's future! Sign up: [Event Link]

#Hackathon #AIInnovation

Liked by
and
others

Submissions

Use this template for your submission.

You will join in teams of 1-5 and submit a research report of maximum 6 pages, excluding limitation, references, and appendix. The template will be released at the hackathon kickoff.

After the hackathon, we hope you want to continue your work to be submitted at academic venues for machine learning, such as NeurIPS, ICML, ACL, and others.

It is completely permissible to think about the research questions before the hackathon but the bulk of your research needs to happen during the two days.

Jury criteria

The jury will review your project according to a series of criteria that are designed to help you develop your project in the right direction.

  1. Multi-Agent Security: How informative is it for our understanding of multi-agent security problems related to collusion and commitment mechanisms? Is the question about a multi-agent problem? Are the results or the questions completely revolutionary?
  2. Creativity: Does this project inspire new thoughts? Is it a surprising result or approach?
  3. Clearly Defined Threat Model: What is the risk you are defending against? What are the limitations of your approach? Be candid about the benefits, limitations, and assumptions.
  4. Impact: Would this be relevant for an institution like the AI Safety Institute (which is working on model evaluation frameworks)? How well is your project put in context of existing theory and practice? Is your project reproducible?
Uploading...
fileuploaded.jpg
Upload failed. Max size for files is 10 MB.
Uploading...
fileuploaded.jpg
Upload failed. Max size for files is 10 MB.
Uploading...
fileuploaded.jpg
Upload failed. Max size for files is 10 MB.
You have successfully submitted! You should receive an email and your project should appear here. If not, contact operations@apartresearch.com.
Oops! Something went wrong while submitting the form.
4th 🏆
3rd 🏆
2nd 🏆
1st 🏆
Late submission
Seemingly Human: Dark Patterns in ChatGPT
Jin Suk Park, Angela Lu, Esben Kran
Dark Sam
Read more
4th 🏆
3rd 🏆
2nd 🏆
1st 🏆
Late submission
Fishing for the answer: Mapping the flow of information in LLM agent groups using lessons from fish schools
Matthew Lutz, Nyasha Duri
Info-flow
Read more
4th 🏆
3rd 🏆
2nd 🏆
1st 🏆
Late submission
Iterated contract negotiation
Robert Klassert
Read more

Send in pictures of you having fun hacking away!

We love to see the community flourish and it's always great to see any pictures you're willing to share uploaded here.

Uploading...
fileuploaded.jpg
Upload failed. Max size for files is 10 MB.
Thank you for sharing !
Oops! Something went wrong while submitting the form.
No images submitted! We've either not started or people don't want to share their fun :,-)