Tweet
Title: Garak
Presenter: Leon Derczynski
Co-Presenter: Erick Galinkin
Location: W306
Day,Time: Fri Aug 9 , 2PM - 3:45PM
Audience: AI, Offense, Defense
Project: https://github.com/leondz/garak/
Abstract:
Garak, Generative AI Red-teaming and Assessment Kit, is a vulnerability scanner for large language models (LLMs) and dialogue systems. It has a host of different probes, each working on different vulnerabilities and payloads. It connects to a broad range of different LLMs. The attacks range between static tests of fixed prompts, to dynamically assembled prompts, to probes that respond to existing model behavior when working out their next move. Community contribution plays a big part of Garak already, with an active repo & over 300 members in the Discord. Garak can assess and attack anything that takes text and returns text, and is already used by many industry players in assessment of internal and external models, including NVIDIA and Microsoft as well as a range of emerging AI Security startups; it’s the #1 ranked tool for LLM security on Hackernews. But we think it’s mostly a lot of fun.
Bios:
* Presenter:
Leon Derczynski is principal research scientist in LLM Security at NVIDIA and prof in natural language processing at ITU Copenhagen. He’s on the OWASP LLM Top 10 core team, and consults with governments and supranational bodies. He co-wrote a paper on how LLM red teaming is like demon summoning, that you should definitely read. He’s been doing NLP since 2005, deep learning since it was more than one layer, and LLM security for about two years, which is almost a lifetime in this field. Finally, Prof. Derczynski also contributes to ML Commons, and regularly appears in national and international media.
* Co-Presenter:
Erick Galinkin is a Research Scientist at NVIDIA working on the security assessment and protection of large language models. Previously, he led the AI research team at Rapid7 and has extensive experience working in the cybersecurity space. He is an alumnus of Johns Hopkins University and holds degrees in applied mathematics and computer science. Outside of his work, Erick is a lifelong student, currently at Drexel University and is renowned for his ability to be around equestrians.
Presenter: Leon Derczynski
Co-Presenter: Erick Galinkin
Location: W306
Day,Time: Fri Aug 9 , 2PM - 3:45PM
Audience: AI, Offense, Defense
Project: https://github.com/leondz/garak/
Abstract:
Garak, Generative AI Red-teaming and Assessment Kit, is a vulnerability scanner for large language models (LLMs) and dialogue systems. It has a host of different probes, each working on different vulnerabilities and payloads. It connects to a broad range of different LLMs. The attacks range between static tests of fixed prompts, to dynamically assembled prompts, to probes that respond to existing model behavior when working out their next move. Community contribution plays a big part of Garak already, with an active repo & over 300 members in the Discord. Garak can assess and attack anything that takes text and returns text, and is already used by many industry players in assessment of internal and external models, including NVIDIA and Microsoft as well as a range of emerging AI Security startups; it’s the #1 ranked tool for LLM security on Hackernews. But we think it’s mostly a lot of fun.
Bios:
* Presenter:
Leon Derczynski is principal research scientist in LLM Security at NVIDIA and prof in natural language processing at ITU Copenhagen. He’s on the OWASP LLM Top 10 core team, and consults with governments and supranational bodies. He co-wrote a paper on how LLM red teaming is like demon summoning, that you should definitely read. He’s been doing NLP since 2005, deep learning since it was more than one layer, and LLM security for about two years, which is almost a lifetime in this field. Finally, Prof. Derczynski also contributes to ML Commons, and regularly appears in national and international media.
* Co-Presenter:
Erick Galinkin is a Research Scientist at NVIDIA working on the security assessment and protection of large language models. Previously, he led the AI research team at Rapid7 and has extensive experience working in the cybersecurity space. He is an alumnus of Johns Hopkins University and holds degrees in applied mathematics and computer science. Outside of his work, Erick is a lifelong student, currently at Drexel University and is renowned for his ability to be around equestrians.