Top five strategies from Meta’s CyberSecEval 3 to combat weaponized LLMs

submited by

Style Pass

2024-09-04 10:30:06

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More

With weaponized large language models (LLMs) becoming lethal, stealthy by design and challenging to stop, Meta has created CyberSecEval 3, a new suite of security benchmarks for LLMs designed to benchmark AI models’ cybersecurity risks and capabilities.

“CyberSecEval 3 assesses eight different risks across two broad categories: risk to third parties and risk to application developers and end users. Compared to previous work, we add new areas focused on offensive security capabilities: automated social engineering, scaling manual offensive cyber operations, and autonomous offensive cyber operations,” write Meta researchers.

Meta’s CyberSecEval 3 team tested Llama 3 across core cybersecurity risks to highlight vulnerabilities, including automated phishing and offensive operations. All non-manual elements and guardrails, including CodeShield and LlamaGuard 3 mentioned in the report are publicly available for transparency and community input. The following figure analyzes the detailed risks, approaches and results summary.