Introduction
At huntr, we love to celebrate the incredible talent working with us to build a safer AI powered world. With over 15,000 hackers and threat researchers, our community is full of dedicated individuals working tirelessly to uncover and fix AI/ML vulnerabilities. Today, we're excited to spotlight one of our standout huntrs, Peng Zhou, also known as @zpbrent. An Associate Professor at Shanghai University, zpbrent has made significant contributions to AI/ML open-source software (OSS) security. In this blog, we’ll dive into zpbrent’s journey, highlight his key achievements, and explore his recent discovery of a critical vulnerability in Hugging Face Transformers.
Meet zpbrent
zpbrent is not only an Associate Professor at Shanghai University but also a dedicated bug hunter.
In a recent conversation, he described himself as "a bug hunter for open-source software and the discloser of over seventy CVEs with high impacts in AI and Web communities." His expertise in uncovering critical vulnerabilities has influenced platforms like Microsoft Azure, Facebook, and Hugging Face.
For a deeper dive into zpbrent's work and experiences, read his detailed blog where he shares his journey of discovering and reporting vulnerabilities, and presenting his findings at Black Hat Asia 2024.
zpbrent's blog: My LLM Bug Bounty Journey on Hugging Face Hub via Protect AI
zpbrent's AI/ML Bug Bounty Journey with huntr
zpbrent's journey with us started in early 2021. While researching Node.js vulnerabilities, he sought out platforms dedicated to bug reporting and rewarding, which led him to huntr.
"I found huntr unique since it covered nearly all the open-source software for bug bounty at that time," zpbrent explains.
In September 2023, huntr shifted its focus to AI open-source software under Protect AI's guidance, where the first AI specific bounty was held targeting Hugging Face Transformers. This new direction caught zpbrent's attention, leading him to invest significant effort into analyzing this target. His dedication paid off when he discovered several vulnerabilities, including the RagRetrieve vulnerability.
Discovering the RagRetrieve Vulnerability in Hugging Face Transformers
One of zpbrent's most notable achievements is the discovery of the RagRetrieve vulnerability in the Hugging Face Transformers library. This vulnerability stems from the risky use of the pickle.loads function, which can be exploited if not properly managed. Peng’s research into the library uncovered how this function could lead to serious security risks.
zpbrent elaborates, "The root cause of these vulnerabilities is the abuse of the risky pickle.loads functions that can be exploited from Hugging Face’s demo codes."
For a detailed account of his discovery process, the challenges he faced, and the broader implications, check out his blog: My LLM Bug Bounty Journey on Hugging Face Hub via Protect AI.
Impact and Significance
The discovery of the RagRetrieve vulnerability has had a considerable impact on the AI/ML community. It highlighted critical security issues, prompting patches in the Hugging Face Transformers library. zpbrent's findings were validated as critical (9.0 CVSS), earning him a $1,500 bounty from Protect AI—cha-ching! This vulnerability, published as CVE-2023-6730, represents an important development in AI/ML security.
Join the Hunt
We're stoked to spotlight zpbrent and his contributions to the huntr community. Through our huntr Spotlight initiative, we celebrate individuals like zpbrent who show exceptional skill and dedication in finding bugs and vulnerabilities in AI/ML applications. Who knows, you could be next! Stay tuned for more spotlights on our amazing huntr members and their journeys in the world of AI/ML security.
Ready to get started? Check out our resources, including our Beginner's Guide to AI/ML Bug Hunting, and join the hunt today!
