At huntr, we’ve got a thing for celebrating the hackers and researchers shaking things up in AI/ML security. So this time, we're throwing the spotlight on Mokrane Abdelmalek (aka m0kr4n3)—a sharp web vulnerability hunter who’s been consistently uncovering bugs where others might overlook them. He’s got a background in CTFs, a knack for code review, and recently uncovered a nasty stored XSS vulnerability in the open-webui project. The kind of bug that could’ve taken down the whole platform. Let’s dive into how he found this gem, shall we?
My name is Mokrane Abdelmalek and I’m a computer science engineer graduate from the National Institute of Computer Science (ESI) in Algiers. I am currently pursuing a Master’s degree in Networks at Sorbonne Université in Paris. As a security researcher, I focus on web vulnerabilities when hunting for bugs in different platform especially huntr and also a big fan of CTFs, I play with the noreply team almost every weekend.
A year ago, I started hunting on bug bounty platforms, notably HackerOne and YesWeHack. Although I succeeded in reporting some critical bugs, my background as a CTF player makes me more comfortable with white-box testing. So, when I heard about huntr, a bug bounty platform focused on AI/ML products that are open source, I was excited to start doing code reviews and spotting bugs. My teammates and I found many interesting bugs and reported them. We felt comfortable because we could deploy the code locally, debug, and modify it—just like we do in CTFs.
Initially, we received a lot of "informative" reports because not all attack vectors were considered security risks, such as privilege escalation, which disappointed us (a lot, xD). Fortunately, many repositories had web UI servers, so we shifted our focus entirely to web attack vectors. After that, we started closely monitoring new repositories and dedicated a lot of time to avoid duplicate reports, as the competition grew.
Recently, when the open-webui repository was released, my friend @ouxs and I submitted six reports. That was very fun and exciting.
Recently, m0kr4n3, alongside his teammate @ouxs, identified a stored XSS vulnerability in the open-webui repository, an AI interface supporting various LLM models. While this project had seen attention from other researchers, m0kr4n3 decided to dig deeper into the sanitization processes in place. His determination paid off with the discovery of a bypass that led to a critical XSS exploit.
Now, let’s toss it over to m0kr4n3, who’s about to break down exactly how he did it—step by step. Grab a snack.
This XSS vulnerability in the context of the open-webui project is highly dangerous, as it allows any user to escalate privileges to an admin. This enables them to view others' chat history, leak API keys, delete users, and perform various other actions that compromise the integrity and confidentiality of the application. Additionally, it is possible to delete the models used by the application if you have admin privileges, thus affecting the application's availability.
First of all it started with noticing a custom sanitization in the function sanitizeResponseContent located in open-webui/src/lib/utils/index.ts :
After reviewing the code, it initially appeared very secure since it doesn't allow any tags or attributes to pass through. However, we were convinced that a bypass might be possible with the permitted <video> tag. I have to say, this was one of the hardest sanitization bypasses I’ve encountered.
As we observed, the function first processes the <video> tag in a specific format and with defined attributes: <video src="${src}" controls></video>. It then replaces it with placeholders formatted like this: }, before sanitizing any other HTML tags or symbols. Afterward, it re-replaces the placeholders with their original source.
What we attempted was to confuse the replacement process by providing a payload like this: <video src="testing" controls></video>. This causes confusion during the final replacement with the original tags since it picks up the first encountered. By exploring this approach, we eventually crafted a payload that triggers an alert box:
<video src="';<img src=x onerror=alert(1)>" controls></video><video src="" controls></video><video src="video src=" controls></video>
The XSS is triggered when interacting with the dropdown menu, which is a critical and common action—more so than simply clicking on a link or triggering a CSRF attack. Since any user or admin would naturally list the available models, this makes every user a potential victim.
To escalate this XSS and make the admin perform sensitive actions, we initially tried to use JavaScript to send requests. However, due to the confusion that allowed us to bypass the sanitization, we couldn't use any single quotes, double quotes, or backticks (\\), as the payload would no longer trigger the XSS, which made the task even more challenging. Therefore, we needed to find an alternative way to reflect the desired payload.
The solution we found was to use the eval() function and encode the payload characters with their ASCII codes, then decode them using the String.fromCharCode() function. This approach avoids the need for single or double quotes. For example, the alert payload would be:
We created a Python script to generate a payload for any JavaScript code we want the XSS to trigger:
Now, we needed an endpoint that is only accessible by the admin and responsible for a critical operation. We found the /api/v1/functions/create endpoint, which allows the admin to create a function. This function is essentially Python code that will be executed.
First, let's craft Python code to achieve remote code execution (RCE), followed by JavaScript code that will send a request to this endpoint with the Python payload.
Here is the Python code we used:
And here is the JavaScript code:
We’ll get a remote interactive shell that looks like this:
Feeling fired up by Mokrane’s story? Yeah, thought so. Time to stop lurking and start hacking with us at huntr! Our community of hackers and researchers are out here cashing in $$ and shredding through AI/ML vulns like it’s their day job—and we want you in on the action.
Whether you’re just starting out or a seasoned pro, we’ve got the tools and resources—like our Beginner’s Guide to AI/ML Bug Hunting—to help you hunt down the next big bug. Let's go!