Abusing the osquery “curl” table for pivoting into cloud environments
About osquery
IT professionals often need to answer questions about what is happening in the operating systems of the fleet they manage or secure. Needs would include performance management, software inventories, or even threat hunting and incident response. In order to solve this problem using an easy to use interface, Facebook created osquery in 2014, and published it as open source software.
The way osquery works is by offering relational tables (some of which are general and others which are OS-specific) which can be queried using SQL and allow you to inspect live information from hosts in your fleet. The information offered through this simple model would otherwise require complex and varied methods for collection and normalization, so this is a huge win.
There are currently 257 tables that can be queried, which are listed at https://osquery.io/schema/. The combination of tables and the queries allows IT and security professionals to answer a variety of questions which can then be continuously monitored (through scheduled queries), with optional alerts if particular values are found or if changes in certain values are detected.
One big underlying assumption, though, is that osquery takes great care to not allow anyone to obtain potentially confidential data from the hosts or environment they run on. For example, great care was taken to not allow reading arbitrary files by default through osquery. This is a great principle to follow.
It is also important to note that, since it’s open source, osquery is widely used in an explicit or hidden way in several management and security offerings such as those in the MDR, MSSP and EDR categories.
Exploiting the curl table
This is when we need to bring up the existence of the curl table in osquery. Querying this table allows you to perform a variety of HTTP requests from the host running osquery and obtain the response returned from the server. So essentially this is a vehicle for remotely executing a limited analog of the versatile command-line curl utility.
During preparations for an upcoming osquery and Kolide course we’ll offer in Brazilian Portuguese (with all revenue being diverted to charities helping those affected by COVID-19), we had an idea. What if we used the curl table to exploit HTTP-accessible services on an organization’s internal network? Or what if we used it to query an Elasticsearch cluster? Or to access the EC2 instance metadata service? And the answer is that yes, this will allow us to gain access to potentially sensitive data in a typical environment.
To put this finding in perspective, here at Tenchi Security are really focused on cloud security. So the angle of accessing the EC2 instance metadata service was the one that immediately grabbed our attention. This is how the AWS documentation describes the EC2 instance metadata service: “Instance metadata is data about your instance that you can use to configure or manage the running instance. Instance metadata is divided into categories, for example, host name, events, and security groups.”
One of the most critical ways in which the metadata service is used is to allow AWS IAM privileges to be granted to EC2 instances. By default any process inside an instance can access the metadata service at http://169.254.169.254/. When privileges were assigned to an instance, this service can be used to obtain temporary credentials (Access Key, Secret Key and Session Token) which can be used to perform AWS API calls using said privileges.
So, any attacker that obtains query access to an osquery fleet can query the AWS metadata service using the curl table. This in turn allows the attacker to obtain valid temporary credentials to access the AWS APIs with the instance-assigned privileges in a convenient and scalable fashion. This is clearly a useful technique for pivoting from the osquery access to attacking the AWS control plane.
See below a proof of concept we performed on a test environment:
Having obtained these credentials, an attacker could for example create a new AWS CLI profile and use them to map which privileges they contain, and also search for interesting ways to obtain persistence or valuable data to exfiltrate. Below we show a non-exhaustive list of commands the attacker could execute:
It’s importante to note that these temporary credentials can typically be used fom anywhere that can reach the AWS API, which are open to the entire Internet. There is no enforcement by default that would stop an attacker from using those credentials on a host outside of your AWS environments to access your AWS configuration and data. Examples of ways in which valuable data could be exfiltrated directly to external environments would be downloading files from S3 buckets or performing queries on DynamoDB or RDS databases.
Mitigations
One effective way to mitigate this on AWS is to enforce the use of v2 of the instance metadata service. Since this requires the use of custom HTTP headers, which the osquery curl table does not allow, it adequately protects against this technique. However, it is worth confirming that all of the software on your environment is using up to date SDKs that support the new version of the service, as even AWS can’t seem to keep up.
If you don’t use the curl table in osquery, we also strongly suggest you disable it (–disable_tables curl), which would prevent this technique from working.
We believe the maintainers of the osquery project should disable this table by default for security reasons, similar to what was done for the carving functionality. We have opened an issue on the osquery GitHub repository to request precisely that, which you can subscribe to and upvote if you are affected by this.
Alternatively a filtering could be implemented to deny queries that access the URLs used to obtain the temporary credentials from the EC2 instance metadata service specifically. This, however, would be a default allow approach that would still not protect against access to unauthenticated Elasticsearch or MongoDB instances, for example.
According to Guillaume Ross, Principal Product Manager at Uptycs, “Yes, the curl table can be used to target other systems, like AWS in your example, but also other local systems. Hypothetically, it could be used by someone to query IoT devices on home networks of employees. Of course, granting access to osquery data period provides those analysts with privileged data, which is the same with most other security tools. To help manage this responsibility, we believe some tables, like curl, should be off by default so it’s a conscious decision by an administrator to grant access. We’ve updated our version of osquery to disable the curl table by default, just like the file carving table. The use of the enable_curl flag is now required to turn it on. The Uptycs platform itself also allows the use of roles, so that not everyone with access to osquery data can change the configurations.”
In short, ensure you are properly limiting query access to your osquery fleet. Depending on which management layer you use for your osquery fleet management, it’s possible you might be able to limit and monitor access to the curl table specifically.
To summarize our finding, if you are the customer of an MSSP, MDR or SaaS offering which involves third parties having query access to osquery deployed in your organization, take special care. The curl table violates the widely held assumption that osquery cannot be abused to pivot or move laterally into your environment. So take our findings into account in your risk management decisions.
Happy detection!
This article was written by Alexandre Sieira and Rodrigo Montoro, based on research performed by Rodrigo Montoro.
Updated May 21st to include a quote from Uptycs.
