Wednesday, February 20, 2008

Making Sure Your Bro Code Does Not Leak

This is for people hacking on Bro's C++ source code. Internally, Bro's memory management is pretty complex and when writing new code it's unfortunately pretty easy to introduce memory leaks. If Bro's memory footprint keeps growing over time (hours or days), and there's no script-level state table to blame for what you see, it might be a leak in Bro's C++ event-engine.

Bro has been in use for many years and we are pretty confident that there aren't any major leaks left in the more common code paths these days. However, there's also code which isn't used very regularly and and it still happens that people run into leaks in these parts.

But most importantly, for new code it is crucial to make sure that it does not introduce new leaks. The lesson we learned in the past is that even the tiniest leak can have devastating effects when it occurs for every connection Bro analyses. Whenever you are writing code for Bro's event engine, such as a new protocol analyzer, it's a good idea to double-check that all the memory it allocates will for sure be released later. As a rule of thumb: whenever you allocate memory but cannot guarantee that it will be freed at a well-defined point of time other than termination, there's likely something wrong.

There are various tools out there which can help with leak-checking. In the past we had most success with valgrind (good but slow with large input traces) and Google's perftools. Bro has some support built in for perftools which I'll summarize in the following. Note that perftools' leak-checking works on Linux systems only at the moment.

In the past, we have found quite a few leaks in Bro with perftools and we recommend to run new code through it to see whether there's something turning up. The nice thing about perftools is that its performance is sufficiently good that one can actually run Bro with some non-trival amount of traffic, which is crucial because often leaks are in code paths not triggered when analyzing just a few connections.

To use perftools for checking Bro for leaks, get the current source code from perftools' download page. Compile and install it with the usual ./configure && make install cycle.

Next, you need to configure Bro with debugging code and enable perftools support:

   > ./configure --enable-debug --enable-perftools 
   > make 

Make sure that the output at the end of the configure run indicates that configure has indeed found the perftools installation:

                    Bro Configuration Summary
    ==========================================================
      - Debugging enabled:      yes
      [...]
      - Using perftools:        yes

If it didn't find perftools, try giving it the paths to your perftools installation directly, as in the following example (replace <prefix> with whatever prefix you were using when configuring the perftools distribution):

 
  > export LDFLAGS=-L<prefix>/lib
  > export CFLAGS=-I<prefix>/include
  > export CPPFLAGS=-I<prefix>/include
  > export LD_LIBRARY_PATH=<prefix>/lib
  > ./configure --enable-debug --enable-perftools 

Once the configure output looks ok, compile Bro as usual with make.

The last preparation step is setting perftools' environment variable HEAPCHECK to local to activate perftool's memory checking:

 
  > export HEAPCHECK=local

Finally, start Bro as usual but add the option -m to the command-line, e.g,:

 
  > bro -m -r trace tcp

You likely want to run from a trace instead of live because the memory checking decreases Bro's performance significantly.

Once Bro terminates (which is fine to trigger via CTRL-C if necessary), perftools will output leak information into /tmp/bro.*.heap files. (There might be output saying that something "has failed" but it seems that you can you can safely ignore this.). The recorded data will only contain leaks which appeared within Bro's main packet processing loop; any leaks during initialization and termination are skipped as they won't cause any trouble during live operation.

At termination, Bro will print out a pre-built command-line to start the pprof utility. pprof is perftools' user interface to inspect the leak information, and you can just literally copy & paste the command line into your shell (except that you might want to skip the -gv option to suppress the graphical output in favor of the interactive interface)

Inside pprof, typing help shows the available commands. Most importantly, there's top which shows the top leaks as determined by perftools. See here for more information about how to interpret its output, and here for more details about perftools' heap checking in general.

Friday, February 15, 2008

Python Bindings for Broccoli

We have developed a set of Python bindings for Broccoli, Bro's client communication library. The bindings make it easy to send and receive Bro events from external Python scripts. See the module documentation for more information.

Wednesday, February 6, 2008

Now Accepting Internship Applications

We are now accepting applications for summer interns for the upcoming summer. Please see this announcement for details.

Friday, February 1, 2008

Sidejacking, Forced Sidejacking, and Gmail

The following is not original, just an analysis and summary of other work and the current security state of Gmail's login cookie and sidejacking. Its been well known (Errata Security) for months that Google's cookie management has some serious problems, even when you use SSL. Mike Perry's Bugtraq posting from over half a year ago explains it well.

Executive summary: Google's failure to properly use the secure attribute on Gmail's cookies means that Gmail over SSL offers effectively no security against an active attacker when compared with Gmail without SSL. This is a well known problem.


First, some general purpose background on cookies and sidejacking:

What is a cookie? A cookie is a small value stored in the user's web browser that is sent to the web server. Whenever a webpage is fetched, the browser sends all cookies associated with that web site. Likewise, the web site can set cookies in the reply.

Cookies are often used as "login cookies" or "session cookies", a way for the server to know who a user is and that he's allowed to log in. Often, these cookies are marked as secure, which means they will only be sent over an encrypted (https) session. But if they aren't marked as secure, they will be sent with every page request made to the server, regardless of encryption.

What is sidejacking? Sidejacking is simply stealing the session cookies when they are sent over an insecure (normal HTTP) session and reusing them. If you can grab someone's login cookie, you can impersonate them on whatever website the cookie is for, such as read their email on Gmail or Hotmail, or whatever other impersonation is desired.

What is forced sidejacking? (Also known as a cross site request forgery attack, but that's not as catchy a name). If a session is unencrypted, someone able to observe packets (such as the guy sitting next to you in Starbucks) can obviously see, intercept, and reuse any cookies seen in the classic Sidejack attack. But often an attacker wants to get the cookies associated with a connection which is not currently active, or when the session is encrypted.

In this case, if the attacker can inject packets, instead of being an eavesdropper he can become a man in the middle. And since listening in on packets to capture people's email accounts is already illegal, there is no reason for the attacker who's sitting next to you in starbucks to limit himself to just passive snooping.

By performing packet injection or any one of half a dozen other attacks, the attacker can fool your web browser into going where the attacker wants and revealing all the cookies for the site he wants, by modifying your web browsing.

As an example, the attacker could replace the victim's fetch of any normal web page with an "HTTP 302 Temporarily moved" pointing to the web site he wants to steal the cookies from. Thus the victim's browser then thinks "Oh, I should visit this instead", so it creates a request which includes all cookies which aren't secure. The attacker then replaces the reply with another "HTTP Temporarily moved", back to the page the victim was trying to visit. [1]

As a result, the attacker has learned what the cookies are for the victim's login to the targeted site, but the victim doesn't observe anything (it just looks like the page took a little longer to load, and the address bar might flicker).

Any site which does not allow HTTPS is trivially vulnerable to sidejacking and forced sidejacking. Thus, Hotmail can always be forced sidejacked. However, sites which use SSL to access can set the secure attribute on the login cookie, which should prevent this attack.


Unfortunately, Gmail, although it supports SSL, does NOT set the secure attribute on the login cookie (which is named "GX"). Thus if you manually type https://mail.google.com, you are safe from a passive eavesdropper (passive sidejacking), but just as vulnerable to an active attacker (forced sidejacking) as you are with the "SSL only for the login" Hotmail or Gmail over plain http! This is because although your session is encrypted, an attacker could still redirect any of your other surfing to "http://mail.google.com/", which reveals your GX cookie to the attacker. And since its just as illegal to sniff a person's cookies to access their email account as it is to modify their browsing, and almost as easy to inject traffic instead of just sniffing traffic, Gmail's SSL offers effectively no security over the non-SSL option [2].

This situation will remain until Google changes the GX cookie to secure. But the situation, as you will see, is more complex then a one-line change.


Naturally, I thought there would be an easy fix: use greasemonkey to change the GX cookie to secure whenever I visit Gmail. (Greasemonkey allows you to inject arbitrary Javascript into a set of pages you view, which can both read and change cookie attributes). Now if the attacker tried to actively sidejack me, my browser just wouldn't send the cookie because its not an HTTPS session.

But that doesn't work, as Google uses a two stage login process for Gmail, Blogger, etc which prevents the quick fix from working. There is a local session cookie (GX) and a global user login cookie group (SID (insecure) and LSID (secure)) which are used by Google's single signon system. The LSID cookie is restricted to www.google.com/accounts, and has the secure attribute set, to the attacker can't grab it directly.

Yet an active attacker can use the global login cookie directly, if he's performing forced sidejacking.

The Gmail login procedure (verified as of 2/1/2008) works as follows:

  • If the GX cookie is set, log in the user.
  • If the GX cookie is NOT set, perform an HTTP redirect to https://www.google.com/accounts/serviceLogin...
  • https://www.google.com/accounts checks if the LSID and SID cookies are valid. If so, it redirects to https://mail.google.com/..{auth_info}.. which sets the GX cookie and redirects the user back to http://mail.google.com (if no SSL was used) or https://mail.google.com (if https://mail.google.com was entered manually).
  • Otherwise www.google.com/account/serviceLogin gives the login prompt

This is why, if you are logged into Gmail, you can log into blogger by just clicking the "log me in" button: blogger redirects you to https://www.google.com/accounts/..., which checks the same SID and LSID cookies and then redirects you back to Blogger.

But the result is Greasemonkey can't fix the Gmail cookie, because all the attacker needs to do is redirect the victim's surfing to http://mail.google.com. If the GX cookie is set, the attacker gets it, and redirects the victim back to his original web site.

Otherwise, the attacker lets http://mail.google.com redirect the victim to https://www.google.com/accounts/.. which redirects the victim to https://mail.google.com/... which sets the GX cookie and gives an HTTP redirect back to http://mail.google.com/... which will happily spit out the GX cookie, followed by the attacker redirecting the request back to where the victim was going in the first place.

Since no page ever loaded (it was all "HTTP 302 moved temporarily"), the browser reveals the GX cookie before any page loaded, and therefore before Greasemonkey-injected Javascript can run.


The result is, IF the victim is logged into ANY single-sign-on Google service (even using SSL) and visits ANY web page while still logged in, an active attacker can capture login cookies for any Google service which allows (not requires, just allows) non-SSL use, even if they aren't currently logged into the particular Google service.

This also suggests that attempting to fix other sites with Greasemonkey wouldn't work either if they have any form of "URL for SSL that transitions to non SSL and updates the cookie" that the attacker can inject to extract a session cookie from. And it points out the problem with any site which uses "SSL-only" to check an existing login cookie to generate a session cookie: to an active attacker, although the attacker can't capture the SSL-only login cookie, he can use the login cookie to generate a session cookie.


In the specific case of Google, Google needs to infer and set "paranoid" cookies. [3] If the user logs into a Google service by manually selecting "https", it checks if the local login cookie also has a corresponding local paranoid cookie. If so, it should just proceed normally after ensuring that the login and paranoid cookies have the secure attribute set.

But if the paranoid cookie is not set, but the user explicitly selected https, it should redirect to https://www.google.com/accounts... regardless of the cookie status. In this case, Google should set a GLOBAL paranoid cookie. This paranoid cookie should cause https://www.google.com/accounts... to always redirect to https if the service supports it, and any local service login cookie to be set with the SECURE attribute and a corresponding local service paranoid cookie.

Now only services which don't support HTTPS for all operation (like blogger) could be actively or passively sidejacked, and once a user logs in with https://, all newly authenticated sessions will default to https until the user manually logs out by pressing the logout button.


Finally, this problem is not unique to Google, as there are rumors of other significant sites which fail to properly set secure on their session or login cookies, even when all traffic should be restricted to encrypted links.


[1] There are other ways to inject content as well to force the victim's browser to visit a targeted site, such as iframe injection through ARP cache poisoning.

[2] The only exception is if ONLY gmail is accessed from the web browser (no other browsing or even RSS fetches) and the user clicks the "logout" button when done. Since only the truely paranoid would do such a thing, I believe it is safe to say that Gmail with https really is not more secure than Hotmail in the face of an active attacker.

[3] Google doesn't use HTTPS-only for gmail for several reasons, but the biggest one is that HTTPS has considerably more latency when starting a connection, something which occurs often in web browsing. Yet this is obviously not a concern for anyone who manually selects an encrypted link.

Many thanks to Christian Kreibech, Vern Paxson, and Mike Perry for comments and feedback.

Edit/Extension: Here's how to find out if a site you use is vulnerable.

  • Clear all cookies (in firefox, Preferences->privacy)
  • Go to the site and log in.
  • Close the browser window.
  • Reopen the site, if you become logged in, then there is a saved login cookie.
  • Now close the window. In Firefox, open up "Preferences/Privacy", manually delete EVERY cookie marked as "SECURE"
  • Reopen the site. If you become logged in again, the site is vulnerable!

    NOTE PLEASE: As of Sept 2008, gmail does appear to be inferring that the user wants SECURE access when the user manually inputs https://mail.google.com/, in addition to providing an "always use SSL" preference. Thus the attack on the GX cookie is NOW OBSOLETE, the problem with gmail has been fixed. Many other sites may remain vulnerable, however.