What the Security Manager Does

SecurityManager is an abstract class that any application developer can extend to implement a set of controls. SecurityManager contains a set of methods with names starting check , for example checkWrite() or checkConnect() . These methods answer the question "is the applet allowed to do this?" either by quietly returning to the caller (an implicit "yes") or by throwing a security exception (an emphatic "no").

Although the class itself is abstract, the methods within it are not, which means that if a subclass of SecurityManager does not implement a particular method, a default behavior will result. The default, in every case, is to deny the check by throwing an exception.

The security manager installed in most browsers overrides some of the default methods, but is still very restrictive, so that it prevents the applet from doing anything that would compromise the system. See Security Manager Controls summarizes the checks and the normal browser implementation.

Operation of the Security Manager

Although any Java program, applet or application, can extend SecurityManager, the JVM will allow only one security manager to be active at a time. To make a security manager active you have to call a static system method: java.System. setSecurityManager(). This can be done only once in an application environment; any subsequent call results in an exception. In the case of an applet, the web browser has already installed a security manager as part of the JVM initialization. This means, assuming that the trusted classes are not subverted, that an applet has no choice but to live within the limitations of the security manager provided by the browser.

The installed security manager is only really active on request: it does not check anything unless it is called by other system functions. See Security Manager Operation illustrates the flow for a specific restricted operation, establishing a network connection. The calling code creates a new Socket class, using one of the constructor methods it provides. This method invokes the checkConnect method of the local SecurityManager subclass instance.

Class Loader/Security Manager Interdependence

Although the three elements of JVM security - class loader, class file verifier and security manager - each have unique functions, this example illustrates their interdependence. The security manager relies on the class loader to keep untrusted classes and local classes in separate name spaces and to prevent the local trusted classes from being overwritten (for example, by a Socket class that failed to invoke checkConnect).

Conversely, the class loader relies on the security manager to prevent an applet from loading its own class loader, which could flag untrusted code as trusted. And everything relies on the class file verifier to make sure that class confusion is avoided and that class protection directives are honored.

The bottom line is this: if an attacker can breach one of the three defenses, the security of the whole system is usually compromised.

Attacking the Sandbox

We have now seen how the different parts of the Java defense act together to create a secure environment in which applets can run. If everything is working correctly, you should be safe from applets that try to attack your browser system or use it to mount attacks on other systems. In theory...

In practice, a number of holes have been found in the implementation of the Java defense, and a variety of attack applets have been demonstrated that exploit them. We do not go into the details of these applets here, partly because all of the publicized holes have already been closed by the main browser vendors, but mainly because most of them have already been described in detail in Java Security: Hostile Applets, Holes and Antidotes , by Gary McGraw and Ed Felten. Dr. Felten is the leader of the Princeton Secure Internet Programming team, which has, more than any other group, subjected the JVM environment to scrutiny and attack in its lab.

Attack techniques do not stand still, so you should also regularly monitor the sources listed in See Sources of Information about Java Security .

It is not surprising that holes have been found in the Java defenses. The JVM is a large piece of code and, inevitably, there are bugs in it. Some of the attacks have exploited bugs, but most of them rely on finding ambiguities: using JVM facilities in a way that the original writers did not envision. If one were to redesign Java from scratch, with the benefit of hindsight, it would be possible to reduce the areas in which there is scope for ambiguity. However, we should not let this detract from the fact that, in general, the Java defenses have proven very strong and effective.

Types of Attack

Although we do not describe any attacks in detail, it is worth summarizing some of the techniques that have been successfully used:

• Infiltrating local classes . You will have realized from the descriptions of the class loader and security manager functions that they depend completely on the integrity of Java classes on the local browser disk. This applies not only to "system" classes - the java.* classes of the JVM - but to any class installed in the browser system (in the browser home directory or in the CLASSPATH). This is because these classes operate outside the controls of the sandbox.

There was a bug (discovered by David Hopwood) that allowed an applet to load a class from any directory on the browser system. This has been fixed, but opportunities still exist for the opportunist cracker. Downloading code packages from the Internet has become a part of everyday life for many people. Any of those packages could have been modified to plant a Trojan horse class file along with their legitimate payload. Of course, this is not just a Java problem, but more like a new form of computer virus. One solution lies in signed content, so that you know that the package you download has not been tampered with. JDK 1.2 also promises some additional protection by applying restrictions to locally-loaded classes, as described in See Protection Domains .

• Type confusion . Java goes to great lengths to ensure that objects of a particular type are dealt with consistently, whenever they are referenced. We see this both in the compiler and later in the third pass of the class file verifier (see See Pass 3 - Bytecode Integrity Check ). It is crucial to the operation of the sandbox that the class of an object and level of access it allows (as specified by the private , protected o r public keywords) is preserved. In the JVM, objects are referenced by entries in the constant pool. As the example in See The Constant Pool showed, each entry includes the type of the referenced object.

If, somehow, an attacker can create an object reference that is not of the type it claims to be, there is a possibility of breaking down the sandbox protection. Several exploits have shown ways to achieve type confusion, by taking advantage of a various flaws, such as:

• A bug that allowed a class loader to be created but avoided calling the ClassLoader constructor that normally invokes checkCreateClassLoader() (see See Security Manager Controls ).
• Flaws in JVM access checking which allowed a method or object defined as private in one class to be accessed by another class as public.
• A bug in the JVM that failed to distinguish between two classes with the same name but loaded by different class loaders.
• Network loopholes . The first JVM flaw to get worldwide attention was a failure to check the source IP address of an applet rigorously enough. This was exploited by abusing the domain name service (DNS, a network service responsible for resolving names to addresses and vice versa) to fool the security manager into allowing the applet to connect to a host that would normally have been invisible to the server from which the applet was loaded. In this way the attacker could access a system that would normally be safe behind a firewall.
• JavaScript back doors . There was a series of JavaScript exploits that allowed a script to persist after the Web page it was invoked from had been exited. This was used to track the user's Web accesses. The flaw was fixed, but then reappeared when Netscape introduced LiveConnect, which allows a JavaScript script to create Java objects and invoke Java methods. Both languages have strict limitations on what they are allowed to do, but the limitations are different limitations. By combining them you effectively get a union of the two protection schemes.

Looking at this catalog of flaws, you may feel gloomy about the whole question of making Java secure. However, the encouraging thing about these examples is that they have all been identified by researchers in the field and fixed rapidly by Sun and the browser vendors.

Malicious Applets

So much for finding holes in the JVM protection scheme. What about the last two categories of exposure - the things that are allowed by the framework but which can still be annoying or damaging?

Setting the rules for the client environment is always a question of striking a balance. The browser needs to give the applet some system and network resources; otherwise, it will not be useful at all. On the other hand, it must not allow an attacker to have free reign over the browser system.

We have said that there are two types of malicious applets, denial of service and impersonation. There is also another type of malice that is not Java-specific. This is based on deception, that is, to try to trick the user into entering information that they would not normally give away. This sort of thing is not specific to Java, in fact there are much easier ways to do the same thing using scripting languages or simple HTML forms, so we won't consider them further here.

Cycle Stealing

Denial of service attacks have, for a long time, been a scourge of the Internet. Normally you think of them taking down a server or even a whole site. A denial of service applet is unusual in that it normally only affects a single system or user.

"Denial of service" implies that the user can no longer use the system, but we refer here to "cycle stealing" to mean any applet that consumes resources, whether computer or human, without the user's permission. The most extreme form of these are denial of service applets, but the most insidious ones may not be detected by their victim at all.

There are obvious denial of service attacks. For example an applet could try to create an infinite number of windows, or it could sit in a tight loop, using up CPU cycles. These are very annoying and they can have a real impact, if the user has to reboot the machine to recover, for example. However, if they are tied to a particular web page the user will quite quickly realize where the problem is coming from and simply not go there. To be effective takes something that is not so easily traced back to its source.

The key to this kind of applet lies in persistent, background, threads. Every implementation of the Java virtual machine supports threads, and the language makes it very easy to use them. In fact there are two ways to implement a thread, either by creating a subclass of Thread, or by implementing the Runnable interface. The danger of threads lies in the fact that they are not tied to a particular Web page. When you leave the page containing an applet, that applet and all of the threads it has started will normally be terminated. This job is handled by the stop() method, which is a final method in Thread (that is, it cannot be overridden by the programmer). However, if you implement Runnable, you can design the stop() method to do anything you like, including nothing at all. See Never Ending Fortune Cookie Applet (Part 1 of 2) shows an example of this technique.

Never Ending Fortune Cookie Applet (Part 1 of 2)

 

Never Ending Fortune Cookie Applet (Part 2 of 2)

In this case the applet is just mildly annoying, popping up a new window containing a fortune cookie every five minutes (well, OK, after the first dozen times the annoyance could be more than mild). The subtle thing about it is that it seems to appear from nowhere; there is no sign of it on the Web page from which it loads and it does not appear until some time after you have left the page and surfed happily onward. The only way to get rid of it is to quit the browser completely.

This applet is fairly benign, and at least it is visible, so you know that something strange has happened. Really, what has happened here is that the attacker has got free use of machine cycles on your system. What sort of thing might he or she want to do with them? One example would be to do brute force cipher cracking. A feature of any good symmetric key encryption algorithm is a unifor m key space. That is, if you want to crack the code there is no mathematical shortcut to finding the key, you just have to try all possible keys until you find one that works. Several recent encryption challenges have been solved by using spare cycles on a large number of computers working as a loosely-coupled complex, each being delegated a range of keys to try, under the direction of a central coordinator.

This sort of effort depends on the cooperation and goodwill of a lot of people who donate machine time and access. But, if we replaced the getFortuneCookie URL in the above example with, for example, getNextKeyRange, it would be possible to do the same thing without having to ask anybody. A number of other applets along the same lines have been demonstrated, such as applets that kill the threads of other applets executing concurrently.

Summary

The applet security manager enforces a well-defined, secure environment in which to run an applet. In doing so, it places some severe restrictions on what the applet can do, which may impede the development of effective network applications. We show in See Java Gets Out of Its Box how signed applets can break out of these restrictions.

There are some types of undesirable behavior that the sandbox does not prevent. These are generally a nuisance, rather than a serious threat, and at present you have to view them as one of the risks of the Internet. As the concept of protection domains in Java develops, we expect to see more granular controls that will prevent this behavior from the general hacker, while offering a wider range of function to the trusted host.