The Traditional Development Life Cycle

As you have seen earlier, Java is a compiled language. That is, source code is written in a high-level language and then converted through a process of compilation to a machine-level language, the Java bytecode, which then runs on the Java Virtual Machine. Before we look more closely at Java bytecode, we'll quickly review the differences between high- and low-level languages, the compilation process and runtime behavior of a more traditional environment.

On the PC, program files are recognized in two ways. The first is by the file extension (.EXE, .COM) and the second is by the file format itself. Executable files contain some information in a header which informs the operating system that this file is a program and has certain requirements in order to run. These requirements include such things as the address at which the program should be loaded, other supporting files which will be required and so on.

When DOS or Windows attempts to run a program file, it loads the file and ensures that the header is legitimate, that is, that it describes a real program. The header also indicates where the starting point of the program itself is. The program is stored in the program file as machine code instructions. These instructions are numeric values which are read and interpreted by the processor as it executes. Having validated the header, the operating system starts executing the code at this address.

From the above description, it should be clear that anyone with a good understanding of the header format and of the machine code for a particular operating system could construct a program file using little more than an editor capable of producing binary files. (Such an individual would be well advised to seek urgent medical attention.)

Of course this is not how programs are produced. The closest that anyone gets to this is writing assembly code. Assembly language programming is very low level. Its statements, after macro expansion, usually translate into one or at most two machine language instructions. The assembly source code is then fed through an assembler which converts the (almost) human readable code into machine code, generates the appropriate header and finally outputs an executable file.

Most programs, however, are written in a high-level language such as C, C++, COBOL and so forth. Here it is the task of the compiler to translate high-level instructions into low-level machine code in the most optimal way. The resultant machine code output is generally very efficient, although - depending on the compiler - it may be possible to write more efficient assembler language. Because different compilers manage the translation and optimization process in different ways, they will produce different output for the same source code. In general it is true to say that the higher level the source language, the more scope there is for variation in the resultant executable file since there will be more than one possible translation of each high-level statement into low-level machine code.

During the compilation process, high-level features such as variable and function names are replaced by references to addresses in memory and by machine code instructions, which cause the appropriate address to be accessed (in the case of variables) or jumped to (in the case of functions).

In the case of both assembly language and high-level language programming, the output of the assembly or compilation phase is generally not immediately executable. Instead, an intermediate file (known as an object module or object file 1 ) is produced. One object file is produced for each source file compiled, regardless of the content or structure of the source code. These object modules are then combined using a tool called a linker which is responsible for producing the final executable file (or shared library). The linker ensures that references to a function or variable in one object module from another object module are correctly resolved.

 

Program Compilation and Linking

In summary then:

• An object file contains the machine code which is the actual program plus some additional information describing any dependencies on other object files.
• An executable file is a collection of object files with all inter-file dependencies resolved, together with some header information which identifies the file as executable.

The Java Development Life Cycle

Moving back to the world of Java, we see that it is a high-level programming language and that bytecode is the low-level machine language of the Java Virtual Machine. Java is an object-oriented language; that is, it deals primarily with objects and their interrelationships. Objects are best thought of in this context as a collection of data (fields in Java parlance) and the functions (methods) which operate on that data. Objects are created at runtime based on templates (classes) defined by the programmer.

A Java source file may contain definitions for one or more classes. During compilation each of these classes results in the generation of a single class file. In some respects, the class file is the Java equivalent of an object module rather than an executable program file; that is, it contains compiled machine code, but may also contain references to methods and fields which exist in other classes and hence in other class files.

Class files are the last stage of the development process in Java. There is no separate link phase as linking is performed at runtime by the Java Virtual Machine. If a reference is found within one class file to another, then the JVM loads the referenced class file and resolves the references as needed.

The astute reader will deduce that this demand loading and linking requires the class file to contain information about other class files, methods and fields which it references, and in particular, the names of these files, fields and methods. This is in fact the case as we shall see in the next section.

Even more astute readers may be pondering some of the following questions.

• Is it possible to compile Java source code to some machine language other than that of the JVM?
• Is it possible to compile some other high-level language to bytecode for the JVM?
• Is there such a thing as an assembler for Java?
• What is the relationship between the Java language and bytecode?

The simple answer to the first three questions is yes.

It is possible with the appropriate compiler (generally referred to as a native code compiler) to translate Java source code to any other low- level machine code, although this rather defeats the "write once run anywhere" proposition for Java programs since the resultant executable program will only run on the platform for which it has been compiled.

It is also possible to compile other high-level languages into Java bytecode, possibly via an interim step in which the source code is translated into Java source code which is in turn compiled. Bytecode compilers already exist for Ada, COBOL, BASIC and NetREXX (a dialect of the popular REXX programming language).

Finally, Jasmin is a freely available Java assembler which allows serious geeks to write Java code at a level one step removed from bytecode.

 

Compiler Models

The Java Class File Format

The class file contains a lot more information than its cousin, the executable file. Of course, it still contains the same type of information, program requirements, an identifier indicating that this is a program and executable code (bytecode) but it also contains some very rich information about the original source code.

The high level structure of a class file is shown in See Class File Contents .

Decompilation Attacks

One of the areas seldom discussed when considering security implications of deploying Java is that of securing Java assets. Often considerable effort is put into developing software and the resultant intellectual property can be very valuable to a company.

Hackers are a clever (if misguided) bunch and there are many reasons why they might want to get "inside" your code. Here are a few:

• To steal a valuable algorithm for use in their own code
• To understand how a security function works to enable them to bypass it
• To extract confidential information (such as hard-coded passwords and keys)
• To enable them to alter the code so that it behaves in a malicious way (such as installing Trojan horses or viruses)
• To demonstrate their prowess
• For their entertainment (much as other people might solve crosswords)

The chief tool in the arsenal of the hacker in these cases is the decompiler. A decompiler, as its name suggests, undoes the work performed by a compiler. That is, it takes an executable file and attempts to re-create the original source code.

Advances in compiler technology now make it effectively impossible to go from machine code to a high-level language such as C. Modern compilers remove all variable and function names, move code about to optimize its execution profile and, as was discussed previously, there are many possible ways to translate a high-level statement into a low- level machine code representation. For a decompiler to produce the original source code is impossible without a lot of additional information which simply isn't shipped in an executable file.

It is, however, very easy to recover an assembly language version of the program. On the other hand, the amount of effort required to actually understand what such a program is doing makes it far less worthwhile to the hacker to do. (Nevertheless, it is done. Much pirated software is distributed in a "cracked" format, that is, with software protection disabled or removed.) So, it is fair to say that it is impossible to completely protect any program from tampering.

When JDK 1.02 was shipped, a decompiler named Mocha was quickly available which performed excellently. It was able to recover Java source code from a class file. It was so successful that at least one person used it as a way of formatting his source code! In fact the only information lost in the compilation process and unrecoverable using Mocha are the comments. If meaningful variable names are used (such as "accountNumber", or "password") then it is readily possible to understand the function of the code, even without the comments.

The current version of Mocha is unable to decompile Java 1.1 class files but this is not because the class files contain any less information, merely because the format has changed slightly. It is only a matter of time before a functional decompiler for Java 1.1 class files is developed.

The Constant Pool

We said earlier that the constant pool contained a great deal of information. In fact it contains a strange mixture of items. The constant pool combines the function of a symbol table for linking purposes as well as a repository for constant values and string literals present in the source code. It may be considered as an array of heterogeneous data types which are referenced by index number from other sections of the class file such as the Field and Method sections. In addition, many Java bytecode instructions take as arguments numbers which are in turn used as indexes into the constant pool. See Constant Pool Entry Types shows the types of entries in the constant pool, as defined by the current JVM.

Beating the Decompilation Threat

The very real threat of decompilation is not going to go away. Decompilers work by recognizing patterns in the generated bytecode which can be translated back into Java source code statements. The field and method names required to make this source code more readable are readily available in the constant pool as we have seen.

To date, there have been two main approaches to thwarting would-be decompilers, code obfuscation and bytecode hosing. 3

The principle of obscuring (or obfuscating) source code to make it more difficult to read is not new. In the UNIX world - where incompatibilities between platforms and implementations make it necessary to distribute many applications in source format - "shrouding" is common. This is the process of replacing variable names with meaningless symbols, removing comments and white space and generally leaving as little human readable content in the source code without impacting its compilability.

After the release of Mocha, the author released Crema, a further appalling coffee pun, which was designed to thwart Mocha. It did this by replacing names in the constant pool with illegal Java variable names and reserved words (such as "if" and "class"). This had no affect on the JVM, which merely used the names as tags to resolve references without attributing any meaning to them. Nor did it actually prevent decompilation. It did however mean that the decompiled code was more difficult to read and understand and also would not recompile as the Java compiler would object to the illegal names.

Bytecode hosing is more subtle and is aimed at preventing the decompiler from recognizing patterns within the bytecode from which it could recover valid source. It does this by breaking up recognizable patterns of bytecodes with "do nothing" instruction sequences (such as the NOP code or a PUSH followed by a POP). A good example of a bytecode hoser is HoseMocha.

Of course, this approach can be defeated since once a hacker has established what types of do-nothing sequences are being generated by a bytecode hoser, he or she can modify the behavior of the decompiler to ignore such sequences. Furthermore, attempts to decompile hosed bytecode will generally result in broadly readable code interspersed with unintelligible passages rather than completely unreadable code.

In addition to this, bytecode hosers present a more insidious problem to Java users. As we have already seen, the principal method of optimizing Java performance is in the JVM and in particular through the use of just in time (or JIT) compilation. And how do JITs work ? Yup, you guessed it, they recognize patterns in the generated bytecode which can be optimized into native code. Breaking up these patterns through the use of a bytecode hoser can seriously impact the performance of JIT compilers.

For this reason, it is safe to assume that Java compilers will not follow the same evolutionary path as their native compiler cousins in terms of generating wildly differing output for the same source code since this too would thwart JIT compilers.

This is a well understood dilemma in security circles, the trade off between security and performance/price/ease of use.

The only safe course of action is to assume that ALL Java code will at some point be decompiled.

For developers this means ensuring that no sensitive information is distributed in the class file either algorithmically or as hard-coded values. This can be accomplished by building client/server type applications with a Java presentation layer which can be run anywhere and a secured server side where sensitive information or algorithms can be stored. This may also involve extending the development and testing process to ensure that distributed Java code is "safe".

Finally you may decide that the existing method of protecting distributed code, that of legal sanction under copyright laws, is sufficient to deal with any serious threat to Java-based intellectual property...in which case we have some real estate you may be interested in buying.

Java Bytecode

In the next chapter we look at how the Java class loader and class file verifier provide a level of security against rogue class files. This section prepares us for that chapter by looking more closely at bytecode.

A Bytecode Example

Though you may not realize it, you have already seen an example of bytecode or at least the human readable format. The output generated by the javap command when we ran it with the -c flag contained a disassembly of each of the methods in the class file. The code snippet in See Decompiled Method (Part 1 of 2) was taken from the actionPerformed method of our pointlessButton class. It was compiled from three lines of Java source code:

public void actionPerformed(java.awt.event.ActionEvent e) {

donowt.setLabel( "Did Nothing " + ++count + " time" + ( count == 1? "": "s" ) );

}

Decompiled Method (Part 1 of 2)