Buffer Overflow Attacks: The Theory

The year is 1988. Die Hard has been released, Rick Astley made a song, Rihanna was born. And Robert Morris a young undergrad student at Cornell University nearly takes down the internet.

The Morris Worm as it became known (or the ‘Great Worm’) relied on several exploits in Unix based machines. One of these exploits was a buffer overflow that allowed the worm to break into the machines and cause chaos. In this mini series on buffer overflows we’ll look at the theory, the practice and the defences against them.

When are buffer overflows a problem?

Avoided mostly in memory managed languages like Java

In C or C++ you have arrays (i.e. buffers). If data is written into a buffer that exceeds its size, an overflow occurs.

A quick look at program memory:

  • Memory is stored in a virtual address space from 0x00000000 at the bottom to 0xFFFFFFFF at the top (32 bit)
  • We are interested in the stack where the data for a function call is stored i.e. it’s parameters, return address and local variables.
  • Each function call happens in a stack frame
  • The stack pointer (esp) points to the last thing that was added to the stack (the top of the stack)
  • The instruction pointer (also called the program counter or EIP) points to the last instruction executed in a program.
  • Note that stack (somewhat counterintuitively) grow downwards i.e.you add (push) things onto the stack and the address of the stack pointer goes down.
  • Conversely the heap grows up. This allows there to be one section of memory for both the stack and heap and when the stack pointer and the heap pointer variables cross each other, we know that the block of memory is full.

A very basic function that just allocates space for two buffers and takes in three parameters but does nothing with them and just returns.

void function(int a, int b, int c)
{
    char buffer1[5];
    char buffer2[10];
}

void main()
{
    function(1,2,3);
}

This is just the stack laid on it’s side.

Working from right to left:

  • We start off in main which makes a call to our function.

Function prologue:

  • We then push onto the stack our three parameters in reverse order a, b, and c and then set the ebp (extended base pointer) at the ‘base’ of the function. This is so the address of the parameters and the local variables can be found in relation to the base of that function.
  • Then the return address is set, which is the address the EIP will return to when our function is finished
  • Then the sfp (save frame pointer) is set to the ebp of the main function which we will return to. Remember this is just so we have a ‘base’ where local variables and paramters can be calculated from.
  • We then execute our function. Buffer 1 and buffer 2 are pushed on.

Function epilogue:

  • When the code is run we return to main.
  • Get rid of our buffers and move stack pointer to stack frame of main and read sfp and use that to return our ebp back to where it should be in main.
  • Finally use the return address to work out where we need to return to and continue the flow of execution.

A buffer overflow occurs when we override one of these buffers and overwrite some of the values in those other variables.

How this might work in practice:

Although we will look at exactly how to perform a buffer overflow in part 2 of this series, this mini-example will give us some further insight.

All the function below does is allocate a 128 byte character buffer and copy the sting that is passed in, into the buffer. This is perfectly normal.

void function(char *str)
{
    char buffer[128];
    strcpy(buffer, str);
}

The problem here is that strcpy only stops when it hits a null character. However as there isn’t a null in this string and there’s no bounds checking in the code, if we give the function a string that is bigger than the size of the buffer we can overwrite the return address.

When we finish executing this function we will read the return address. If we can align things correctly then we can get the return address to return to a virus.