Protection Against Buffer Overflow Attack in .NET


This paper attempts to explain one of the critical buffer overflow vulnerabilities and its detection approaches that checks the referenced buffers at run time, moreover suggesting other protection mechanics to be applied during software deployment configuration. Programs typically written in the C or C++ language are inherently susceptible to buffer overflow attacks in which methods are often passed pointers or arrays as parameters without any indication of their size and such malpractice can be exploited later. Buffer overflow remains one of the most critical threats to systems security, especially for deployed software. Successful mistreatment of a buffer overflow attack often leads in arbitrary code execution in so called shell code and thorough control of the vulnerable application in a vicious manner.


We shall showcase buffer overflow vulnerability in the Windows environment via C++ or VC++ code that is typically written via Visual Studio 2010 or Turbo C++. Moreover, it is expected from researchers having a comprehensive understanding of C++ syntax and concepts, especially pointers and arrays by creating a Win32 console application.

                     Turbo C++ compiler
                     VC++ .NET
                     GCC Compiler (optional)

Buffer Overflow Bug Demo

An overflow typically happens when something is filled beyond its capacity. So, buffer overrun attacks obviously occur in any program execution that allows input to written beyond the end of an assigned buffer (memory block). Thus, it leads the data to overwrite into an adjacent memory location that are already occupied to some existing code instruction. In buffer overflow attacks, the hackers encroach the preoccupied memory segments for other operation instruction sets, to inject malicious arbitrary code and the pre-determined program behavior is changed eventually. These buffer overflows are the implication of poor programming practice by not putting any boundaries on the size of the input the program can handle. C and C++ programmed code are the great source to produce buffer overflow attacks because these languages allow direct access to application memory.

Sometimes hackers find other ways to exploit overflow besides getting their code to run. Certain overflows do not actually allow hackers to take control, but might instead allow them to manipulate extra data. Let's examine the following bofVul.exe login console base program that accepts user name and password at the command-line to validate users. If they enter a correct username and password, it allows access otherwise access is denied as in the following;


This program was running perfectly until now, but now imagine, if a person with vicious intention enters the parameter in the following form. He is trying to overflow the buffer by entering some garbage values and finally notices that we successfully penetrated the program even without having the correct user name and password. Bingo! It is even revealing the welcome message that shall be flashed when the user enters the correct credentials. So, this is a bit strange, how is this possible. We have just entered a sequence of raw data in spite of the password and successfully obtained access.

overflow bug demo

Using a different password with the same user id still worked! So it is a clear case of a buffer overflow bug because the strange behavior of the program allows you to log on if you specify a long password, regardless of whether the password is correct.

Buffer Overrun Internal

Buffer overflows is one of the costliest security vulnerabilities known to affect computer software. When input is larger than the space allocated for it, but it is written there anyhow and the memory is overwritten outside the allocated location. In some cases, overflows result from incorrect handling of a mathematical operation or attempts to use memory after the memory has already been allocated. Although many overflows occurs when the program receives more data than it expects, in fact there are many kinds of overflows. It is important to distinguish among various classes of overflows to be able to develop good test cases to identify specific types of overflows.

  • Integer overflow: Occurs when a specific data type of a CPU register that is meant to hold values within a certain range is to be assigned a value outside that range. Integer overflow often leads to buffer overflow for cases in which integer overflow occurs when computing the size of the memory to allocate.

  • Stack Overflows: such overflows occur when data is written past the end of buffers allocated on the stack.

  • Heap Overflow: It occurs when data is written outside the space that was allocated for it on the heap.

  • Format String Attacks: Format String attacks occur when the %n parameter of the format string is used to write data outside the target buffer.

It is important to delve deep into CPU internal infrastructure by examine various registers that play a significant role during memory allocation.

  • EIP [Extended Instructor Pointer]: It is only administrated by the CPU and determines the next machine instruction in memory to be executed. They contain the offsets of data and instructions.

  • ESP [Extended Stack Pointer]: It points to the top of the stack for the CPU to do push and pop operations.

  • EBP [Extended Base Pointer]: It is used to as reference memory for indirect addressing.

  • EAX/EBX/ECX/EDX: They are used for arithmetic and data movement.

  • Segments [CS/DS/SS/FS/ES/GS]: They are used as a base location for program data, instructions and the stack.

If a method is called by assembler "call" commands, a new stackframe is created and the boundary is defined by the EBP and ESP. First, the call instruction pushes the EIP onto the stack. The previous ESP becomes the new EBP and then the space for the variables is allocated by subtracting its size from the earlier ESP. Finally, at the end of the function call, the ESP becomes the new EBP.

stack frame

Now, let's consider one more buffer overflow sample that is developed under VC++ Studio. Here the user name and password is supplied as a command-line argument that is copied into corresponding fixed length array of character variable using a strcpy method. Later, the supplied credentials is validated against a predefined password via the strcmp method as in the following;

  1. #define BUFF_SIZE 10  
  3. void creed(char *usr,char *password)  
  4. {  
  5.    char uN[10];  
  6.    char pass[10];  
  8.    strcpy(uN, usr);   
  9.    strcpy(pass, password);   
  11.    if(strcmp(pass,"ajay"))  
  12.    {  
  13.         printf ("\n Access Denied \n");  
  14.    }  
  15.    else  
  16.    {  
  17.          printf ("\n Welcome:");  
  18.                ..  
  19.    }  
  21. }  
  22. int main(int argc, char* argv[])  
  23. {  
  24.     ..  
  25.     creed(argv[1],argv[2]);  
  26.     return 0;  
  27. }  
The moment the user enters tom as a user name and ajay as a password via command-line argument, this program validates successfully those credentials and allows access as in the following:

moment user

Now try to enter some bogus data as credentials. As assumed, the program won't allow us to get access as in the following. At this movement, everything is running fine and under control.


The character variable uN and pass can hold up to only 10 characters and if we input data beyond this fixed length, since we are not doing any bounds checking, we are just directly copying the entered data into the buffer directly via the strcpy method. The program would be confused and can't handle such abundant data. That leads to buffer overflow as in the following:

demo exe

Since we are testing this program in the Windows environment, the OS throws the exception that eventually causes the application to crash because the program accepts too much data beyond the limit of 10. In case of compiling this program via the Turbo compiler, it notifies the buffer overflow exception in a different manner as in the following;

execute status

When executing the aforesaid code, it first pushes the two arguments (user name and password) to the creed() method backwards onto the stack. It then calls the creed() function. The instruction CALL then pushes the instruction pointer (EIP) onto the stack. The creed() function now pushes the stack frame pointer onto the stack. The current stack pointer (ESP) is then copied into the EBP, making it the new frame pointer (SFP) as in the following;


Now, the creed() function instruction's next instruction address 0x00412206 is saved to the stack and execution jumps to the ebp in the creed() instruction code where the user name and password values are copied into the eax that is pushed onto the stack. Finally on behalf of both strcpy offsets the strcmp instruction is executed.

program code

Thereafter, the ret opcode is executed that points to the end of program instructions. If the parameter is entered in the correct form or lesser than the fixed length, the program doesn't show any abnormal behavior. But since we are passing an argument beyond the limit, here the register EBP value 79797979 is examined that become the ESP now as in the following;

buffer overflow

As we move ahead, the execution should be jump to 00412209 instead of 0079797979. Hence, Visual Studio throws a run time exception at the 79797979 offset where the program denied reading the address space at 79797979 locations. So, the program crashes because here the execution is halted due to an access violation and a buffer overflow attack occurs as in the following:

buffer overflow attacks

Protection Mechanisms

Buffer overrun attacks can be thwarted in the Windows environment by making critical configuration changes. The Visual Studio C++ compiler offers several options to enable certain checks at runtime such as /GS, RTC, Runtime library check and DEP. These options can be enabled using a specific compiler flag. The /GS option shields against vulnerable parameters passed into a function in the form of a pointer, string buffer, or C++ reference. Normally, the incoming method parameters are assigned on the stack and are susceptible to being overwritten, just like the return address. To avoid this situation, the compiler makes a copy of the vulnerable incoming parameters after the storage for the local buffers where they are not in threat of being overwritten. On the other side, the RTC compiler option control specifies that at run-time such things as underflow and overflow are checked and stack verification is done and variable use without initialization is detected. However, these run-time checks introduce a performance overhead that is not acceptable for release builds. We must to enable these compiler checks at least.

                              Buffer Security check (/GS)
                              Runtime Library check (Both /RTC1…)
                              Basic Runtime checks (Enable VC++ Run time Library)

buffer security check

Visual Studio also provides the Data Execution Prevention (DEP) option during compilation in case of not disabling it at the operating system level.

data execution prevention

There is another important feature called Data Execution Prevention (DEP) to protect from buffer overflow attacks. This feature has been available in Windows and assumes that no code is intended to be executed that is not part of the program itself. It uses NX technology to prevent the execution of instructions stored in data segments. This feature requires administrative rights to changes its setting. We can alter this configuration from the command prompt as in the following.

To disable the Data Execution Protection setting:

                                                      bcdedit.exe /set {current} nx AlwaysOff

To enable the Data Execution Protection setting:

                                                      bcdedit.exe /set {current} nx AlwaysOn

We can enable this setting from My Computer advanced settings under the Performance option. These options are disabled by default. In order to enable them, log-in via Administrative account as in the following:

data execution prevention(DEP)

After finishing with all the necessary configuration or BOF attack thwarting option enabling, run the program and supply some bogus argument beyond the buffer limit, the operating system issues a run time buffer overflow exception as in the following:

debug liberary

Even though the /GS compiler option aborts the program, the overrun should be fixed. Buffer overflow attacks can be avoided at the time of coding by ensuring that input data does not exceed the size of the fixed length buffer that it stores. Here, the fixed length buffer size is 10, so calculate the entered data length and ensure it is lesser than 10 as in the following:
  1. #define BUFF_SIZE 10  
  3. void creed(char *usr, char *password)  
  4. {  
  5.       ..  
  6.       if (strlen(password)<BUFF_SIZE)  
  7.       {  
  8.             strcpy(uN, usr);   
  9.             strcpy(pass, password);   
  10.       }  
  11.       else  
  12.       {  
  13.             printf ("\n Program doesn't support this password \n");  
  14.             exit(1);  
  15.       }  
  16.       ...  
  17. }  
  18. int main(int argc, char* argv[])  
  19. {  
  20.       ..  
  21.       creed(argv[1],argv[2]);  
  22.       return 0;  
  23. }  
Now the buffer overflow attack can be thwarted even if the other protection, such GS and DEP is not applied in the solution configuration. Here, the program alters and exits if the data is entered beyond the buffer limit as in the following:

password status

As we have stated earlier, C and C++ source is most vulnerable to buffer overrun attacks. I will pinpoint some C library methods that make you vulnerable. Hence, it is recommended to avoid using these methods in your source code.


Functions Potential Problem
Strcpy(char *str, const char * str2) Str buffer could be overflow
Gets(char *arr) arr buffer could be overflow
Getwd(char *arr) arr buffer could be overflow
Scanf() Arguments can be overflow
Fscanf() Arguments can be overflow
Sprint(char * str,const char *str2) Str buffer could be overflow
Strcat(char * str, const char * str2) Str buffer could be overflow

Final Note

This article explained how buffer overflows works, the varieties of overflows that can materialize and ways to control the flow of execution to our arbitrary code. We have also covered various forms of prevention mechanisms that can be taken to thwart buffer overrun attacks. Memory management and CPU registers have also been covered giving us the elementary knowledge indispensable to detect and exploit buffer overflow vulnerability. We looked into actual exploits on how they were written and where the control on the flow of execution took place. Understanding all these sections will aid us in the future when it comes to analyses, debugging and exploitiing the buffer overflow vulnerability.

Similar Articles