The end of buffer overflows

[bascule]

While many C/C++ zealots shun garbage collection as a leaky abstraction, advocates of languages such as Java and Lisp often like to describe languages which allow for manual memory management as "unsafe". This is because most operating systems do not mark pages with executable data as read only in a process context, and writable pages as non-executable, primarily because the IA32 architecture provided no mechanism for this in protected mode. Consequently anyone who can overwrite any part of the memory with CPU instructions and knows that the instruction pointer will at some point fall in that range, or can overwrite the address used by a jump instruction with the location of their own instructions (such as, say, the return pointer of a stack frame) can execute arbitrary code.

Linux has taken a step towards implementing a security feature which many architectures, most notably Solaris/SPARC, have had for years, a non-executable user stack, on the new AMD64 architecture which sports page permissions. You can read about it here:

http://kerneltrap.org/node/view/3240

What they've implemented is typically described as a "non-executable user stack", namely that if the instruction pointer points at a page which does not have an execute bit, the CPU generates an exception which can be handled in the same way by the kernel as an attempt to access protected memory in process context. Any processes attempting to execute data in the user stack will now die with "Segmentation fault" rather than being exploitable.

Of course, this is only half of the solution as executable pages can still be modified in process context. This is why even with Solaris's noexec_user_stack enabled for sparcv8 binaries (and it's the default behavior of sparcv9 binaries) we still see buffer overflows on Solaris... if someone is able to maliciously write past a page boundary into an executable page they can still exploit a buffer overflow.

Let's hope Linux takes the extra step and implements executable pages which are read-only in process context. This will likely require some additional system calls for proper dynamic linker thunking, but that can all be abstracted in the kernel and through libdl.

With Intel's adoption of both the AMD64 instruction set and its page permission features, I hope we see this silver bullet to buffer overflows applied to Linux and other architectures in the next few years.

But suffice it to say the end of buffer overflows is here, and the answer needn't necessarily be managed code

[TheCotMan]

Solar Designer has been providing Nonexecutable stack patches as well as a slew of other useful kernel patches since the days of Linux 2.0 or was it 2.2? http://www.openwall.com/

Also, I think the GRsecurity patches have provided much the same.

As I understand it Dead Rat (ReadHat) was planning to make nonexecutable stack space a feature in the Enterprise version.

It is good to see this being taken into the mainstream.

You should know (however) that elimination of executable stack space is not 100%. There is a marvelous thread on bugtraq from several years ago where SD and other argued about risks associated with library based exploits if you know something about the remote system being targetted.

I have been using several security patches (including non-executable stack features) to my Linux boxes for about 5 or 6 years (maybe longer) but as good as they can be, they have been slow to become popular.

Garbage collection is a very nice thing, but there is an overhead of accounting, and can be issues with reduction in speed. This is why languages which did not offer it seemed to do better. (Well, one reason.) Their resulting code (though riskier for problems) could run faster (in many cases) than code for the same purpose as built on other systems.

[Edit: that last line sucks at conveying what it was meant to convey: added content]
[STUPID WORDS! UG! KILL WORDS!]

A program written in a language which has garbage collection tends to run slower than the same
effective program written in C and sometimes C++, but this loss of garbage collection often leads to many of the security problems in these programs. However, the commercial world (historically) has not been as impressed with security as it has with: 1) features, and 2) performance. This is one of many reasons why C/C++ have done better than other languages which actually seem to be more advanced in many ways.

[bascule]

Solar Desiner has been providing Nonexecutable stack patches as well as a slew of other useful kernel patches since the days of Linux 2.0 or was it 2.2? http://www.openwall.com/

The advantage of using page permissions is that it's implemented at the hardware level, making it literally impossible to exploit while in process context (unless there's an implementation flaw in the CPU) as opposed to entirely software based methods which still provide an opportunity for circumvention.

You should know (however) that elimination of executable stack space is not 100%. There is a marvelous thread on bugtraq from several years ago where SD and other agured about risks associated with library based exploits if you know something about the remote system being targetted.

As I mentioned in my post, a non-executable user stack is only half the solution. Executable pages must also be marked read only to prevent the exploitation of buffer overflows... otherwise anything that can write past page boundaries can still write into executable memory.

Garbage collection is a very nice thing, but there is an overhead of accounting, and can be issues with reduction in speed. This is why languages which did not offer it seemed to do better. (Well, one reason.) Their resulting code (though riskier for problems) could run faster (in many cases) than code for the same purpose as built on other systems.

Well, actually, since garbage collection can provide profiling and memory pooling, it can actually gain some performance by "intelligently" managing the heap, versus a typical malloc()/realloc()/free() approach which must implement a generalized heap management algorithm. While some operating systems are quite fast with heap management due to tight libc/VMM integration (FreeBSD, Solaris) others have been plagued by performance problems (namely glibc). Regardless, any general purpose heap will suffer from fragmentation issues, and the heap can often exhibit similar "stalling" problems to garbage collected languages, especially in a multithreaded environment. The solution is proactive usage of resource pooling, which "wastes" resources in a similar manner to a garbage collected environment, but can greatly improve performance. With manual management though, debugging becomes more complicated (as you have to use a memory debugger tool like ElectricFence, GlowCode, or valgrind), and thus development times increase. The saving in developer time is what's largely touted by advocates of higher level languages, and there I can't disagree, but I'd say in general good programs developed in native code languages with manual memory management tend to be superior to similar programs which either provide garbage collection and compile to native code (Lisp) or execute inside a runtime (Java, Python, C#/VB.NET/etc)

[TheCotMan]

The advantage of using page permissions is that it's implemented at the hardware level...

(Forgot to mention this in my other post, but will include it here:)
Hardware support for this is a very cool thing indeed.

And this rocks! As a result, we will be able to see elimination of code-based workarounds and emulation of trampolines which can cause some programs to not work under these patches. (For example, we found versions of "at" did not work well on early Linux systems, but cron worked just fine.)

As I mentioned in my post, a non-executable user stack is only half the solution....

Yep, sorry about that. I was commenting on the software-based patches from SD not the inclusion of the hardware support in CPU and the suggestion you made in your post.

Well, actually, since garbage collection can provide profiling and memory pooling, it can actually gain some performance by "intelligently" managing the heap, versus a typical malloc()/realloc()/free() approach which must implement a generalized heap management algorithm... {Excellent content clipped here}

I can't really argue with you here based on the present state of OS. What you have is true in many cases (obviously exceptions exist for certain applications, but you outline a trend which is an emergent behavior in modern OS and tools.)

I can only bring up the history of events which lead to the rise of C++ in the early years. Even though we have systems which can take advantage of heap and memory pools, back in the old days when C and LISP were still young, such complex support for management of these was not included due to size limits of addressable memory and research. As a result, these early machines limits caused differences between products of languages with garbage collection vs. languages without garbage collection (C) to be even more obvious. At such an early stage, this became a significant difference and bone of contention from people doing System Engineering.

Once such a trend develops at an early stage it can be very difficult to generate enough attraction to a new thing to break the coefficient of static friction holding people to a product with which they are familiar. O'Reilly (owner of O'Reilly and associates) gave a nice lecture for a group where he stated he believed a new product needs to be 10 times better than the existing product in order to take over the market and convince people to switch. He actually stated that he did not think Linux was 10 times better than Windows (yet) and would not take much in the way of market share from Windows, but *would* take market share from other *NIX systems. (This was maybe 7 years ago.)

Side comment:
Thanks much for your original post and reply in this thread... High quality content. :-)