I posted the following to sbcl-devel on Oct 31, 2007:

Since it seems to me that some interesting optimizations are held back by not having out-of-band unwind info, I started looking into the x86-64 ABI-standard unwind info format: DWARF. It seems to me that it's a good idea to generate unwind info in the standard platform way, given that there is a standard. If this is made to work, in addition to getting an unwinder for free, C++ and lisp exceptions may even be able to interoperate. (e.g. throw a C++ exception, catch it in lisp, and v.v.).

As far as I can tell, the same unwind info format is actually used on all gcc-supported platforms except ia64 and arm (and it seems like it's pretty similar on those, just with some different details). And, of course, windows will as usual have to be done some completely different way.

So, I've made a (pretty rough) proof of concept for generating the unwind info. This proof of concept works on linux/x86-64. 

http://fuhm.net/software/sbcl-eh-frame/

A demo:

* (load "t")

Now, first, asking for a backtrace from glibc's "backtrace" function shows only the topmost frame (this address is in the function "glibc-backtrace"). How sad. I'd really like it to go further.
* (do-backtrace)
[0x1002f299bb]

So, I register some DWARF3 unwind-info I wrote for the glibc-backtrace function, by calling:
* (register-glibc-backtrace-unwind-info)

The unwind-info I created is pretty much totally bogus, and doesn't describe anything like the actual stack/register layout of the function. However, it does describe the location of the return address at the point at which a call to glibc's backtrace function occurs. So, it sufficies for this demo.

Now, if I get a backtrace again:
* (do-backtrace)
[0x100320bd23]
[0x100320c26f]

TA-DA! It shows another frame. 



Now, that's all well and good, but I've really only done the easy part. There's some things left to do:

1) Teach the compiler how to generate the unwind info. This should be such a piece of cake, it hardly deserves further mention. *grin*

2) Figure out how to work around the problem of the unwind-info registry not being able to handle large numbers of registrations.

The problem here is that the lookup function does a linear search over each __register_frame registration. So, the more registrations, the slower an unwind will become. However, each registration can contain an unlimited number of Frame Description Entries. And *those* are searched with a binary search.

So, one trick is to combine multiple functions's FDEs into one registration. (that's how this system is designed to work: each shared library is one registration containing a bunch of FDEs). But there's a few issues I can see with actually accomplishing that. First off, registrations cannot overlap. So, to do one registration containing FDEs for a bunch of functions, those functions' code really must be contiguous (or else you can never put other functions between them). 

And then, what about if a function's code moves (or is GC'd away). Presumably you'd want to call __deregister_frame before moving it, and __register_frame afterwards. But if one registration is shared amongst many functions, you can't deregister one without deregistering them all. And calling __[de]register_frame at all at GC time seems iffy, due to a mutex used in both lookup and registration. Ugh.

Alternatively, SBCL could override the _Unwind_Find_FDE function to provide its own lookup mechanism. This seems like it'd probably work, at least for linux/ELF, where an app can override symbols in a shared library.

Whatever solution is found, it's probably a good idea to bring up the issue of registering unwind info for dynamically generated functions on the gcc mailing list, to get this fixed in a better way for future versions.

3) Modify SBCL to take advantage of having out-of-band unwind info available. E.g. improve the calling convention, make unwind-protect free in the no-throw case, etc. It might also be possible to make the GC fully precise (although you'd want to be careful that a foreign function without unwind-info (and thus causing an unwind failure) doesn't cause the values on the stack to be ignored).

Anyhow, it does seem to me as if this could work. Unfortunately, I don't have the time to push this beyond proof of concept stage. Anyone else interested in running with this?

James