Half-baked idea: Libraries should run in their own security context

For more half-baked ideas, see the ideas tag.

When your program links to a library, the program and the library run in the same process and so have the same SELinux and other security context (UID, resource limits, etc).

Why can’t libraries have their own context? This would protect your program from bugs in libraries, and allow you to control more precisely what the library is doing, eg. what files it is trying to access.

I originally thought this would be simple: just have the kernel set the context according the userland instruction pointer (ie. %rip). Program code in certain pages would therefore have a different context when calling into the kernel from library code in other pages. You might also have to be careful with system calls that read and write user data. Anyway, this doesn’t quite work because a library could bypass this by changing the memory protection on some code in another part of the program, overwrite that code, then jump to it. Perhaps if the library’s security context prevented it also from calling mprotect etc it could be made to work? Edit: no it can’t be made to work, see the comments.

Another way to do it would be to (behind the scenes) run the library really in a separate process, turning local calls into RPCs. This is also not totally trivial if you consider global variables (shared memory?) and callbacks, and the performance implications could sink this idea too.

In any case the important thing would be to make any technique as transparent to the programmer as possible, so that programmers don’t have to explicitly split programs up into separate communicating processes in order to gain the security advantages. If no code changes were needed, we could apply the technique to existing programs in order to harden Linux still further.


Filed under Uncategorized

9 responses to “Half-baked idea: Libraries should run in their own security context

  1. Ethan

    You might be interested in Robusta (http://www.cse.lehigh.edu/~gtan/paper/robusta.pdf) which proposes to sandbox native code libraries for java programs.

  2. Perry Lorier

    The problem with this, as I see it, is if I know that you have a syscall instruction that does something I want, I can setup the arguments as I want it, then jump to that instruction no matter where it is in memory and it’ll be trusted as if it came from you.

    Which is a pity, because this would be a nice way to deal with the problem🙂

    • rich

      Yes, and another problem is that if the library can write to any data structure (eg. one owned by another library or the main program) then it can modify that data structure to be invalid, causing at least an assertion failure, and at worst some incorrect code path to be taken.

      To avoid that you need to have memory protection, which implies either some sort of weird context switching in a process or (since that would never fly upstream) a separate process.

      It looks like the second method — running the library as a separate process — is the only possible way to do this.

  3. DDD

    Yay! Out of process COM servers🙂

    • rich

      Yeah, well, indeed. Doesn’t Windows still implement DLLs like this? I notice that Windows DLLs can’t seem to use global variables, and each public function needs special linkage.

      • DDD

        The original COM concept was sound, if a bit heavyweight, IMO – Create a formalized OO infrastructure for C/C++. Basically what everyone reinvents when they write “python/perl/Java bindings for $CLASSLIBRARY”

  4. Bill Davidsen

    In MULTICS the libraries ran in another processor ring, at least on x86 that’s not impossible today. However, it’s not clear that it solves your implied criteria for behavior. I’m not sure that what I think you want can be done efficiently in software alone.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s