experiment: optionalising shared libraries without dl_open via generating stub libraries
Reading the discussion on debian-devel about shared library dependencies in C/C++, I wondered if it's possible to link with a shared library without having an absolute dependency on it.
The issue comes often when one has a piece of software which could use extra functionality the shared library (let's call it biglib) provides, but the developer/maintainer doesn't want to force all users to install it. The common solution seems to be either:
- defining a plugin interface and a bridge library (better detailed at here);
- dlopen/dlsym to load each library symbol by hand (good luck doing that for high-level C++ libraries).
Both solutions involve dlopen at one stage or another to avoid linking with a biglib, since if you do that, the application loader will refuse to load the application unless biglib and all of its dependencies are present. But what if application is prepared to fallback at run time, and just needs a way to be able to start without biglib?
I went ahead to check what if there was a stub library to provide all symbols which the application uses (directly or indirectly) out of biglib. Turns out that yes, at least for simple cases it seems to work. I've published my experimental stub generator at https://github.com/jackyf/so-stub .
For practical use, we'd also need a way to tell the loader that a stub library has to be loaded if the real library is not found. While there is many ways to instruct the loader to load something instead of system libraries (LD_PRELOAD, LD_LIBRARY_PATH, runpath, rpath), I found no way to load something if a system library was not found. In other words, I'd like to say "dear linker/loader, when you're loading myapp: if you didn't find libfoo1.so.4 in any of system directories configured, try also at /usr/lib/myapp/stubs/ (which'd contain stubbed libfoo1.so.4)". Apparently, nothing like "rpath-fallback" exists right now. I'm considering creating a feature request for such a "rpath-fallback" if the "optional libraries via stubs" idea gets wider support.
The issue comes often when one has a piece of software which could use extra functionality the shared library (let's call it biglib) provides, but the developer/maintainer doesn't want to force all users to install it. The common solution seems to be either:
- defining a plugin interface and a bridge library (better detailed at here);
- dlopen/dlsym to load each library symbol by hand (good luck doing that for high-level C++ libraries).
Both solutions involve dlopen at one stage or another to avoid linking with a biglib, since if you do that, the application loader will refuse to load the application unless biglib and all of its dependencies are present. But what if application is prepared to fallback at run time, and just needs a way to be able to start without biglib?
I went ahead to check what if there was a stub library to provide all symbols which the application uses (directly or indirectly) out of biglib. Turns out that yes, at least for simple cases it seems to work. I've published my experimental stub generator at https://github.com/jackyf/so-stub .
For practical use, we'd also need a way to tell the loader that a stub library has to be loaded if the real library is not found. While there is many ways to instruct the loader to load something instead of system libraries (LD_PRELOAD, LD_LIBRARY_PATH, runpath, rpath), I found no way to load something if a system library was not found. In other words, I'd like to say "dear linker/loader, when you're loading myapp: if you didn't find libfoo1.so.4 in any of system directories configured, try also at /usr/lib/myapp/stubs/ (which'd contain stubbed libfoo1.so.4)". Apparently, nothing like "rpath-fallback" exists right now. I'm considering creating a feature request for such a "rpath-fallback" if the "optional libraries via stubs" idea gets wider support.