Botstrapping and cross-compiling
Maybe it’s me, but I like to know the environment I work in and add facilities for helping in my daily workflows. I have more than 50 executables (mostly shell script) under
~/bin, some are used daily, others once or twice a year. The main advantage is that for both cases, shell scripts provide very good documentation of how I do accomplish specific tasks.
This is one of the main reasons for using a computer: automating tasks, reducing repetitive work. If not, I tend to prefer human interaction, as generally human beings tend to be friendlier than a robot.
So when I get a new device, a new environment, or when I have to work from a different workstation, one of the first things I do is looking around what’s available and eventually add those little programs I find so useful.
In some environments, some policies do not allow installing new programs, downloading executables, or using USB drives.
Fortunately, both Windows and POSIX-based systems have shells, and I am still allowed to open a text editor and save textual files.
While the language given by the shell is not always ideal, it is possible to write many utilities with it. If lucky, it is possible to let do the hard work to external programs and just glue them together.
This is coincidentally part of the Unix philosophy; provides little specialized tools that can be put together easily.
Normally one should take advantage of programs available on the systems, most scripts just call some programs and glue them together. Of course, being able to install third-party programs makes everything easier, why reinvent the wheel if there is not enough time?
But this year I had to work in a (for me) new environment. A Linux distribution, missing most of the
coreutils executables (not even
sort where available!), and with no package manager.
/bin directory is painfully empty. Only the absolute minimum is available: an old
busybox and half a dozen of other executables.
Also, the programs that are available (like
chmod), are crippled, missing many options, and of course, other useful programs, like
awk, were missing completely.
Oh, and because there is only a minimal
sh shell, there is no autocompletion and no history.
While it might seem like a very improbable scenario to encounter such a device, nearly everyone has one with approximately the same limitations: android phones.
It is possible to enable access to the shell, but it is a very stripped-down environment. If you can install programs,
termux provides a package manager and nearly all programs one can expect on a GNU/Linux distribution. Unless you have an older phone, as
termux does not support versions previous to Android 7.
Maybe it’s just me, but the idea of working in such an environment is horrible.
The only possibility to work efficiently, without changing the environment, is offloading as much work as possible on a separate PC. But as tools like
ssh are missing too, even copying a file is a process difficult to automate (and of course
tar and other archiving utilities are missing).
So either I could do everything by hand, or try to offload as much as possible.
I know by experience that by doing everything by hand I will do the most improbable and dumbest errors, and burn a lot of time and energy solving non-issues multiple times. I tried offloading as much work as possible, but just synchronizing content and trying to debug the issue is such a hurdle that I decided I had to come up with something else.
Bootstrapping my toolbox on that machine.
The downside is that it will cost some time to get something useful.
The upside is that I do need to do it only once.
If I do everything by hand, I will need more time for every task. And every time, if something goes wrong, there will be the doubt if I had done something differently.
The first tool I noticed I would very like to have is
grep. Searching in text files or piping output is such a common operation that I decided I would try to compile it for the platform.
The first issue is that this platform has a different architecture: 32-bit arm.
Thus I need to cross-compile.
Fortunately, both clang and GCC can compile for arm, so I installed
arm-linux-gnueabi-gcc package on my GNU/Linux machine, and after extracting the sources of
grep I’ve executed
> ./configure CC=arm-linux-gnueabi-gcc --host=arm-none-linux-gnueabi && make # configure and make output ... > file src/grep src/grep: ELF 32-bit LSB pie executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.3, BuildID[sha1]=384ab6445e534210b6552592459dc745ea098a8c, for GNU/Linux 3.2.0, not stripped
I copied the binary on the device, but it did not start correctly. Turns out that the glibc on the device is too old for the binary I just compiled.
Thus I tried to compile
> ./configure CC=arm-linux-gnueabi-gcc --host=arm-none-linux-gnueabi CFLAGS="-static" && make # configure and make output ... > file src/grep src/grep: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, BuildID[sha1]=590ab26ed4124bbe2592f1f963ebfc06d37151e4, for GNU/Linux 3.2.0, not stripped
And copied again the compiled binary to the device. Again, it would not start. The kernel on the device, which cannot be updated, is too old.
I could not find any option for creating binaries for older kernels in GCC, apparently, I need to recompile
glibc with an the appropriate
Fortunately, I found a toolchain for this device. I’ve "installed" (copied under
/opt) it on my PC, and could use it for building
./configure --host=arm-none-linux-gnueabi CC=/opt/my-toolchain/bin/arm-none-linux-gnueabi-gcc CFLAGS="-w -static" && make
And finally, I’ve got a working binary.
I also tried using clang with the toolchain, but with no luck, as it seems that
configure uses compiler flags not supported by clang when cross-compiling (or probably I’m doing something wrong and still have not figured it out):
./configure --host=arm-none-linux-gnueabi CC=clang CFLAGS="-w -static --target=arm-none-linux-gnueabi --gcc-toolchain=/opt/mytoolchain --sysroot=/opt/my-toolchain/arm-none-linux-gnueabi/libc" && make
grep has some optional dependencies, like PCRE. While C (and C++) are ubiquitous, unfortunately, there is no standard practice or convention for handling dependencies (actually not even for building, even less for cross-compiling).
Thus to keep things simple, at least at the beginning, I’ve decided to avoid binaries with external dependencies.
After grepping happily on my crippled device, I decided that it would be great to
cut some text, and noted that
cut is missing too!
As I’ve managed to get
grep working, I’ve decided to see how difficult it is to download the GNU
Turns out it is a little more complex. First I needed to download some dependencies for building, then I could
> sudo aptitude install automake autopoint bison gperf rsync > ./bootstrap > ./configure CC=arm-linux-gnueabi-gcc --host=arm-none-linux-gnueabi && make
Unfortunately, configure for the target platform does not work. Apparently, it tries to execute a compiled file:
src/make-prime-list: cannot execute binary file: Exec format error.
The solution, found by trial-and-error (not great, I know…), is to compile first for the host platform, and afterward for the target.
> ./bootstrap > ./configure && make > make clean > ./configure CC=arm-linux-gnueabi-gcc --host=arm-none-linux-gnueabi && make
I decided to report it as a bug, but then noticed it is a known bug from 2012.
Now I have
md5sum, a non-crippled
chown, and a lot more utilities, some of them I have never used and probably do not even need.
With those basic tools, I could finally create some useful scripts for automating manual tasks, like copying some part of an output and pass it to another program, monitoring some activities, and so on.
As the POSIX shell is Turing complete it would have been possible to accomplish the same task without cross-compiling anything. It would only have been much more difficult to write what would have been a simple script.
I would love to have other programs like
tmux too, I’ll see what will be the next step.
Now I can work more efficiently. Some tasks that were tedious and error-prone can be done fast and correctly anytime with no effort, just by providing some building blocks available in every sane GNU/Linux distribution.
It is always worth improving the working environment, it is the common denominator for most tasks. Improving, and especially adding tools available in other environments makes it easier to reuse programs (or use different programs with similar interfaces), no matter how small or big, and work more efficiently.
Do you want to share your opinion? Or is there an error, same parts that are not clear enough?
You can contact me here.