musl - broken shared executables on armeb (illegal instruction)

Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Message-ID: <20180930215317.GA11313@zx2c4.com>
Date: Sun, 30 Sep 2018 23:53:19 +0200
From: "Jason A. Donenfeld" <Jason@...c4.com>
To: musl@...ts.openwall.com
Subject: broken shared executables on armeb (illegal instruction)

Hello,

There appears to be a problem with shared linking on big-endian
ARM (armeb).

First I'll show that static linking works correctly:

$ armeb-pc-linux-gnueabi-gcc -v
Using built-in specs.
COLLECT_GCC=armeb-pc-linux-gnueabi-gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/armeb-pc-linux-gnueabi/8.2.0/lto-wrapper
Target: armeb-pc-linux-gnueabi
Configured with: /var/tmp/portage/cross-armeb-pc-linux-gnueabi/gcc-8.2.0-r3/work/gcc-8.2.0/configure --host=x86_64-pc-linux-gnu --target=armeb-pc-linux-gnueabi --build=x86_64-pc-linux-gnu --prefix=/usr --bindir=/usr/x86_64-pc-linux-gnu/armeb-pc-linux-gnueabi/gcc-bin/8.2.0 --includedir=/usr/lib/gcc/armeb-pc-linux-gnueabi/8.2.0/include --datadir=/usr/share/gcc-data/armeb-pc-linux-gnueabi/8.2.0 --mandir=/usr/share/gcc-data/armeb-pc-linux-gnueabi/8.2.0/man --infodir=/usr/share/gcc-data/armeb-pc-linux-gnueabi/8.2.0/info --with-gxx-include-dir=/usr/lib/gcc/armeb-pc-linux-gnueabi/8.2.0/include/g++-v8 --with-python-dir=/share/gcc-data/armeb-pc-linux-gnueabi/8.2.0/python --enable-languages=c,c++,fortran --enable-obsolete --enable-secureplt --disable-werror --with-system-zlib --enable-nls --without-included-gettext --enable-checking=release --with-bugurl=https://bugs.gentoo.org/ --with-pkgversion='Gentoo 8.2.0-r3 p1.3' --disable-esp --enable-libstdcxx-time --enable-poison-system-directories --with-sysroot=/usr/armeb-pc-linux-gnueabi --disable-bootstrap --enable-__cxa_atexit --enable-clocale=gnu --disable-multilib --disable-altivec --disable-fixed-point --with-float=soft --enable-libgomp --disable-libmudflap --disable-libssp --disable-libmpx --disable-systemtap --enable-vtable-verify --enable-libvtv --enable-lto --without-isl --enable-libsanitizer --enable-default-pie --enable-default-ssp
Thread model: posix
gcc version 8.2.0 (Gentoo 8.2.0-r3 p1.3) 
$ tar xf musl-1.1.20.tar.gz 
$ cd musl-1.1.20/
$ export CFLAGS="-O2 -march=armv7-a -mtune=cortex-a15 -mabi=aapcs-linux"
$ CC=armeb-pc-linux-gnueabi-gcc ./configure --prefix=$PWD/prefix --enable-static --disable-shared --build=armeb-pc-linux-gnueabi
[...]
$ make -j$(nproc)
[...]
$ make install
[...]
$ cd prefix/
$ printf '#include <stdio.h>\nint main(){puts("hello world");}' | bin/musl-gcc -xc -o helloworld $CFLAGS -
/usr/libexec/gcc/armeb-pc-linux-gnueabi/ld: /usr/lib/gcc/armeb-pc-linux-gnueabi/8.2.0/libgcc.a(_dvmd_lnx.o): in function `__aeabi_idiv0':
/var/tmp/portage/cross-armeb-pc-linux-gnueabi/gcc-8.2.0-r3/work/gcc-8.2.0/libgcc/config/arm/lib1funcs.S:1545: undefined reference to `raise'
collect2: error: ld returned 1 exit status
[This appears to be a well-known bug in some other mailing list post. Working around with the next command:]
$ printf '#include <stdio.h>\nint main(){puts("hello world");}' | bin/musl-gcc -xc -o helloworld $CFLAGS -static -
$ cp /usr/bin/qemu-armeb .
$ sudo chroot $(readlink -f .) /qemu-armeb /helloworld
hello world

Now let's try with shared linking, and you'll see it generates a
broken binary:

$ tar xf musl-1.1.20.tar.gz 
$ cd musl-1.1.20/
$ export CFLAGS="-O2 -march=armv7-a -mtune=cortex-a15 -mabi=aapcs-linux"
$ CC=armeb-pc-linux-gnueabi-gcc ./configure --prefix=$PWD/prefix --disable-static --enable-shared --build=armeb-pc-linux-gnueabi
[...]
$ make -j$(nproc)
[...]
$ make install
[...]
$ cd prefix/
$ printf '#include <stdio.h>\nint main(){puts("hello world");}' | bin/musl-gcc -xc -o helloworld $CFLAGS -
$ cd lib/
$ ln -s libc.so ld-musl-armeb.so.1
$ cd ..
$ cp /usr/bin/qemu-armeb .
$ sudo chroot $(readlink -f .) /qemu-armeb /helloworld
Illegal instruction

I've experienced similar failures when trying to boot with armeb
executables as init with a real kernel that are generated this way.
I've also experienced this with both my own toolchain (above) and
with linaro's toolchain.

I expect the commands above should result in an easily reproducable bug.

Any idea what's up?

Thanks,
Jason
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.