Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20160913204304.GY15995@brightrain.aerifal.cx>
Date: Tue, 13 Sep 2016 16:43:04 -0400
From: Rich Felker <dalias@...c.org>
To: musl@...ts.openwall.com
Subject: Re: incompatibility between libtheora/mmx and musl ?

On Tue, Sep 13, 2016 at 08:06:49PM +0200, u-uy74@...ey.se wrote:
> Hello,
> 
> Libtheora contains an assembler part for ia32 and x86_64 which has been
> in use for many years, with at least both glibc and uclibc.
> 
> Now, musl-based builds of libtheora for ia32 with this code enabled
> lead to the encoder segfaulting. No problem when using the C version.
> 
> This is seen when building at Aetey. Ffmpeg in Alpine exhibits the same
> problem, too (strictly speaking, possibly another problem but ffmpeg
> crashes there with the same pattern, when encoding to theora).
> 
> People at Xiph do not have any musl-based system and want really detailed
> debugging info which implies tracing through musl and following malloc's
> internal data (which is what seems to become corrupted).
> 
> In other words, the detailed knowledge (xiph and musl) is located
> separately from each other and also from the motivation (aetey, alpine
> and possibly others).
> 
> I would appreciate some help and suggestions. The alternative is diving
> into mmx-assembler and musl code or resorting to C-only routines which
> give roughly half of the performance of the assembler ones.
> 
> More details are there at
> 
>  https://trac.xiph.org/ticket/2287
> 
> There is also a corresponding
> 
>  https://bugs.alpinelinux.org/issues/6132

The most likely explanation is that they're overflowing a heap buffer.
Perhaps it would be possible to pad all their malloc/realloc calls
with +64 or so to see if that makes the problem go away. If so that
gives a good starting point for tracking down the bug.

Rich

Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.