oss-security - Apache web server use after free bugs (unfixed)

Follow @Openwall on Twitter for new release announcements and other news

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <20190121090535.227a1db9@computer>
Date: Mon, 21 Jan 2019 09:05:35 +0100
From: Hanno Böck <hanno@...eck.de>
To: oss-security@...ts.openwall.com
Subject: Apache web server use after free bugs (unfixed)

Apache use after free bugs
==========================

While doing some fuzz testing on the apache httpd server
with address sanitizer we regularly observed use after free
bugs. We originally observed these issues in the http2
module, but we were also able to reproduce them without
http2 enabled, so either we're facing multiple bugs or
there's an underlying bug in the core apache code.

Originally we used fuzzing payloads to trigger this bug,
but we later observed that sending random garbage in
parallel is enough to trigger the bug.

We reported this behavior to the apache security team for the first
time in June 2018. The apache developers did not seem to take the 
issue as seriously as we had expected. 

It was pointed out to us that some fixes already in their code may
fix the issue, however we are still able to reproduce these bugs in
the latest version (2.4.37).

The apache developers indicated to us that they'd not consider
these security issues unless we can show a practical exploit.
Due to the complexity of the apache code base and our lack
of specialization in binary memory exploitation we feel unable
to do this. It is, however, our belief that use after free bugs 
should generally be seen as potential security bugs.

For this reason, we have chosen to share this information with the
community and hope others will continue the analysis.

apr pool allocator
==================

For memory allocations apache http uses the apr library's
pool allocator that allows reserving a larger chunk of
memory as a pool and do memory allocations within that pool.
This can, and in our case does, hide memory safety issues.

apr has an option --enable-pool-debug=yes that will cause
a single malloc call for each memory allocation, allowing
the use of memory safety checkers like ASAN.

The apache developers suggested that our ASAN reports may stem
from an incompatibility between the pool debugger and the http2
module. However we were later able to reproduce these issues
without the http2 module.

We were also able to reproduce these issues with valgrind and
without the pool allocator.

threading related error
=======================

In addition to the ASAN use after free reports, httpd logs threading
related errors:

AH00052: child pid [pid] exit signal Aborted (6)
apache2: tpp.c:84: __pthread_tpp_change_priority: Assertion `new_prio
== -1 || (new_prio >= fifo_min_prio && new_prio <= fifo_max_prio)'
failed.

We found a ten year old bug in the Apache bug tracker
mentioning such errors:
https://bz.apache.org/bugzilla/show_bug.cgi?id=46185

It was closed as "INVALID".

asan stack traces
=================

We share asan stack traces from these bugs at
  https://github.com/hannob/apache-uaf/tree/master/asan

reproduction
============

To reproduce the issue:

1. Compile apr with the pool debugger and address sanitizer.

2. Compile apache with address sanitizer.

3. Run a command like this to send random garbage to the server:
for x in $(seq 1 50); do for i in $(seq 1 1000); do head -n
10 /dev/urandom | nc 127.0.0.1 80 & done; sleep 5; done

The bugs appear very irregularly, you may need to
"attack" it for a while.

Hanno Böck
Craig Young (Tripwire VERT)

Thanks to Markus Vervier and Luis Merino of X41 D-SEC GmbH for double
checking.

-- 
Hanno Böck
https://hboeck.de/

mail/jabber: hanno@...eck.de
GPG: FE73757FA60E4E21B937579FA5880072BBB51E42

Please check out the Open Source Software Security Wiki, which is counterpart to this mailing list.

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.