oss-security - CVE-2018-11770: Apache Spark standalone master, Mesos REST APIs not controlled by authentication

Follow @Openwall on Twitter for new release announcements and other news

[<prev] [next>] [day] [month] [year] [list]

Message-ID: <CAEccTyy0JwT+1B4qHVhMB3SRYw-g2x6gmwWUZbd5dcVDk0H0tw@mail.gmail.com>
Date: Mon, 13 Aug 2018 09:24:46 -0500
From: Sean Owen <srowen@...che.org>
To: "oss-security@...ts.openwall.com" <oss-security@...ts.openwall.com>
Subject: CVE-2018-11770: Apache Spark standalone master, Mesos REST APIs not
 controlled by authentication

Severity: Medium

Vendor: The Apache Software Foundation

Versions Affected:
Spark versions from 1.3.0, running standalone master with REST API enabled,
or running Mesos master with cluster mode enabled

Description:
>From version 1.3.0 onward, Spark's standalone master exposes a REST API for
job submission, in addition to the submission mechanism used by
spark-submit. In standalone, the config property
'spark.authenticate.secret' establishes a shared secret for authenticating
requests to submit jobs via spark-submit. However, the REST API does not
use this or any other authentication mechanism, and this is not adequately
documented. In this case, a user would be able to run a driver program
without authenticating, but not launch executors, using the REST API. This
REST API is also used by Mesos, when set up to run in cluster mode (i.e.,
when also running MesosClusterDispatcher), for job submission. Future
versions of Spark will improve documentation on these points, and prohibit
setting 'spark.authenticate.secret' when running the REST APIs, to make
this clear. Future versions will also disable the REST API by default in
the standalone master by changing the default value of
'spark.master.rest.enabled' to 'false'.

Mitigation:
For standalone masters, disable the REST API by setting
'spark.master.rest.enabled' to 'false' if it is unused, and/or ensure that
all network access to the REST API (port 6066 by default) is restricted to
hosts that are trusted to submit jobs. Mesos users can stop the
MesosClusterDispatcher, though that will prevent them from running jobs in
cluster mode. Alternatively, they can ensure access to the
MesosRestSubmissionServer (port 7077 by default) is restricted to trusted
hosts.

Credit:
Imran Rashid, Cloudera
Fengwei Zhang, Alibaba Cloud Security Team

Reference:
https://spark.apache.org/security.html

Please check out the Open Source Software Security Wiki, which is counterpart to this mailing list.

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.