john-dev - Re: New SIMD generations, code layout

Follow @Openwall on Twitter for new release announcements and other news

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20150403100736.GA24729@openwall.com>
Date: Fri, 3 Apr 2015 13:07:36 +0300
From: Solar Designer <solar@...nwall.com>
To: john-dev@...ts.openwall.com
Subject: Re: New SIMD generations, code layout

On Fri, Apr 03, 2015 at 10:36:56AM +0800, Lei Zhang wrote:
> On Apr 3, 2015, at 12:32 AM, magnum <john.magnum@...hmail.com> wrote:
> > So what speeds do you see on the MIC now, for these formats? Are we
> > hitting the bottleneck with single-thread candidate generation?
> 
> Single-threaded:
> 
> [zhanglei@...0 zhanglei]$ jumbo/john --test --format=raw-sha256-ng
> Benchmarking: Raw-SHA256-ng [SHA256 512/512 AVX512 16x]... DONE
> Raw:    1407K c/s real, 1407K c/s virtual
> 
> [zhanglei@...0 zhanglei]$ jumbo/john --test --format=raw-sha512-ng
> Benchmarking: Raw-SHA512-ng [SHA512 512/512 AVX512 8x]... DONE
> Raw:    249164 c/s real, 249164 c/s virtual

I think we should call this MIC and not AVX512, since we know we've
built for MIC.

> Multi-threaded:
> 
> [zhanglei@...0 zhanglei]$ jumbo/john --test --format=raw-sha256-ng
> Will run 240 OpenMP threads
> Benchmarking: Raw-SHA256-ng [SHA256 512/512 AVX512 16x]... (240xOMP) DONE
> Raw:    3072K c/s real, 50477 c/s virtual
> 
> [zhanglei@...0 zhanglei]$ jumbo/john --test --format=raw-sha512-ng
> Will run 240 OpenMP threads
> Benchmarking: Raw-SHA512-ng [SHA512 512/512 AVX512 8x]... (240xOMP) DONE
> Raw:    2457K c/s real, 90728 c/s virtual

While this is a nice test, for practical use I think only --fork=240
currently makes sense with fast hashes like this.  Perhaps you can run
some --fork=240 tests, with incremental mode and length locked to 8 (to
exclude the length switching overhead, which is significant early on)?

At a later time, we may introduce partial candidate password generation
(perhaps for mask mode) into crypt_all(), just like Sayantan has worked
on for GPUs, and then we'd get sane speeds with OpenMP as well.

> BTW, the latest commit makes sha256-ng fail on MIC. The issue is related to the macro "SIMD_COEF_32". See this function from rawSHA256_common_plug.c:
> 
> void * sha256_common_binary(char *ciphertext) {
> 	...
> #ifdef SIMD_COEF_32
>     alter_endianity (out, BINARY_SIZE);
> #endif
> 	...
> }
> 
> For MIC, SIMD_COEF_32 is not defined yet at this point while it should be. Maybe we should include pseudo-intrinsics.h in this file, or put the definition of SIMD_COEF_32/64 to some other common header?

I'll let magnum comment on this.

Thanks,

Alexander

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.