Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20110218195846.GA2467@openwall.com>
Date: Fri, 18 Feb 2011 22:58:46 +0300
From: Solar Designer <solar@...nwall.com>
To: john-users@...ts.openwall.com
Subject: Re: MD5 Generic improvements

Jim, Simon, Erik -

I've just "tested" Jim's john-1.7.6-jumbo-12-early-release-md5-gen-1 on
linux-x86-64 and linux-x86-sse2 (both with gcc 4.5.0).  All self-tests
pass. :-)  Then I went to test "Incorporation of SSE2 intrinsic code":

On Thu, Feb 17, 2011 at 11:41:13AM -0600, jfoug wrote:
> 1.       Incorporation of SSE2 intrinsic code.  This is triggered by the
> MD5_PARA #define.  For this code to 'work', the SSE intrinsic patch will
> need to be installed, and for most performance gains, the ICC will need to
> be used as the compiling environment.  NOTE, this intrinsic code is
> 'pre-patched' in.  It is not part OF the intrinsic patch, but it will not
> compile in UNTIL the intrinsic patch code is put in.  The intrinsic patch is
> also found on the wiki 'patches' page.

The first hurdle was that Simon's intrinsics patch was against -jumbo-6
and it would not apply against -jumbo-12 cleanly.  I took care of that,
uploading a revised patch to the wiki:

http://openwall.info/wiki/john/patches

as well as to:

http://download.openwall.net/pub/projects/john/contrib/

In fact, I uploaded two revisions: john-1.7.6-jumbo-12-intrinsics-1
(under historical/) is a mere update to -jumbo-12, whereas
john-1.7.6-jumbo-12-intrinsics-2 also fixes some compiler warnings, at
least one of which was for a real bug.  It looks like MD5_apache_fmt.c
in jumbo got BINARY_SIZE wrong, which was not exposed until gcc gave me
these warnings (with the intrinsics patch applied):

MD5_apache_fmt.c: In function 'cmp_all':
MD5_apache_fmt.c:166:60: warning: array subscript is above array bounds

For now, this is fixed only in john-1.7.6-jumbo-12-intrinsics-2,
although it needs a fix in jumbo.

Anyway, regardless of these warning fixes,
john-1.7.6-jumbo-12-intrinsics-2 (as well as -1) fails two self-tests on
linux-x86-64 with gcc 4.5.0:

Benchmarking: Apache MD5 [12x]... FAILED (get_hash[1](0))
[...]
Benchmarking: Raw MD5 [raw-md5 SSE]... ERROR buffer not cleaned properly when setting key  index 1

At this point it just exits, without testing further formats.  My guess
is that my self-test enhancements have exposed some incompatibilities
with this patch (maybe bugs).  Simon - can you please take a look and
likely upload a corrected patch against -jumbo-12?

Finally, I went to test john-1.7.6-jumbo-12-intrinsics-2 applied along
with john-1.7.6-jumbo-12-early-release-md5-gen-1.  These two apply
together cleanly (with the only exception being the version string in
params.h).  Self-tests fail in the exact same way as they do for
john-1.7.6-jumbo-12-intrinsics-2 alone.

Jim - did you get these patches working together?  Did you also have
some self-tests fail?

Thanks,

Alexander

Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.