Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <4D9A67B3.7060706@bredband.net>
Date: Tue, 05 Apr 2011 02:52:03 +0200
From: magnum <rawsmooth@...dband.net>
To: john-dev@...ts.openwall.com
Subject: Re: binary hashes and BINARY_SIZE

On 2011-03-29 19:25, Solar Designer wrote:
> So John uses the former approach - keep both things in memory - but then
> many "formats" make an optimization where only partial binary hashes are
> kept in memory.  Really, say, 32 bits per hash is enough to weed out
> most candidate passwords.  And in the relatively rare cases that partial
> hashes match, John can always re-decode the ASCII strings (which it has
> code for anyway) and do the full comparison.
>
> Thus, cmp_all(), cmp_one(), and all of binary_hash[]() and get_hash[]()
> functions work on potentially partial binary hashes, whereas cmp_exact()
> works on full ASCII-encoded hashes.

Thanks a lot for that explanation! I just played with this in my 
modified version of NETNTLM and realised this format benefits a lot more 
than just saving memory: It originally performs three independant DES 
operations in crypt_all() resulting in 24 bytes of output. I made it 
just do the first one for partial binaries, and added a crypt_one() that 
does the rest but is only called from cmp_exact. Not sure I got 
everything 100% right yet but there was obviously a significant 
performance boost:


$ ./john-openwall -fo:netntlm -test=3
Benchmarking: NTLMv1 C/R MD4 DES [netntlm]... DONE
Many salts:	637295 c/s real, 639427 c/s virtual
Only one salt:	613289 c/s real, 613289 c/s virtual


$ ./john-unicode -fo:netntlm -test=3
Benchmarking: NTLMv1 C/R MD4 DES [ESS MD5] [netntlm]... DONE
Many salts:	2256K c/s real, 2264K c/s virtual
Only one salt:	1462K c/s real, 1462K c/s virtual


...and using OMP, two threads:

$ ./john-unicode-omp -fo:netntlm -test=3
Benchmarking: NTLMv1 C/R MD4 DES [ESS MD5] [netntlm]... DONE
Many salts:	4333K c/s real, 2192K c/s virtual
Only one salt:	2129K c/s real, 1069K c/s virtual


Running a 700 MB wordlist against 20 hashes/3 salts is now down from 
6:07 (jumbo 12) to 1:51 (without OMP).

I'll polish this a bit and post it later.

magnum

Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.