[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20170203154726.GA18493@openwall.com>
Date: Fri, 3 Feb 2017 16:47:26 +0100
From: Solar Designer <solar@...nwall.com>
To: john-users@...ts.openwall.com
Subject: Re: to Single or not to Single
On Fri, Feb 03, 2017 at 04:35:03PM +0100, Patrick Proniewski wrote:
> So I tweaked the pw-1M file by screwing up randoms candidates (~38.4% of total candidates count):
>
> sed 's,\([iwnfjzahlc]\):,\1x:,' pw-1M >pw-1M-real
>
> then benchmarking the resulting file:
>
> $ time ./john --single=None --nolog --verbosity=1 pw-1M-real --pot=pw-1M-real.pot
> Using default input encoding: UTF-8
> Loaded 1000000 password hashes with 1000000 different salts (dynamic_25 [sha1($s.$p) 128/128 AVX 4x1])
> Press 'q' or Ctrl-C to abort, almost any other key for status
> 0g 0:00:00:02 50.00% (ETA: 16:24:20) 0g/s 0p/s 0c/s 0C/s
> 30769g 0:00:00:08 50.00% (ETA: 16:24:33) 3445g/s 5563p/s 5563c/s 5563C/s aoygm
> 48159g 0:00:00:18 50.00% (ETA: 16:24:53) 2544g/s 4108p/s 4108c/s 4108C/s agmys
[...]
> 145131g 0:00:03:06 50.00% (ETA: 16:30:29) 776.4g/s 1257p/s 1257c/s 1257C/s airzx
> 193370g 0:00:06:35 50.00% (ETA: 16:37:27) 488.4g/s 791.9p/s 791.9c/s 791.9C/s amoxr
> ...
>
> So I guess non-good candidates have a huge performance hit somewhere :/
That's interesting. It could well indicate a bug. Jim, since this is
about dynamic, you could want to experiment with it.
Also, the loader is slow, and it spends several minutes after the
"Loaded" line and before starting cracking, doing I'm not sure what.
(It's supposed to initialize the per-salt hash tables at that time,
but here we have just one hash per salt, so it shouldn't.)
Using my non-SIMD build described in my previous posting:
$ time ./john --single=None --nolog --verbosity=1 pw-47M
Using default input encoding: UTF-8
Loaded 47000000 password hashes with 47000000 different salts (dynamic_25 [sha1($s.$p) 64/64])
Press 'q' or Ctrl-C to abort, almost any other key for status
0g 0:00:00:55 50.00% (ETA: 18:36:56) 0g/s 0p/s 0c/s 0C/s
3533874g 0:00:01:00 50.00% (ETA: 18:37:05) 58478g/s 58478p/s 58478c/s 58478C/s aeydcg
13100375g 0:00:01:12 50.00% (ETA: 18:37:29) 180869g/s 180869p/s 180869c/s 180869C/s bkwdbp
28969829g 0:00:01:32 50.00% (ETA: 18:38:09) 313424g/s 313424p/s 313424c/s 313424C/s bwnrki
36824698g 0:00:01:42 50.00% (ETA: 18:38:29) 359510g/s 359510p/s 359510c/s 359510C/s kahrd
47000000g 0:00:02:18 DONE (2017-02-03 18:37) 339325g/s 339325p/s 339325c/s 339325C/s abtvb
Session completed
real 20m32.338s
user 19m17.369s
sys 0m46.745s
It does manage to load the 47M hashes at once, but this takes 19 minutes.
The "Loaded" line appears after around 10 minutes, at which point memory
consumption is at around 16 GB. Then after ~9 minutes more, it's at 60 GB
and cracking finally starts (and completes in another 2 minutes in this
test, with no further memory usage growth).
Jim, I guess dynamic isn't handling these large salt counts efficiently.
I also tried "--save-memory=2" and "=3" on pw-1M, and it wasn't very
different from the defaults - neither in memory usage, nor in speed.
That's no surprise since we have only 1 hash per salt (so the per-salt
bitmaps and hash tables that this setting would primarily affect are
presumably non-existent anyway).
Alexander
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
