Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20170203121456.GA15165@openwall.com>
Date: Fri, 3 Feb 2017 13:14:56 +0100
From: Solar Designer <solar@...nwall.com>
To: john-users@...ts.openwall.com
Subject: Re: to Single or not to Single

Patrick,

What version of JtR are you using?  Is it almost the latest
bleeding-jumbo (like this month's off GitHub) or something older?

On Fri, Feb 03, 2017 at 07:26:29AM +0100, Patrick Proniewski wrote:
> Oops. forgot to write a title on the graph. It's the number of "Cracked" per minute, extracted from the log file.

Well, it's in fact puzzling to me why this would be decreasing given
what you have stated before.  This would be non-surprising if you had
rules enabled, or had "SingleRetestGuessed = Y".

> 2799705g 1:08:45:08 50.00% (ETA: 2017-02-04 15:31) 23.74g/s 392.6p/s 392.6c/s 392.6C/s erikita222

Ouch.  That's extremely low speed for dynamic_25 (you said this is what
you had?), which is sha1($s.$p).  It should be giving speeds of about
10M c/s per CPU core.

One obvious reason for a lower speed is that you're presumably testing
only 1 password per salt, whereas we're optimizing for many passwords to
be tested at once.  On a non-OpenMP build with AVX:

$ ./john --list=format-all-details --format=dynamic_25 | fgrep keys
Min. keys per crypt                  64
Max. keys per crypt                  1680

So you possibly get a 64x performance hit by testing only 1 password,
and maybe worse.

You could want to ensure you're using a non-OpenMP build (although
recently OpenMP is disabled by default for "fast hash" formats,
including this one, so simply using recent bleeding-jumbo should suffice).

> (the 50.00% is totally bogus, it's displayed from the start to the end)

For "single crack", it's based on progress through the rules - and you
said you have none.

> > along with their preceding Loaded and Remaining lines.
> 
> How do I get those numbers?

John prints them at startup.

> > Your use of single mode sounds appropriate for what you're doing.
> > To reduce its memory needs, try "--save-memory=2".
> 
> I'll give it a try. For now, I've split the huge file and I reclaim memory by stoping/restoring john every hour (yielding to this result <https://www.patpro.net/~patpro/john-memory-day.png>).

That graph clearly shows your hourly restarts, but it doesn't show a
reason for you to be doing them - there's no obvious growth in memory
usage during those hours.

"Single crack" is not restore-friendly.  It's "--restore" is based on
rules, which you don't use.  You're probably having it do a lot of
duplicate work each time you restart.

Anyway, at ridiculously low speeds like what you're getting (need to
figure out exactly why, but that's a separate issue), you could as well
go for "--save-memory=3" and probably not notice a further slowdown -
but you'd save lots of memory, and should be able to load all of your
hashes at once.

Alexander

Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.