[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-Id: <2231735E-EA7D-4CCD-89F2-377E69B20D77@m.patpro.net>
Date: Fri, 3 Feb 2017 13:53:46 +0100
From: Patrick Proniewski <p+password@...atpro.net>
To: john-users@...ts.openwall.com
Subject: Re: to Single or not to Single
Alexander,
On 03 févr. 2017, at 13:14, Solar Designer wrote:
> What version of JtR are you using? Is it almost the latest
> bleeding-jumbo (like this month's off GitHub) or something older?
I'm running a not-too-old Bleeding Jumbo, from the 6th of november 2016. (on FreeBSD 10.x). I've fully upgraded my OS few days ago, I should probably build a newer Bleeding jumbo on top of that.
> On Fri, Feb 03, 2017 at 07:26:29AM +0100, Patrick Proniewski wrote:
>> Oops. forgot to write a title on the graph. It's the number of "Cracked" per minute, extracted from the log file.
>
> Well, it's in fact puzzling to me why this would be decreasing given
> what you have stated before. This would be non-surprising if you had
> rules enabled, or had "SingleRetestGuessed = Y".
>
>> 2799705g 1:08:45:08 50.00% (ETA: 2017-02-04 15:31) 23.74g/s 392.6p/s 392.6c/s 392.6C/s erikita222
>
> Ouch. That's extremely low speed for dynamic_25 (you said this is what
> you had?), which is sha1($s.$p). It should be giving speeds of about
> 10M c/s per CPU core.
This is indeed very low, but I can't remember having achieved millions c/s on this hash type, even with other attack mode. I'll make some tests to check that.
> One obvious reason for a lower speed is that you're presumably testing
> only 1 password per salt, whereas we're optimizing for many passwords to
> be tested at once. On a non-OpenMP build with AVX:
>
> $ ./john --list=format-all-details --format=dynamic_25 | fgrep keys
> Min. keys per crypt 64
> Max. keys per crypt 1680
>
> So you possibly get a 64x performance hit by testing only 1 password,
> and maybe worse.
If I understand well, I should be able to test 64 candidates in the same time I currently test 1. Meaning that if I write a few simple rules (1st char upper case, all char upper case, suffix 123, ...) that create less than 64 candidates, it could be a huge win.
> You could want to ensure you're using a non-OpenMP build (although
> recently OpenMP is disabled by default for "fast hash" formats,
> including this one, so simply using recent bleeding-jumbo should suffice).
OpenMP is not enabled on my install (./configure CC=gcc5 LDFLAGS=-L/usr/lib -L/usr/local/lib/gcc5 CFLAGS=-g -O2 -I/usr/include/openssl --disable-pkg-config --disable-openmp)
>>> along with their preceding Loaded and Remaining lines.
>>
>> How do I get those numbers?
>
> John prints them at startup.
Oh! Of course.
>>> Your use of single mode sounds appropriate for what you're doing.
>>> To reduce its memory needs, try "--save-memory=2".
>>
>> I'll give it a try. For now, I've split the huge file and I reclaim memory by stoping/restoring john every hour (yielding to this result <https://www.patpro.net/~patpro/john-memory-day.png>).
>
> That graph clearly shows your hourly restarts, but it doesn't show a
> reason for you to be doing them - there's no obvious growth in memory
> usage during those hours.
The reason I restart is not good: I just want to give RAM back to the OS (other things running). I'll probably split my files again to make them smaller, and stop stoping/restarting.
> "Single crack" is not restore-friendly. It's "--restore" is based on
> rules, which you don't use. You're probably having it do a lot of
> duplicate work each time you restart.
Damn. I'm going to have to do something about that.
> Anyway, at ridiculously low speeds like what you're getting (need to
> figure out exactly why, but that's a separate issue), you could as well
> go for "--save-memory=3" and probably not notice a further slowdown -
> but you'd save lots of memory, and should be able to load all of your
> hashes at once.
loading the whole file would be impressive, but I would really prefer a huge speed increase. I'll update my bleeding jumbo, and make few benches.
thanks a lot!
patpro
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
