[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <CANWtx02cu2MDqE=9T+MYED-TeNs0PdRk=Dg4AmzufbP81SXiPg@mail.gmail.com>
Date: Mon, 9 Apr 2018 16:13:55 -0400
From: Rich Rumble <richrumble@...il.com>
To: john-users@...ts.openwall.com
Subject: Re: Splitting workload on multiple hosts
On Mon, Apr 9, 2018 at 3:03 PM, Solar Designer <solar@...nwall.com> wrote:
> On Mon, Apr 09, 2018 at 01:37:30PM -0400, Rich Rumble wrote:
> > Sorry to dredge this subject back up, I'm not convinced Fork is fully
> using
> > all 24 CPU's in my single machine to the best if it's ability, on an
> > "incremental" run I'm doing. Will some modes work better in fork than
> > others? I know certain algorithms do, and mine is one of them
> (raw-sha1). I
> > have a few (other)issues, one being the hashes I'm going after are
> enormous
> > and I can't fit them all in ram at once (HaveIBeenPwnd v2) so I've split
> > them up into 20 1Gb slices. Perhaps a new thread may be needed for the
> > incremental issue I'm not sure, but using -fork=24 seems to only see 6-8
> > threads of 100% util, and status updates are also between 6-8 when
> pressing
> > a key. So I have found I can load four 1Gb slices in ram (save-mem=2),
> and
> > run fork=6 on those. In doing that I appear to have some overlap, in that
> > some threads are being used twice for work, but I'm not 100% sure. But
> if I
> > stop one of the four runs, as soon as it's stopped one or two of the
> > remaining three start churning out passwords like crazy. I do not think
> > this is a problem fork/node are there to solve, but was curious if there
> > was a way to make sure work in cpu/threads 1-6 are only done by this john
> > instance, and work for the other john instance 1-6 are only done by
> > cpu/threads 7-12. Since I'm doing different work, I didn't think node
> would
> > be the answer for that, I figured the potential for overlap would be the
> > same even if I specified node=0-5 for each instance.
>
> How much RAM do you have? It sounds like some of the child processes
> are simply getting killed on out-of-memory. Unfortunately, when JtR
> cracks a password the child processes deviate from each other in their
> memory contents, and their combined memory usage grows. This is not
> ideal, but that's how it currently is with "--fork".
>
> You'll want to get most of those HaveIBeenPwnd v2 passwords cracked
> while running fewer processes (e.g., initially just one or two so that
> you can possibly load all of the hashes at once) before you proceed to
> attempt using all 24 that you need for your machine.
>
> Helpful john.conf settings:
>
> NoLoaderDupeCheck = Y
>
> This is the default anyway, but maybe worth double-checking:
>
> ReloadAtCrack = N
>
> These are not obviously an improvement (with these at "N", the pot file
> may grow larger from more duplicate entries, but cracking will be faster
> and the memory usage increase from copy-on-write across --fork'ed
> processes should be less, so more of them may be run):
>
> ReloadAtDone = N
> ReloadAtSave = N
>
> Helpful command-line options:
>
> -verb=1 -nolog -save-mem=1
>
> "-save-mem=1" should actually speed things up by not wasting memory on
> pointers to (non-existent) login names, which also improves the locality
> of reference. "-save-mem=2" has performance impact and is probably not
> worth it in this case.
>
> You may also want to increase PASSWORD_HASH_SIZE_FOR_LDR in params.h by
> one (from 4 to 5) to speedup loading of large hash files like this, and
> rebuild. (The same change slows down loading of small files, which is
> why it's not the default.)
>
> FWIW, I previously experimented with HaveIBeenPwnd v1, which was 320M
> hashes. I loaded those all at once (without splitting) and was able to
> run a few forks at first (4 or so) and all 40 forks eventually on a
> machine with 128 GB RAM with 40 logical CPUs.
>
> You really need to watch your RAM usage when you do things like this.
> If you see less than a half of RAM free, chances are it will be eaten up
> and some children will die as they crack more passwords. So try to keep
> your fork count such that you leave a half of RAM free when cracking
> just starts.
>
> Alexander
>
I have 32Gb of ram, and I never tried without fork this whole time :) I'm
past 300M
cracked (out of 501M), and I know hashes.org is already at 95% or more, I
was just
trying to do it myself. Until the last few days I was always using
fork=24, and unless
I used save-memory=2 it was going over and into swap for the 1G slices. I
was also
using a min-length=6 so at most I would of thought only a few processes
would of
died off/finished in the short runs I'm doing. Wordlist, Prince and hybrid
mask also
ate up tons of ram on large wordlists (300Mb+), incremental so far has been
the
lesser user so far. After these threads run a few more days I may just
start the whole
process over again with your suggestions and see how things go. Thanks
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
