[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Mon, 9 Apr 2018 16:22:17 -0400
From: Stephen John Smoogen <smooge@...il.com>
To: Rich Rumble <richrumble@...il.com>, john-users <john-users@...ts.openwall.com>
Subject: Re: Splitting workload on multiple hosts
On 9 April 2018 at 13:37, Rich Rumble <richrumble@...il.com> wrote:
> On Mon, Aug 4, 2014 at 7:31 PM, magnum <john.magnum@...hmail.com> wrote:
>
>> On 2014-08-04 16:44, Rich Rumble wrote:
>>
>>> On Mon, Aug 4, 2014 at 5:50 AM, magnum <john.magnum@...hmail.com> wrote:
>>>
>>>> Basically the syntax for MPI with --node is the same as for --fork with
>>>> --node. So these two examples are equivalent in terms of work space:
>>>>
>>>> ./john -fork=16 -node=1-16/10000 (...)
>>>>
>>>> mpirun -host=alpha,bravo -np 16 ./john -node=1-16/10000 (...)
>>>>
>>>> Since Fork is no good for Windows and i currently want to dumbforce
>>> something. I don't have network connectivity to most of the hosts I'm
>>> pooling, so I may have to go and use live CD's and use Fork after all, but
>>> in linux. I am using node like the example, but I'm not sure it will work
>>> on external modes as well as fork would.
>>>
>>
>> The key space distribution is 100% identical whether you use fork, MPI or
>> just manual instances of --node.
>>
>> magnum
>>
> Sorry to dredge this subject back up, I'm not convinced Fork is fully using
> all 24 CPU's in my single machine to the best if it's ability, on an
> "incremental" run I'm doing. Will some modes work better in fork than
> others? I know certain algorithms do, and mine is one of them (raw-sha1). I
> have a few (other)issues, one being the hashes I'm going after are enormous
> and I can't fit them all in ram at once (HaveIBeenPwnd v2) so I've split
> them up into 20 1Gb slices. Perhaps a new thread may be needed for the
> incremental issue I'm not sure, but using -fork=24 seems to only see 6-8
> threads of 100% util, and status updates are also between 6-8 when pressing
> a key. So I have found I can load four 1Gb slices in ram (save-mem=2), and
> run fork=6 on those. In doing that I appear to have some overlap, in that
> some threads are being used twice for work, but I'm not 100% sure. But if I
> stop one of the four runs, as soon as it's stopped one or two of the
> remaining three start churning out passwords like crazy. I do not think
> this is a problem fork/node are there to solve, but was curious if there
> was a way to make sure work in cpu/threads 1-6 are only done by this john
> instance, and work for the other john instance 1-6 are only done by
> cpu/threads 7-12. Since I'm doing different work, I didn't think node would
> be the answer for that, I figured the potential for overlap would be the
> same even if I specified node=0-5 for each instance.
> -rich
Are you able to use taskset to push each one to a CPU? I found that
sometimes the kernel would shove multiple processes to the same CPU.
This was done more by the kernel and not the process itself so taskset
or similar tools needed to be done to get the forks off to their own
client. [If this is not possible for obvious reasons, I am sorry.. saw
this while debugging a different problem with taskset ]
--
Stephen J Smoogen.
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
