[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Sat, 30 Jan 2016 05:00:52 +0100
From: magnum <john.magnum@...hmail.com>
To: john-users@...ts.openwall.com
Subject: Re: MPI with Spot Instances?
On 2016-01-30 04:47, japhar81 wrote:
> I'm still a bit fuzzy on the modes.. I'm going after a RAR file's hash (via
> rar2john) in incremental mode.. which of those cases does that fall into?
I'm not sure exactly how much redundant work may be issued for
incremental when resuming after a crash but I only recall seing notable
issues with single (many salts) and prince.
magnum
> On Fri, Jan 29, 2016 at 10:44 PM, magnum <john.magnum@...hmail.com> wrote:
>
>> On 2016-01-30 04:25, japhar81 wrote:
>>
>>> They basically just disappear, whatever they were in the middle of -- i
>>> guess my question is, will the resume re-run whatever jobs those nodes
>>> were
>>> in the middle of and didn't report back? And if one of them happens to
>>> have
>>> hit a match, will that get saved somehow too?
>>>
>>
>> A cracked password is very unlikely to not end up in the pot file. The
>> beauty of Solar's design is that if a session dies before it wrote a
>> recently cracked password to the pot file, it will also not have written
>> the corresponding unit of work to the session file. So a resume will almost
>> certainly re-crack the password.
>>
>> Jumbo's MPI functionality is very KISS minded and doesn't rely on
>> "reporting back" anything to anyone at all. Each node runs in its own daft
>> world not giving a dang about the others. Each node writes its own session
>> file just as any non-MPI session would. In fact, the code paths are *very*
>> near 100% the same as when running --node=x/y in a single process except
>> the "x" and "y" is filled in automagically.
>>
>> Worst-case scenario is supposedly that a resume will do a bit of redundant
>> work. This is obviously by design - better safe than sorry. The default
>> "Save" timer in john.conf is 60 seconds for Jumbo so you will hopefully not
>> lose more than that. Some modes (eg. single w/ many salts and PRINCE
>> regardless of salts) may be much worse than that though, to the point that
>> a stop/resume once an hour may end up never proceeding past this hour at
>> all.
>>
>> magnum
>>
>>
>> On Fri, Jan 29, 2016 at 10:20 PM, magnum <john.magnum@...hmail.com> wrote:
>>>
>>> On 2016-01-29 18:12, japhar81 wrote:
>>>>
>>>> Ok, so corollary question -- does the session stuff work with MPI? i.e.
>>>>> lets say I start the spot instances externally, and mpiexec jtr with
>>>>> some
>>>>> flavor of --session (on a box that wont die). If those nodes die
>>>>> mid-process, will that be recorded in the session to enable a resume
>>>>> later
>>>>> when I spin new nodes and start mpiexec again?
>>>>>
>>>>>
>>>> Sure (as far as I can imagine how spot instances work). Session file
>>>> integrity is very well tested.
>>>>
>>>> magnum
>>>>
>>>>
>>>> On Wed, Jan 27, 2016 at 4:03 PM, magnum <john.magnum@...hmail.com>
>>>> wrote:
>>>>
>>>>>
>>>>> On 2016-01-27 17:25, japhar81 wrote:
>>>>>
>>>>>>
>>>>>> I've been playing around with MPI clustering JtR for a while, and I've
>>>>>>
>>>>>>> managed to get it running smoothly on static nodes. What I'd like to
>>>>>>> do
>>>>>>> next is create an auto-scaling group in AWS, using spot instances.
>>>>>>> What
>>>>>>> this basically means is nodes will come and go, with their
>>>>>>> hostnames/IPs
>>>>>>> changing at random.. I can not figure out how I would run JtR in that
>>>>>>> instance -- since it requires a node list in a file on startup to
>>>>>>> mpirun.
>>>>>>>
>>>>>>> If it matters, I'm looking to do a brute-force using the ASCII mode.
>>>>>>> Has
>>>>>>> anyone found a way to do a dynamic cluster that adds/removes nodes at
>>>>>>> random? Is this even possible?
>>>>>>>
>>>>>>>
>>>>>>> I'm not aware of any existing work that would do this. A solution
>>>>>> using
>>>>>> JtR as-is, but with some yet-to-be-implemented master issuing jobs,
>>>>>> could
>>>>>> involve looking at the existing "-node=x/y" as describing "pieces"
>>>>>> instead
>>>>>> of "nodes". So instead of saying -node=2/8 as in "you are node 2 of 8"
>>>>>> you'd say -node=4321/100000 as in "do piece 4321 of 100000". Then you'd
>>>>>> submit pieces to active nodes. Obviously you'd have to handle dying
>>>>>> nodes
>>>>>> that never reported back their given piece, and re-issue those pieces.
>>>>>>
>>>>>> magnum
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
