[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <CAJ9ii1FZt9yBWDCr444F9TgY4QGx7_G79zj4zKfXpsobZNCJRQ@mail.gmail.com>
Date: Wed, 24 Aug 2022 12:13:46 -0400
From: Matt Weir <cweir@...edu>
To: john-users@...ts.openwall.com
Subject: Re: JtR Rules Questions
Thanks Alexander!
That clears up a lot of questions for me. I might try to capture some of
what you put here and submit a MR for /doc/RULES. For example, the
[!-/:-@\[-`{-~]
and:
[ -~]
are extremely useful.
One edge case I found, which makes sense with how you describe the rules
preprocessor, is you can't include a '?' in the replacement rule 's'. So
for example the following will not work:
s[ -~]A
s[?]A
In fact, it took me a while to figure out how to do that even without the
rule preprocessor since s/?A did not work either. I finally figured out I
had to do a s??A, which once again makes sense (as ?? is the character
class for ?), but it took scanning from the top of the RULES file to the
bottom to put all those conditions together.
To expand on what I'm doing, I was talking with some of team Hashcat and
they mentioned that they had very good success in the CMIYC contest
performing "rule-forcing" where they basically either used the random rules
generator of Hashcat "-g", and/or one of them had a very large file of all
the possible rules. This isn't that useful for real life cracking sessions,
but for password cracking competitions it is helpful to discover mangling
rules the organizers employed. I'm playing around with JtR to see if I can
create a similar set of rules, with the difference being that I can
leverage JtR's rule preprocessor to significantly shrink the size of the
rules file.
Oh, and I'm asking these questions here vs.john-contest since I think other
people might be interested in the answers as well :)
Cheers,
Matt / Laki
On Wed, Aug 24, 2022 at 8:56 AM Solar Designer <solar@...nwall.com> wrote:
> Hi Matt,
>
> All of your questions are about the rules preprocessor, and the thing
> you seem to be missing is that the preprocessor is actually unaware of
> rules - it works on individual characters.
>
> On Tue, Aug 23, 2022 at 08:34:43PM -0400, Matt Weir wrote:
> > As some further background, part of this investigation is into trying to
> > replicate the "random rule" (-g flag) option in Hashcat.
>
> The preprocessor isn't sufficient for that. You'd end up with many
> invalid rules, whereas any one invalid rule command is currently a fatal
> error. Maybe we need a mode where JtR would just skip invalid rules.
>
> > 1) It doesn't seem like the delete first character '[' rule is working on
> > the command line:
> >
> > echo "test" | john --pipe --stdout --rules=':['
> > Using default input encoding: UTF-8
> > test
> > 1p 0:00:00:00 0.00% 1.515p/s test
>
> That character starts a preprocessor expression. To refer to the rule
> command, you need to escape it:
>
> $ echo test | ./john --pipe --stdout --rules=':\['
> Using default input encoding: UTF-8
> Enabling duplicate candidate password suppressor
> est
> 1p 0:00:00:01 0.00% 0.9090p/s est
>
> > 2) With the rule preprocessor, is there an easy way to specify all
> > 'special' characters? Trying to capture them while getting the escaping
> > correct is very difficult. Aka for alphanumeric I can
> > specify Az"[a-zA-Z0-9]". (Side note, if there is an easier way I'd be
> > interested in that). But how should I do that for all special characters,
>
> You can use "[!-/:-@\[-`{-~]". There isn't currently an easier way to
> do that. You can test this with (I just did):
>
> echo test | ./john --pipe --stdout --rules=':$[!-/:-@\[-`{-~]'
>
> and:
>
> echo test | ./john --pipe --stdout --rules=':AzA[!-/:-@\[-`{-~]A'
>
> The latter example illustrates how you can have this inside a string
> with the "A" command, by using a quoting character that is never part of
> the quoted string (I chose "A" for that character here).
>
> > or for that matter, all ASCII characters?
>
> For all printable ASCII, it's "[ -~]":
>
> echo test | ./john --pipe --stdout --rules=':$[ -~]'
>
> > 3) I notice that some commands play well within the rule preprocessor and
> > others do not. For example the following works:
> >
> > echo "test" | john --pipe --stdout --rules=':[cul]'
> > Using default input encoding: UTF-8
> > Enabling duplicate candidate password suppressor
> > Test
> > TEST
> > test
> > 3p 0:00:00:00 0.00% 4.225p/s test
> >
> > But you can't seem to do an append in the rule preprocess. So for
> example:
> >
> > echo "test" | john --pipe --stdout --rules=':[$0]'
> > Using default input encoding: UTF-8
> > Invalid rule in Command Line Rule at line 0: Unexpected end of rule $
> >
> > This isn't so much a question vs. a comment I guess... To turn it into a
> > question, is there a list or set of rules that are "approved" to work in
> > the rule preprocessor
>
> No, and this isn't a right question to ask. Rather, you need to
> remember that the preprocessor works on individual characters. So
> "[cul]" expands into 3 rules, and "[$0]" expands into two (which are
> incorrect). If you wanted to have just a single rule "[$0]", then you
> didn't need the preprocessor - just write "$0" without the brackets.
> If you meant this as a starting step to be adding more rules next, then
> you need to more fully define what you're trying to do first. For
> example, to have both appends and prepends of any digit, you could:
>
> echo test | ./john --pipe --stdout --rules=':[$^][0-9]'
>
> When specifying rules on the command line, you can also separate
> multiple "lines" with semicolons, like this:
>
> echo test | ./john --pipe --stdout --rules=':[cul]; [$^][0-9]'
>
> Also try removing the semicolon from there.
>
> I hope this helps.
>
> Alexander
>
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
