Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <20090512012133.GA15973@openwall.com>
Date: Tue, 12 May 2009 05:21:33 +0400
From: Solar Designer <solar@...nwall.com>
To: john-users@...ts.openwall.com
Subject: Re: Problem with [List.Rules:NT]

On Thu, Apr 23, 2009 at 11:45:52AM -0500, Minga Minga wrote:
> I have this in my john.conf file. ( I didn't write it - and I don't
> even understand it).
> 
> [List.Rules:NT]
> l
> lMT[*0]T[*1]T[*2]T[*3]T[*4]T[*5]T[*6]T[*7]T[*8]T[*9]T[*A]T[*B]T[*C]T[*D]Q

I wrote it.  As the comment above it says, the intent was to quickly
crack NTLM hashes given already cracked LM ones.  There was no intent to
eliminate duplicates, because all 16384 combinations (at most) are
checked in a few milliseconds.  Well, perhaps with thousands of cracked
LM hashes, this can turn into a few seconds to crack the corresponding
NTLM hashes.

> So - 28675 possible combinations. The problem is this:
> 
> /john -w:a.dic --rules:nt -stdout | sort -u | wc
>      24      24     112
> 
> There are only 24 REAL combinations. So the --rules:nt wastes a lot of effort.
> Its not THAT big of a deal - but does waste a lot of CPU time on some large
> wordlists.

This was never meant to be used with arbitrary wordlists, let alone with
large ones.

> Is there a better way/rule to get all the possible capitalization combinations
> for a word list?

You can use this longer list of rules instead:

[List.Rules:NT]
:
T0Q
T1QT[*0]
T2QT[*0]T[*1]
T3QT[*0]T[*1]T[*2]
T4QT[*0]T[*1]T[*2]T[*3]
T5QT[*0]T[*1]T[*2]T[*3]T[*4]
T6QT[*0]T[*1]T[*2]T[*3]T[*4]T[*5]
T7QT[*0]T[*1]T[*2]T[*3]T[*4]T[*5]T[*6]
T8QT[*0]T[*1]T[*2]T[*3]T[*4]T[*5]T[*6]T[*7]
T9QT[*0]T[*1]T[*2]T[*3]T[*4]T[*5]T[*6]T[*7]T[*8]
TAQT[*0]T[*1]T[*2]T[*3]T[*4]T[*5]T[*6]T[*7]T[*8]T[*9]
TBQT[*0]T[*1]T[*2]T[*3]T[*4]T[*5]T[*6]T[*7]T[*8]T[*9]T[*A]
TCQT[*0]T[*1]T[*2]T[*3]T[*4]T[*5]T[*6]T[*7]T[*8]T[*9]T[*A]T[*B]
TDQT[*0]T[*1]T[*2]T[*3]T[*4]T[*5]T[*6]T[*7]T[*8]T[*9]T[*A]T[*B]T[*C]

In fact, it can be optimized even further, in terms of reducing the
number of duplicates:

[List.Rules:NT]
:
T0Q
T1Q
T1QMT0Q
T2Q
T2QMT1Q
T2QMT0Q
T2QMT0QMT1Q
T3QT[*1]T[*2]
T3QMT0QT[*1]T[*2]
T4QT[*1]T[*2]T[*3]
T4QMT0QT[*1]T[*2]T[*3]
T5QT[*1]T[*2]T[*3]T[*4]
T5QMT0QT[*1]T[*2]T[*3]T[*4]
T6QT[*1]T[*2]T[*3]T[*4]T[*5]
T6QMT0QT[*1]T[*2]T[*3]T[*4]T[*5]
T7QT[*1]T[*2]T[*3]T[*4]T[*5]T[*6]
T7QMT0QT[*1]T[*2]T[*3]T[*4]T[*5]T[*6]
T8QT[*1]T[*2]T[*3]T[*4]T[*5]T[*6]T[*7]
T8QMT0QT[*1]T[*2]T[*3]T[*4]T[*5]T[*6]T[*7]
T9QT[*1]T[*2]T[*3]T[*4]T[*5]T[*6]T[*7]T[*8]
T9QMT0QT[*1]T[*2]T[*3]T[*4]T[*5]T[*6]T[*7]T[*8]
TAQT[*1]T[*2]T[*3]T[*4]T[*5]T[*6]T[*7]T[*8]T[*9]
TAQMT0QT[*1]T[*2]T[*3]T[*4]T[*5]T[*6]T[*7]T[*8]T[*9]
TBQT[*1]T[*2]T[*3]T[*4]T[*5]T[*6]T[*7]T[*8]T[*9]T[*A]
TBQMT0QT[*1]T[*2]T[*3]T[*4]T[*5]T[*6]T[*7]T[*8]T[*9]T[*A]
TCQT[*1]T[*2]T[*3]T[*4]T[*5]T[*6]T[*7]T[*8]T[*9]T[*A]T[*B]
TCQMT0QT[*1]T[*2]T[*3]T[*4]T[*5]T[*6]T[*7]T[*8]T[*9]T[*A]T[*B]
TDQT[*1]T[*2]T[*3]T[*4]T[*5]T[*6]T[*7]T[*8]T[*9]T[*A]T[*B]T[*C]
TDQMT0QT[*1]T[*2]T[*3]T[*4]T[*5]T[*6]T[*7]T[*8]T[*9]T[*A]T[*B]T[*C]

As you can see, further optimizations increase the size of the input
wordlist rules.

For all-lowercase wordlists with purely alphabetic entries only (and in
some other cases), this should produce no duplicates at all.  With
password.lst included with JtR, this produces about 5% of duplicates.

A drawback of this approach is that some of the rules will reject almost
all input words in a typical wordlist, making JtR appear to "freeze".
Perhaps JtR's approach of applying the first rule to each word, then the
second rule to each word, and so on is undesirable for the many rules
produced by the above rule preprocessor expressions.  It would have been
better to apply all rules to the first word, then all rules to the
second word, and so on.  However, some CPU time would be wasted on
rejecting {word, rule} combinations that would result in duplicates
either way.

Possibly a better alternative would be to enhance JtR to allow for
filter() to request that the same input word be tried again (and
modified by filter() differently).  Then this case-toggling could be
implemented as an external mode for use in combination with the wordlist
mode (possibly even on top of other wordlist rules).  This is a fairly
minor modification to the JtR source code; I had it on my to-do list for
JtR for a while.

Alexander

-- 
To unsubscribe, e-mail john-users-unsubscribe@...ts.openwall.com and reply
to the automated confirmation request that will be sent to you.

Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.