[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <57014235.5060707@openwall.net>
Date: Sun, 3 Apr 2016 11:17:57 -0500
From: jfoug <jfoug@...nwall.net>
To: john-users@...ts.openwall.com
Subject: Re: New hybrid external elite mode
On 4/3/2016 9:52 AM, Rich Rumble wrote:
> A lot of this can be done with some better rules, see and old thread I
> started in 2010: http://www.openwall.com/lists/john-users/2010/08/02/8
> I wanted to leet replace each possible cobmination: remember ->
> r3member-> r3m3mber -> r3m3mb3r I haven't tried this external mode
> yet, but does it do the same thing? -rich
Yes, I listed that a lot of this 'can' be done with rules, but it is
very inefficient to do this using rules. You have to run your wordlist
through the rules sets 1000's of times, so produce the proper words. On
each run through, the rule will throw away almost every word, with the
exception of the 'few' that have the specific letters in the exact
locations you want.
Take a look at this new external mode. It is pretty fast. I also wrote
a case mangling external script. For a baseline speed check, I did a
comparison with it against rexgen code doing case conversion (which now
has to be done using replacement alphabets, since case folding was
removed from the library). When used in an actual production run
(raw-md5 against a single hex line), the external case was getting about
9.3m/s while the rexgen was getting about 4.6m/s, so this external mode
is running pretty fast.
The external leet code will likely run MUCH faster than multiple leet
rules, since it is not wasting a lot of time disqualifying words. It
simply takes each word as it comes over, and builds the transform set,
and then properly permutes the next word at each step.
The external hybrid mode is a very powerful word mutation engine. It can
also be pretty fast, if scripts are written properly. For transforms,
where 1 word will be converted into 10's or 100's or 1000's of final
words, if the next() function is written to be very thin and fast, then
overall the script will run very fast.
For another qwik-n-dirty speed test, I wanted to test against rules,
where I KNOW the rules are efficient. So I tested with a simple append
000 to 999 (1000 items), and ran it using raw-md5 against a 20m
wordlist. I then re-ran using mask (which should be faster than rules),
again just to get a baseline
$ ../run/john -rules=:'$[0-9]$[0-9]$[0-9]' in -form=raw-md5
-w=/c/phpbb/dictionaries/mc-allpass.txt
0g 0:00:03:32 DONE (2016-04-03 10:23) 0g/s 9215Kp/s 9215Kc/s 9215KC/s
▒▒▒▒▒▒▒▒▒999..▒▒▒▒▒▒▒▒▒▒▒999
$ ../run/john -extern=app_3_nums in -form=raw-md5
-w=/c/phpbb/dictionaries/mc-allpass.txt
0g 0:00:04:11 DONE (2016-04-03 11:05) 0g/s 7811Kp/s 7811Kc/s 7811KC/s
▒▒▒▒▒▒▒▒▒▒▒920..▒▒▒▒▒▒▒▒▒▒▒999
$ ../run/john -mask=?w?d?d?d in -form=raw-md5
-w=/c/phpbb/dictionaries/mc-allpass.txt
0g 0:00:01:53 DONE (2016-04-03 10:53) 0g/s 17234Kp/s 17234Kc/s 17234KC/s
▒▒▒▒▒▒▒▒▒▒▒127..▒▒▒▒▒▒▒▒▒▒▒777
One BIG difference, is that rules produces 1000 rules. Each word is
given to ONE rule until all words are used, and then all words are given
to the next rule, etc. In the external mode, a word is sent to the
engine, and then ALL candidates for that word are produced one after the
other, then the next word is done. -mask mode works in the same way
that the external-hybrid engine does (depth first).
Btw, here is the external script. It could have been written to be a
little faster, but I wrote it quickly, and to produce candidates in the
same order as the rules mode produces. Well, it produces the numbers
in the same order.
[List.External:app_3_nums]
int len;
int val;
void init() {/*do nothing*/}
void new() {
val=len=0; while (word[len++]) ; ++len;
/* note, len is pointing 3 chars past the word */
word[len+1] = 0;
}
void next() {
word[len] = '0'+val%10;
if (val%10==0) {
word[len-1] = '0'+(val/10)%10;
if (val%100==0) word[len-2] = '0'+val/100;
}
if (++val > 1000) word[0] = 0;
}
PS, I was able to rewrite the script, and get it to be almost as fast as
the rule.
$ ../run/john -extern=app_3_nums in -form=raw-md5
-w=/c/phpbb/dictionaries/mc-allpass.txt
0g 0:00:03:47 DONE (2016-04-03 11:16) 0g/s 8617Kp/s 8617Kc/s 8617KC/s
▒▒▒▒▒▒▒▒▒▒▒920..▒▒▒▒▒▒▒▒▒▒▒999
[List.External:app_3_nums]
int len;
int val;
void init() {/*do nothing*/}
void new() {
val=len=0; while (word[len++]) ; ++len;
word[len-2] = '0'-1;
word[len-1] = '0';
word[len] = '0';
word[len+1] = 0;
}
void next() {
++word[len];
if (val%10==0) {
word[len] = '0';
++word[len-1];
if (val%100==0) {
word[len-1] = '0';
++word[len-2];
}
}
if (++val > 1000) { word[0] = 0; return; }
}
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
