[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <20100319221931.GA16985@openwall.com>
Date: Sat, 20 Mar 2010 01:19:31 +0300
From: Solar Designer <solar@...nwall.com>
To: john-users@...ts.openwall.com
Subject: Re: avoid duplicates with rules
On Fri, Mar 19, 2010 at 07:57:31PM +0100, websiteaccess@...il.com wrote:
> I have in my wordlist 2 words :
> justine
> justone
[...]
> my john.conf is now
>
> $1
> /i si[aeouy]$[0-9]
>
> As you can see, the 2 differents rules will generate 2 times the same word
Yes, "justone1" will be generated twice - the first time by applying the
"$1" rule to "justone" and the second time by applying the "/i sio $1"
rule to "justine".
> I want keep both rules, how can I avoid duplicate words ?
You can't. Well, if you really have to (e.g., if your hashes are of a
slow to compute type), you'll have to run "--stdout" through "unique".
> There is a special command in JTR's rule to avoid duplicates, but i'm
> not very comfortable with these.
There are many rule commands that are not "special", but that can be
used to avoid or reduce duplicates. However, this only works for
duplicates that would result from application of different rules to the
same input word, not for those resulting from similar input words.
For this reason, the default rulesets supplied with JtR _do_ produce
some duplicates on some wordlists, although they would have produced
many more duplicates if they did not try to avoid those resulting from
application of different rules to the same input word (they do).
For example, the default wordlist mode ruleset of JtR 1.7.5 run against
password.lst included with that version of JtR with "--stdout" (and no
length limit) produces 141,273 candidate passwords, of which only
133,871 are unique (that's 94.8%). However, converting password.lst to
all-lowercase and then sorting it alphabetically reduces these numbers
to 134,476 and 133,867, respectively (99.5% unique), which is why such
conversion is recommended for large and not-so-targeted wordlists.
Alexander
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
