[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <BLU0-SMTP1285D2D8AFF8EE96C5EB971FDFC0@phx.gbl>
Date: Tue, 25 Jan 2011 10:56:16 +0100
From: Frank Dittrich <frank_dittrich@...mail.com>
To: john-users@...ts.openwall.com
Subject: Re: Password file hash type analysis in JtR
Robert Harris wrote:
> I know that Unix/Linux and Windows password files can have multiple hash
> types in them, I think the following output/utility would be useful in
JtR.
What is your use case?
> Potential new output:
>
> The following password hash format(s) was/were discovered in the input
file
> <file>:
>
> <format1>
>
> <format2>
>
> ...
>
> <formatx>
Is that just one line describing the format?
As you would specify it on the command line (with --format=) or as john
would report it (DES vs. Traditional DES, BF vs. OpenBSD Blowfish, ...)?
> Line(s) 1-30 contained <format1> type passwords
>
> Line(s) 31-60 contained <format2> type passwords
>
> ...
>
> Line(s) 61-90 contained <formatx> type passwords
Without listing the corresponding hashes/passwords?
Depending on your needs, something like this might work:
sample pot file:
$LM$44efce164ab921ca:123456
10xCP/g3c1sQc:password
20ewVEw7R2GAw:password
10vrzSg43HfWI:123456
10xCP/g3c1sQc:password
10.6.TrD/0sas:trustno1
trEJ1uz6ul/WU:trustno1
$1$12$1YZMQPfPxS3.kStGKUBCO.:1111
e10adc3949ba59abbe56e057f20f883e:123456
$LM$b757bf5c0d87772f:1234
cut -d: -f 1 -s john.pot|grep -n "^" > cracked_hashes
cracked_hashes would then look like this:
1:$LM$44efce164ab921ca
2:10xCP/g3c1sQc
3:20ewVEw7R2GAw
4:10vrzSg43HfWI
5:10xCP/g3c1sQc
6:10.6.TrD/0sas
7:trEJ1uz6ul/WU
8:$1$12$1YZMQPfPxS3.kStGKUBCO.
9:e10adc3949ba59abbe56e057f20f883e
10:$LM$b757bf5c0d87772f
Rename your john.pot file before continuing.
(Actually, the jumbo-patched john has an option --pot=NAME,
but I never used it... That's why I just renamed john.pot)
The next step probably requires the jumbo patch, due to --show=LEFT.
(And, for our use case, it requires an empty pot file.)
for f in DES BSDI MD5 BF AFS LM; do ./john --format=$f --show=LEFT
cracked_hashes > cracked_hashes_$f; done
You may also write it across multiple lines, the shell will prompt you
for the continuation:
for f in DES BSDI MD5 BF AFS LM
> do
> ./john --format=$f --show=LEFT cracked_hashes > cracked_hashes_$f
> done
Check how the hashes are distributed according to hash type:
wc -l cracked_hashes_*
$ wc -l cracked_hashes_*
0 cracked_hashes_AFS
0 cracked_hashes_BF
0 cracked_hashes_BSDI
5 cracked_hashes_DES
4 cracked_hashes_LM
1 cracked_hashes_MD5
10 total
On first glance, this looks good.
However, I know I included 6 DES hashes (one duplicate).
2:10xCP/g3c1sQc
6:10xCP/g3c1sQc
Just 5 of these appear in cracked_hashes_DES, because the duplicate
has been removed.
(I'm not sure if this is unintended or not.
If a user wants to remove cracked hashes, but nevertheless
continue to use --single mode, keeping the duplicates might be better.)
So, where does the additional hash come from?
I included 2 LM hashes:
1:$LM$44efce164ab921ca
10:$LM$b757bf5c0d87772f
cracked_hashes_LM, however, contains (in addition to the expected content):
9:2:$LM$be56e057f20f883e
9:1:$LM$e10adc3949ba59ab
These 2 additional lines are the first and second half of the hash
e10adc3949ba59abbe56e057f20f883e in line 9 of my original
cracked_hashes file.
(While john stores LM hashes with a preceding $LM$ marker,
the algorithm internally accepts output generated by other tools.
That's why there's an ambiguity.)
e10adc3949ba59abbe56e057f20f883e is a raw-MD5 hash which didn't
appear in any cracked_hashes_* file, because I just used the format list
of the unpatched john.
Using the complete list:
$ for f in DES BSDI MD5 BF AFS LM NT XSHA PO raw-MD5 MD5-gen \
> IPB2 raw-sha1 md5a hmac-md5 phpass-md5 KRB5 bfegg \
> nsldap ssha openssha oracle oracle11 MYSQL \
> mysql-sha1 mscash lotus5 DOMINOSEC \
> NETLM NETNTLM NETLMv2 NETNTLMv2 NETHALFLM \
> mssql mssql05 epi phps mysql-fast pix-md5 sapG \
> sapB md5ns HDAA DMD5 crypt
> do
> ./john --format=$f --show=LEFT cracked_hashes > cracked_hashes_$f
> done
produces this output (to stderr):
Generic crypt(3) module: hash encoding string length 20, type id $L
appears to be unsupported on this system; will not load such hashes.
Generic crypt(3) module: hash encoding string length 32, type id #0
appears to be unsupported on this system; will not load such hashes.
And this list of files:
$ wc -l cracked_hashes_*
0 cracked_hashes_AFS
0 cracked_hashes_BF
0 cracked_hashes_bfegg
0 cracked_hashes_BSDI
6 cracked_hashes_crypt
5 cracked_hashes_DES
0 cracked_hashes_DMD5
0 cracked_hashes_DOMINOSEC
0 cracked_hashes_epi
0 cracked_hashes_HDAA
0 cracked_hashes_hmac-md5
0 cracked_hashes_IPB2
0 cracked_hashes_KRB5
4 cracked_hashes_LM
1 cracked_hashes_lotus5
1 cracked_hashes_MD5
0 cracked_hashes_md5a
0 cracked_hashes_MD5-gen
0 cracked_hashes_md5ns
1 cracked_hashes_mscash
0 cracked_hashes_mssql
0 cracked_hashes_mssql05
0 cracked_hashes_MYSQL
0 cracked_hashes_mysql-fast
0 cracked_hashes_mysql-sha1
0 cracked_hashes_NETHALFLM
0 cracked_hashes_NETLM
0 cracked_hashes_NETLMv2
0 cracked_hashes_NETNTLM
0 cracked_hashes_NETNTLMv2
0 cracked_hashes_nsldap
0 cracked_hashes_NT
0 cracked_hashes_openssha
0 cracked_hashes_oracle
0 cracked_hashes_oracle11
0 cracked_hashes_phpass-md5
0 cracked_hashes_phps
0 cracked_hashes_pix-md5
0 cracked_hashes_PO
1 cracked_hashes_raw-MD5
0 cracked_hashes_raw-sha1
0 cracked_hashes_sapB
0 cracked_hashes_sapG
0 cracked_hashes_ssha
0 cracked_hashes_XSHA
19 total
A mysteriously increased number of hashes?
$ sort -u cracked_hashes_*|wc -l
In total I have 12 different hashes.
Several hashes appear to be valid for multiple hash types.
$ sort cracked_hashes_*|uniq -c |grep -v "^ *1 "
2 2:10xCP/g3c1sQc
2 3:20ewVEw7R2GAw
2 4:10vrzSg43HfWI
2 6:10.6.TrD/0sas
2 7:trEJ1uz6ul/WU
2 8:$1$12$1YZMQPfPxS3.kStGKUBCO.
2 9:e10adc3949ba59abbe56e057f20f883e
These are the duplicates.
e10adc3949ba59abbe56e057f20f883e appears twice,
because it is valid for lotus5 and raw-MD5.
(Additionally, it has been stored in cracked_hashes_mscash as
9:M$9#e10adc3949ba59abbe56e057f20f883e.)
And crypt recognizes all hash types supported by your system, see
man 3 crypt
That means you could exclude crypt from the list of "hash formats",
unless you used john --format=crypt to crack hashes not currently
supported by an individual implementation (SHA-256 or SHA-512).
I didn't include any SHA-256 or SHA-512 hash.
After removing cracked_hashes_crypt, these hashes remain:
$ $ sort -n cracked_hashes_*
1:$LM$44efce164ab921ca
2:10xCP/g3c1sQc
3:20ewVEw7R2GAw
4:10vrzSg43HfWI
6:10.6.TrD/0sas
7:trEJ1uz6ul/WU
8:$1$12$1YZMQPfPxS3.kStGKUBCO.
9:1:$LM$e10adc3949ba59ab
9:2:$LM$be56e057f20f883e
9:e10adc3949ba59abbe56e057f20f883e
9:e10adc3949ba59abbe56e057f20f883e
9:M$9#e10adc3949ba59abbe56e057f20f883e
10:$LM$b757bf5c0d87772f
Line 5 of my original cracked_hashes cracked hashes file disappeared
(same hash as in line 2).
Line 9 produced 5 lines of "output", 4 of them unwanted:
The 2 lines in cracked_hashes_LM, one in cracked_hashes_mscash, and
either the one in cracked_hashes_lotus5 or the one in
cracked_hashes_raw-MD5.
Because the hash e10adc3949ba59abbe56e057f20f883e appears
unchanged in cracked_hashes_lotus5 and in cracked_hashes_raw-MD5,
the only way to find which is the right format is:
Try to crack the password again, using the contents of your original
john.pot file.
Before you continue working with john, don't forget to rename
the .pot file back to john.pot.
Unless I misunderstood your requirement, I think there's no need
to develop a new tool.
(On the other side, a tool could remove some of the ambiguity I
mentioned. But this could also be done with some more thought put
into some scripting.)
Regards, Frank
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
