john-users - Finding fake hashes

Follow @Openwall on Twitter for new release announcements and other news

[<prev] [next>] [day] [month] [year] [list]

Date: Wed, 2 Jan 2013 04:43:34 +0100
From: magnum <john.magnum@...hmail.com>
To: "john-users@...ts.openwall.com" <john-users@...ts.openwall.com>
Subject: Finding fake hashes

Here's food for thought. I tried hacking raw-md5's valid() so it detects when all of the binary consists of printable characters (in other words: it may be just a hexified string and not an md5 hash at all). If so, print it as string:hash to stderr, and reject it.

Trying with raw-md5.hashes.txt from InfoSecSouthwest2012_Ripe_Hashes.tgz, it found 7835 suspects. Some of them might theoretically be real hashes but from looking at them, most are definitely not.

$ ./john test/hashes/raw-md5.hashes.txt -fo:raw-md5 2>reject.rawmd5
Loaded 139436660 password hashes with no different salts (Raw MD5 [128/128 SSE2 intrinsics 12x])

$ wc -l test/hashes/raw-md5.hashes.txt
 139444502 test/hashes/raw-md5.hashes.txt

$ wc -l reject.rawmd5 
    7838 reject.rawmd5
(three lines of John output are included in that figure)

Typical examples from the resulting file:

  SELECT COUNT(*:202053454c45435420434f554e54282a
" alt="FIFA logo:2220616c743d2246494641206c6f676f
"https://s3.amaz:2268747470733a2f2f73332e616d617a
$1$$xV3qpJrYJ7V3:2431242478563371704a72594a375633
$1$DNTu9/HA$dr2G:243124444e5475392f48412464723247
$1$EADn$ZQ2d3gY7:2431244541446e245a51326433675937
$1$RlxLyFYj$aBiS:243124526c784c7946596a2461426953
$1$cTpht$Obu9PLS:2431246354706874244f627539504c53
*0C0782863D010B2:2a304330373832383633443031304232
*356519AB98399BC:2a333536353139414239383339394243
*38756473D3CF4C6:2a333837353634373344334346344336
*6E194D5E188BD64:2a364531393444354531383842443634
*7A174D52C0AEDF1:2a374131373444353243304145444631
--><a href="mail:2d2d3e3c6120687265663d226d61696c
:"div#content_me:3a2264697623636f6e74656e745f6d65
Entrer votre PIN:456e7472657220766f7472652050494e
MousePosition;}e:4d6f757365506f736974696f6e3b7d65
Mozilla/5.0 (Mac:4d6f7a696c6c612f352e3020284d6163

Not sure what was the point other than I was right in thinking some of these hashes are in fact not md5 at all. Also, some forensics can be made from looking at this ;-)

A cooler approach would be to do entropy checks similar to ent(1) on the binary and see if you can sort some out with near certainity. This is probably better done outside of John though. Or should we have a --reject-low-entropy option? :-)

magnum

Download attachment "0001-Checks-for-printable-binaries-in-valid-and-if-they-a.patch" of type "application/octet-stream" (1637 bytes)

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.