john-users - raw-sha1-opencl patch 1

Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Date: Sun, 16 Jan 2011 19:45:42 +0100
From: Samuele Giovanni Tonon <samu@...uxasylum.net>
To: john-users <john-users@...ts.openwall.com>
Subject: raw-sha1-opencl patch 1

hello,
attached here patch 1 for raw-sha1 with many improvements:
- added fix for nvidia cl compiler now it should work on both ati and 
  nvidia (not tested on nvidia atm)
- fixed some faulty loops of previous version due to my 
  misunderstanding of crypt_all() and cmp_all() calls
- moved prepare of sha1 msg from cpu to GPU saving some data transfer

what still need to do:
- some more optimization of both cl and C code
- add md5 cl hack to pass 1/4 (in sha1 case 1/5) of the hash back to jtr
  and send the whole hash only for cmp_one

how to add to jtr:
add the patch to a vanilla jtr 1.7.6 *after* adding jumbo-9-patch and
opencl-9 patch .

benchmark:
I took a sample dictionary file and encoded sha1 twice with a simple
python program:

#!/usr/bin/python
import sys
import hashlib

fp = file(sys.argv[1],'r')
fw = file(sys.argv[2],'w')
for line in fp.readlines():
        x= hashlib.sha1()
        y= hashlib.sha1()
        x.update(line.strip())
        #print "%s"%line.strip()
        y.update(x.hexdigest())
        yy = y.hexdigest()
        fw.write(yy)
        fw.write("\n")

fp.close()
fw.close()

this to make an "unbreakable" hashfile to test with


$../run/john --format=raw-sha1 ~/ppp       
Loaded 9000 password hashes with no different salts (Raw SHA-1
[raw-sha1])
guesses: 0  time: 0:00:09:52 (3)  c/s: 29179M  trying: Momesoo
guesses: 0  time: 0:00:10:04 (3)  c/s: 29185M  trying: cafespe1
guesses: 0  time: 0:00:10:05 (3)  c/s: 29185M  trying: cbmteapp
Session aborted
$../run/john --format=raw-sha1-opencl ~/ppp       

OpenCL Platform: <<<ATI Stream>>> and device: <<<Juniper>>>
Loaded 9000 password hashes with no different salts (Raw SHA-1 OpenCL
[SHA-1])
guesses: 0  time: 0:00:02:24 (3)  c/s: 3786M  trying: trckfox7 -
tomcltco
guesses: 0  time: 0:00:09:26 (3)  c/s: 4037M  trying: padhco49 -
phouiouw
guesses: 0  time: 0:00:10:06 (3)  c/s: 4034M  trying: 00SW091 - 0ryohlm
guesses: 0  time: 0:00:10:09 (3)  c/s: 4033M  trying: pldiftt - assh26t


so still 6 times slower than raw sha1 cpu version but..
gpu wasn't used very much: i could still see my desktop with no load at
all, when i benchmark pyrit my gpu hangs for some seconds .

i've tried changing SHA_BLOCK - which is the fixed size i send to the
gpu for each password - from 16 to 64 . performance decreased by 20% no
more so it's no more a matter of low work_size and data exchange.

Cl code can be optimized, for example the prepare_msg routine 
is ugly, some Opencl reading could help a bit.


Regards
Samuele 



-- 
While various networks have become deeply rooted, and thoughts have been
sent out as light and electrons in a singular direction, this era has
yet to digitize/computerize to the degree necessary for individuals to
become a singular complex entity.
  KOUKAKU KIDOUTAI Stand Alone Complex


View attachment "john-1.7.6-jumbo-9-opencl-9-sha1-1.diff" of type "text/x-patch" (17953 bytes)
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.