john-dev - raw-md5-opencl does not crack some hashes

Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [day] [month] [year] [list]
Message-ID: <CABob6ioysPSEb3cQKsETbURDqm+Om7aXPdBzu38F4wCZw6bP=g@mail.gmail.com>
Date: Fri, 9 Aug 2013 06:14:11 +0200
From: Lukas Odzioba <lukas.odzioba@...il.com>
To: john-dev@...ts.openwall.com
Subject: raw-md5-opencl does not crack some hashes

Sayantan,all:
seems that our raw-md5-opencl is not working as it should (tested on
bleeding a583eb8dd) at least on 6950.
I ran a session trying to crack ~10k md5 hashes using mask mode and
this format missed 2 of them.
100k would be better, but I don't have so much patience to wait for completion.
Later I tried the same with incremental mode - same behaviour.
This means the problem is not in mask mode but in the format itself.

$ cat md5_10k | sort | uniq | wc -l
9999

6950:
$  ./john -mask=?l?l?l?l?l?l md5_10k -format=raw-md5-opencl
(...)
9997g 0:00:03:13 51.54g/s 1592Kp/s 1592Kc/s 8049MC/s zzvzug..CHECK

$ ./john -i:Lower6 md5_10k -format=raw-md5-opencl
(...)
9997g 0:00:03:10 DONE (2013-08-09 04:27) 52.53g/s 1623Kp/s 1623Kc/s
8083MC/s uwzcqn..CHECK

1 core of 3770k:
$ ./john -mask=?l?l?l?l?l?l md5_10k -format=raw-md5
(...)
9999g 0:00:00:12 807.6g/s 24949Kp/s 24949Kc/s 128564MC/s zzxxci..zzxxct

$ ./john -i:Lower6 md5_10k -format=raw-md5
(...)
9999g 0:00:00:10 DONE (2013-08-09 04:23) 920.7g/s 28442Kp/s 28442Kc/s
152521MC/s xvqsvn..xvqzvc

I used the following code to generate samples:

#include <stdio.h>
#include <stdlib.h>
#include <time.h>
#include <openssl/md5.h>
int main(int argc, char *argv[]){
	int i,j;
	char w[100];
	unsigned char digest[16];
	if(argc<3){
		printf("Usage: %s <hashes> <plaintext lengths>\n",argv[0]);
		return 1;
	}
	srand(time(NULL));
	for(i=0;i<atoi(argv[1]);i++){
		for(j=0;j<atoi(argv[2]);j++)
			w[j]=(random()%26)+'a';
		w[j]=0;
		MD5(w,j,digest);
		printf("%s:",w);
		for(j=0;j<16;j++)
			printf("%02x",digest[j]);
		puts("");
	}
	return 0;
}
$ gcc gen.c -o gen -lcrypto
$ ./gen 10000 6 >md5_10k

[Incremental:Lower6]
File = $JOHN/lower.chr
MinLen = 6
MaxLen = 6
CharCount = 26

Those two plaintexts thare were not cracked are:
$ cat crackme
ltwaat:e84a6a7673eb1fc965ee4af9e616220b
xdsrll:f0991036698615c3efc923f65466caef

Those two hashes are uncrackable on 6950:

$ ./john -i:Lower6 crackme -format=raw-md5-opencl
Device 0: Cayman (AMD Radeon HD 6900 Series)
Local worksize (LWS) 128, global worksize (GWS) 2097152
Loaded 2 password hashes with no different salts (Raw-MD5-opencl [MD5
OpenCL (inefficient, development use only)])
Press 'q' or Ctrl-C to abort, almost any other key for status
0g 0:00:00:06 DONE (2013-08-09 04:49) 0g/s 48117Kp/s 48117Kc/s
96235KC/s sdhugu..CHECK

But all 9999 are cracked fine on 5850:

$  ./john -i:Lower6 crackme -format=raw-md5-opencl -dev=1
Device 1: Cypress (ATI Radeon HD 5800 Series)
Local worksize (LWS) 128, global worksize (GWS) 1048576
Loaded 2 password hashes with no different salts (Raw-MD5-opencl [MD5
OpenCL (inefficient, development use only)])
Press 'q' or Ctrl-C to abort, almost any other key for status
ltwaat           (ltwaat)
xdsrll           (xdsrll)
2g 0:00:00:04 DONE (2013-08-09 04:49) 0.4784g/s 42896Kp/s 42896Kc/s
84287KC/s xdsrll..CHECK

$ ./john -i:Lower6 md5_10k  -format=raw-md5-opencl -dev=1
Device 1: Cypress (ATI Radeon HD 5800 Series)
Local worksize (LWS) 128, global worksize (GWS) 1048576
Loaded 9999 password hashes with no different salts (Raw-MD5-opencl
[MD5 OpenCL (inefficient, development use only)])
Press 'q' or Ctrl-C to abort, almost any other key for status
(...)
9999g 0:00:04:02 DONE (2013-08-09 04:55) 41.26g/s 1274Kp/s 1274Kc/s
6323MC/s uwzcqn..CHECK

This didn't look good so I tried on other devices:
570: 9997g (same 2 hashes as on 6950)
ltwaat
xdsrll

7979: 9996g
ltwaat
xdsrll
xvrswe

Things look even worse on 8120:
9931g 0:00:46:28 DONE (2013-08-09 08:04) 3.561g/s 110790p/s 110790c/s
557894KC/s ikvqvi..CHECK
That's 185x slower than raw-md5 format.

Other notes:
-currently raw-md5-opencl is way slower than cpu version in cracking
multiple hashes
-I noticed some segfaults while trying to break session by pressing
ctrl-c, or just enter to see status
-mask mode is slower than incremental (this test showed over 13%
difference on cpu)

Sayantan I would be happy if you could take a look at this and/or run
similar tests on other fast gpu formats.
We should perform such tests before next jumbo release anyway.

Lukas
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.