Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <4D76BE86.5000302@bredband.net>
Date: Wed, 09 Mar 2011 00:40:54 +0100
From: magnum <rawsmooth@...dband.net>
To: john-dev@...ts.openwall.com
Subject: Re: --utf8 option, proof of concept

I have solved the fmt_tests problem (so a couple of utf8 tests kick in 
when --utf8 is used together with --test) and I just realised mscash and 
the other salted formats does NOT fully work with utf8 yet, because I 
have yet to properly convert the salt from utf8 too. I just need to 
digest the code a while.

On 03/08/2011 06:40 PM, jfoug wrote:
> The order of operations:
>
> methods.valid() and methods.split(), may be called prior to methods.init().
> No other function will be called prior to methods.init().  methods.init() is
> called one time only, or none, but if it is not called, nothing else in the
> format will be called.  Thus, methods.setkey() can NOT be called, prior to
> methods.init() being called.

Thanks. Could you please also very very briefly explain the rest of the 
"chain" from a salt and a candidate, to a crypt and cmp_*? Looking at 
the code it seems I should put the utf8 conversion in get_salt() while I 
would have guessed set_salt()... I'm confused.

> Also, within the format, within the fmt_main structure setup, you might
> simply initialize the set_key function pointer to the set_key_ansi function.
> Then within init, only have the 'if' side of it, and do not worry about the
> else, since by default, the format is already setup to use the set_key_ansi.

Yes, I dropped the _ansi from its name too. I enclose my current 
NT_fmt.c including utf8-aware fmt_tests. The latter is highest priority 
for all the other formats, testing both salt and plaintext with utf8 
encoding. Now I'm happy with NT_fmt.c except for optimisations and bugs.

Current benchmarks (not using --test which show a much worse hit than this):

$ ./john-unicode -fo:nt unicode-nt-hashes.txt -pot:testpot -inc -utf8
Loaded 100 password hashes with no different salts (NT MD4 [128/128 X2 
SSE2-16])
guesses: 0  time: 0:00:00:10  c/s: 1051M  trying: oouak - ooufe

$ ./john-unicode -fo:nt unicode-nt-hashes.txt -pot:testpot -inc
Loaded 100 password hashes with no different salts (NT MD4 [128/128 X2 
SSE2-16])
guesses: 0  time: 0:00:00:10  c/s: 1564M  trying: n515so - n515my

$ ./john-jumbo12 -fo:nt unicode-nt-hashes.txt -pot:testpot -inc
Loaded 100 password hashes with no different salts (NT MD4 [128/128 X2 
SSE2-16])
guesses: 0  time: 0:00:00:10  c/s: 1565M  trying: n5ao01 - n5ao1t

This is not that bad, and you'll only use this when you need it anyway. 
I hope to be able to completely mute the minor performance hit still 
there when not using the utf8 option. It should ideally perform like a 
non-patched version.

cheers,
magnum

View attachment "NT_fmt.c" of type "text/x-csrc" (20556 bytes)

Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.