Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [day] [month] [year] [list]
Message-ID: <4E3095DF.1060407@bredband.net>
Date: Thu, 28 Jul 2011 00:49:03 +0200
From: magnum <rawsmooth@...dband.net>
To: john-dev@...ts.openwall.com
Subject: Full UTF-8 support (as opposed to UCS-2)

I just uploaded a "0012" patch. This patch re-enables full support for 
UTF-16 (as opposed to UCS-2). This “surrogate pair” support [meaning two 
UTF-16 words are used to compose one character above U+FFFF] was removed 
during development of the initial “UTF-8 patch” because of NT 
performance. Since then, NT and mscash (dcc1) got their own, inline, 
utf8-straight-into-keybuffer functions (which still are UCS-2 after this 
patch, for performance reasons) so we could now add the full support 
again for all other formats.

The added (17x larger) Unicode space may not contain that much of 
interest but there are some things that may be expected in passwords, 
for example emoji symbols (very commonly in Japan and now spreading) and 
some CJK characters "important for personal and place names".

I enclose a test file and dict for use with raw-md5-unicode or 
md5_gen(29), and of course --enc=utf8. This file contains the character 
U+1033C, "GOTHIC LETTER MANNA" in combinations up to (and over) the max 
length of the format. I picked that random character because it's 
printed correctly on my Ubuntu laptop. There are 20 hashes but only 16 
are expected to be cracked because the other 4 is beyond max length. 
This test file can *not* be cracked without applying the 0012 patch.

magnum

View attachment "manna.rawmd5u" of type "text/plain" (2045 bytes)

View attachment "high_unicode.dict" of type "text/plain" (794 bytes)

Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.