john-dev - Re: USB-FPGA development

Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20161109221111.GA4267@openwall.com>
Date: Wed, 9 Nov 2016 23:11:11 +0100
From: Solar Designer <solar@...nwall.com>
To: john-dev@...ts.openwall.com
Subject: Re: USB-FPGA development

Hi Denis,

I am sorry I didn't reply to your questions earlier.

On Wed, Oct 12, 2016 at 05:13:51PM +0000, apingis@...nwall.net wrote:
> 2. Self-test.
> 2.1. For test array, I've generated several hashes with same salt and partial binaries. That is, on-device comparator is loaded with first 35 bits of hashes and they are the same, resulting in false positives. That cause self-test to fail. If that hashes are used for creation of password file then it works as expected, false positives successfully ruled out with cmp_exact() including the case where several false positives occur in one crypt_all() call.

I guess you mean these two, which you had to comment out because of the
issue above? -

        // These cause self-test to fail:
        // cmp_one() returns true and cmp_exact() returns false
        // because of partial binaries in the comparator
        //
        //{"35LSBeq.RUJA.", "==tCG*l2"},
        //{"35LSBeq.Xbkho", "==*]fyOo"},

> 2.2. "Warning: salt() returned misaligned pointer" self-test message. Format has 2-byte salt and I've set salt alignment to 2 bytes - is that correct or I should set salt_align to ARCH_WORD? Salt on host system is rarily accessed.

"is that correct or I should set salt_align to ARCH_WORD?" - neither.

You have:

static void *salt(char *ciphertext)
{
	static unsigned char out[2];

	int salt = DES_raw_get_salt(ciphertext);
	out[0] = salt;
	out[1] = salt >> 8;

	return out;
}

which doesn't provide any guaranteed alignment for its return value (the
"out" array).  Thus, to match this function above, you should set
SALT_ALIGN to 1.  This would be correct assuming that you don't depend
on the salts having any specific alignment in other places; if you do,
then you'd need to adjust salt() to provide that alignment and to set
SALT_ALIGN accordingly.  (But I think you're fine with no alignment and
SALT_ALIGN set to 1.)

> 2.3. I've implemented a warning when mask is too short and that results in performance degradation because USB 2.0 link has no enough bandwith. The warning appears during self-test and it looks confusing. How can format know when self-test is running, to suppress the warning? Is it planned to add usage of mask in self-test?

Oh, so this is why I saw that spurious warning.

I think yes, you can detect self-test, but I'd rather someone else
(more familiar with jumbo specific) answers this.  magnum, Jim?

> 3. FMT_REMOVE. How does format know when some binary was removed? If that's possible I'd prefer to keep comparator configuration until it actually changes - skipping unnecessary transfers to device would improve performance in case where only one salt is being audited. So far FMT_REMOVE is not implemented.

We added FMT_REMOVE retroactively, after Sayantan's OpenCL formats
started relying on salt->list being up to date, so that we wouldn't
break their assumption.  Thus, I think your specific question wasn't
actually discussed.  I think Sayantan just did it somehow.  My guess is
formats should be checking salt->count in order to detect whether
anything was removed or not without having to traverse the list first.
Looking at the code now, I see e.g. opencl_DES_bs_plug.c:
update_buffer() implement something like this, albeit with some
thresholds - perhaps that's to avoid too frequent updates of the GPU's
list of hashes (not after every hash cracked, but once in a while, not
to slow down cracking when a lot of hashes are getting cracked quickly).

I hope the above helps.

Now some requests from me:

Please get your Verilog sources and other source files required for
rebuilding the bitstream into the JtR source tree as well, in some
subdirectory.  They won't be used during normal JtR builds, but we'd
like to have the full source code in the same tree anyway.

Please add copyright and license statements to all of your source files.

If some of the files directly build upon others' work (and still contain
remnants of it sufficient to be subject to copyright), then please be
sure to include their copyright statements as well, and make sure their
work can be placed under the same license.  (That is, was already
licensed in the same or in a compatible way.)

Thanks,

Alexander
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.