[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <9240d141764a2ce5d13bcf1df32c9fd0@smtp.hushmail.com>
Date: Fri, 17 Aug 2012 06:04:17 +0200
From: magnum <john.magnum@...hmail.com>
To: john-dev@...ts.openwall.com
Subject: Re: RAR early reject
Milen,
Thanks to JimF's pkzip code and some crucial hints from Pavel Semjanov,
I finally nailed the early reject we wanted! Have a look at this:
commit 541ee9fee8f785179b9757a79a7b7d8d9ed6a6cf
Author: magnum <john.magnum@...com>
Date: Fri Aug 17 04:33:16 2012 +0200
"RAR: Finally, quite effective early rejection! Verified with almost
70,000 test files, with no false rejects. And we are rejecting over 96%
of the candidates without resorting to a slow full check (which in turn
may reject semi-early)"
We can now crack -p archives at virtually the same speed as -hp ones,
almost regardless of size. Now I'll just need to find out why my kernels
run at half the speed they should. I tried using Sayantan's SHA1 code,
but there was no difference at all. I need to do something really clever
with the key stretching loop.
magnum
On 2012-05-16 00:21, Milen Rangelov wrote:
> True.
>
> If I understand correctly, it should never go beyond 63 as no rar archiver
> I've seen allows it. I think a similar check can be done for maxmb (winrar
> imposes a limit of 128MB). This helps. But we definitely need some early
> reject in the LZ case.
>
> On Wed, May 16, 2012 at 12:59 AM, magnum <john.magnum@...hmail.com> wrote:
>
>> On 05/15/2012 09:05 AM, Milen Rangelov wrote:
>>> I am reaching the same conclusion. I was able to do another check for the
>>> PPM part, order should never be more than 63 (as this is the archiver
>>> limit) and this is safe I think.
>>
>> Do you mean max_order in ppm_decode_init() right after reading it from
>> rar_get_char()? Or did you test somewhere else?
>>
>> I also found from original unpack.cpp that filter_pos in add_vm_code()
>> should never be > 1024 but I haven't ever seen it happen. Maybe I'll
>> drop that test again even though it's there in the original code (for
>> some reason it was not in clamav's code). BTW I only recently realized
>> how easy it is to compare the original .cpp code with clamav's code.
>> They really just ported it. The original code has more comments.
>>
>> We are currently rejecting 93% of the data on average, compared to
>> reading all of the data each time. That may sound good, but for a 500 MB
>> file we still need to decrypt and read 3.6 MB on average.
>>
>> magnum
>>
>>
>
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
