[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <1363875193.51019.YahooMailNeo@web171504.mail.ir2.yahoo.com>
Date: Thu, 21 Mar 2013 14:13:13 +0000 (GMT)
From: Costin Enache <e_costin@...oo.com>
To: "john-dev@...ts.openwall.com" <john-dev@...ts.openwall.com>
Subject: Re: err in rules processor and dirty fix [rules.c]
>> So 0x100 is not enough even if we did not have the hex encoding? Your
rules must be extreme! Anyway, I think even core John should emit a
warning when rules get truncated (should be possible to catch outside
the performance critical code).
Not really: I have an unknown selection of characters
from ISO-8859-1, say 97 bytes, c1 ... c97, machine generated, some may be
consecutive, some not. Let's assume we do not need hex encoding. I need a rule
to append all this characters, three times to each word in the wordlist, covering
all the combinations:
$[c1-c97]$[c1-c97]$[c1-c97]
(BTW, I will NOT use Az"[c1-c97] [c1-c97] [c1-c97]"
as \x22, should it be amongst the c1-c97, will be decoded as double quotes, will
be interpreted by the pre-processor, and will mess up everything(bug? should we escape escaped hex chars?). The same does
not happen for [], lucky me.)
I will end up with 100x3=300 characters per rule line.
This is a conservative example. To be on the safe side, 0x400-1 will be an
uncompressed range, hex encoding (maybe the order the characters are tried matters for
some). Say we append 4 characters, makes it ~0x1000++. I also think 0x5000 is
heavily exaggerated, but some complex rules could get really long. I know have the buffer size at 0x1000 in params.h, with the 4x patch in rules.c. This means an effective rule text line length of max 4096.
And yes, John silently truncating rules was a nasty and unexpected surprise :) Warnings are welcome.
I guess that the best alternative would be to change the pre-processor to check and reject invalid rules in a separate function, with an appropriate buffer. Everything else, up to the reject check, works fine. The expanded rules are also checked fine by the aforementioned function.I may also be overlooking a lot and drifting in the wrong direction here :)
Costin
________________________________
From: magnum <john.magnum@...hmail.com>
To: john-dev@...ts.openwall.com
Sent: Wednesday, 20 March 2013, 20:53
Subject: Re: [john-dev] err in rules processor and dirty fix [rules.c]
On 20 Mar, 2013, at 19:04 , Costin Enache <e_costin@...oo.com> wrote:
> The 4x change solves the hex encoding, but I don't think it's enough. The rules_reject function is used for both expanded and unprocessed data. For expanded rules, the size is ok, and with the 4x increase, it fits also hex values. However, the same RULE_BUFFER_SIZE must fit (fortunately, only to check for invalid rules) the unprocessed ones, which can be very long, mostly because of character ranges.
Even a list of the full 1-255 range, fully encoded in hex and not using ranges to compress it, can't be longer than 0x400. I can't dream of a line that needs 0x5000 like you write below:
> Take my case: I need to prepend and append machine-generated characters, and I am too lazy to check what is "common" and what is "special" character, so the easiest way is to encode all of them to hex representation. It is very easy to reach the buffer limit. As a temporary solution, I have increased the RULE_BUFFER_SIZE value in params.h, to 0x5000. This is overkill for already processed/ expanded rules. We could have two "rules_reject" functions, one of expanded and one for unprocessed rules ... not sure this is ok.
So 0x100 is not enough even if we did not have the hex encoding? Your rules must be extreme! Anyway, I think even core John should emit a warning when rules get truncated (should be possible to catch outside the performance critical code).
> I'd avoid patching for now, and it would be better if the original author of the rules parsing code could have a look into this, and see what needs to be changed (otherwise we might overlook some implication of the changes, get funny errors later, or waste memory on already processed rules).
Unfortunately the original author is Solar while the hex encoding magic was made by Jim and me (I added missing stuff for range support). Solar don't usually go out of his way to solve Jumbo bugs - even if he wanted to, he doesn't have the time. On a good day he'll give us a clue off the top of his head. This situation is not too uncommon in Jumbo: Usually we fix issues one at a time as we find them :-/
I think we should either 1) apply the fix you posted, and this will do until we know anything else specific. Or 2) we just bump RULE_BUFFER_SIZE to 0x400 instead. That size ought to be enough for sane use ;-) and maybe it benefits some other situations than just hex encodings.
Solar, any thoughts?
magnum
Content of type "text/html" skipped
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
