[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <20110829143624.GA10611@openwall.com>
Date: Mon, 29 Aug 2011 18:36:24 +0400
From: Solar Designer <solar@...nwall.com>
To: john-dev@...ts.openwall.com
Subject: Re: #include "john.conf2" (a wish list item)
Jim -
On Mon, Aug 29, 2011 at 07:33:56AM -0500, jfoug wrote:
> All changes so far are in config.c (h). However, loading a section while
> loading the conf file gets to be VERY ugly, because the section may not have
> been read yet. That is why I postponed the loading of the sections until
> used.
There's a simple solution to this: support including only previously
defined sections. That is:
[section1]
...
[section2]
.include [section1]
will work, but:
[section2]
.include [section1]
[section1]
...
will result in "Unknown section" error for the .include directive.
I think this is fine and maybe even preferable: users will know that an
included section may only be defined prior to the include, so looking
for it manually (while editing config files) is easier/quicker than it
would be if it were possible to include a not-yet-defined section.
> What we may be able to do, is loop to load the whole config, doing all the
> includes. Once that is done, then scan for the section includes, and
> populate them. I am not sure if we can walk the entire list generated from
> the config like that or not, but if so, that would be one pretty easy
> solution.
There's no need.
> I really do not think startup will suffer much at all. I highly doubt it
> will be really noticeable at all, unless the set of conf files gets very
> huge. The big items that cause the most time lag at startup, are loading
> multi MB/GB wordlists to memory, loading huge pot files, loading huge
> candidate files, and validating rules when you have 10's/100's of thousands
> of rules. That takes noticeable amounts of startup time. I doubt the .2s
> extra used including a few files/sections, will be noticed, much more than
> if all of that data was in the .conf file to start with. It may actually
> be faster to include a bunch of smaller rules sections into several
> 'super-rules', than to have several of these super rules inlined. I bet the
> section including will be faster than the original file IO.
I agree with you on the above.
> >actually in use during a specific invocation of JtR - but it simplifies
> >the program structure. To optimize this a little bit, you can copy
> >pointers to config file lines instead of making copies of the strings.
>
> What happens when a section gets included in more than one place? I am
> pretty sure we have to make new lists, and then link that data into the
> other list, like I am doing.
IIRC, each section gets its own linked list - either list of lines or
list of name=value pairs. When we include another section, we should
add that section's content to the current place in the including
section's linked list. No new lists need to be created, I think - but
I haven't actually tried implementing this yet, so I might be wrong.
> I had not made plans on including anything other than rules section data.
I think we shouldn't single out the rules. Implement the includes in a
generic fashion, for all sections at once, without even knowing what
top-level sections there may be.
> I will what was written about 'include' paths to make sure it is doing what
> you expect it to do. I wonder if adding a LIBPATH may not be a bad thing to
> do? Also, in jumbo, if the --conf=/path/file is used, should '/path/' be
> part of the search path? The user has already explicitly specified it as a
> path holding the john.conf file he wants loaded.
It feels like there's still a misunderstanding between us on this. Once
again, I think there should be no search path, with the only exception
being that $JOHN may result in checking two directories in some cases,
as I mentioned before. Aside from this exception, only one directory
should be checked for the included files in each case. If the file is
not found in that one directory, John should report an error and stop.
As to your question on the effect of --config - yes, it should have an
effect similar to what you describe, but not exactly affecting the
non-existent search path (I interpret the "search path" words as
implying potentially more than one directory to search in, whereas
there should be just one). I think I specifically addressed this in my
previous message.
Please let me know if anything is still unclear.
Thanks,
Alexander
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
