kernel-hardening - Re: kmalloc() nofail allocations

Follow @Openwall on Twitter for new release announcements and other news

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20110822092429.GA29597@albatros>
Date: Mon, 22 Aug 2011 13:24:29 +0400
From: Vasiliy Kulikov <segoon@...nwall.com>
To: kernel-hardening@...ts.openwall.com
Subject: Re: kmalloc() nofail allocations

Solar,

Major problem with the idea in general:

static inline struct page *
__alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
	struct zonelist *zonelist, enum zone_type high_zoneidx,
	nodemask_t *nodemask, struct zone *preferred_zone,
	int migratetype)
{
    ...
	/* Avoid allocations with no watermarks from looping endlessly */
	if (test_thread_flag(TIF_MEMDIE) && !(gfp_mask & __GFP_NOFAIL))
		goto nopage;
    ...
}

Unlikely, but fully possible case - we want some memory and the OOM killer
kills us for our wish.

Also there are some flags, which I didn't take into consideration in the
checks of the previous patch, like __GFP_WAIT, but these are solvable
problems.

We cannot use __GFP_NOFAIL as it is marked as a legacy flag, which
shouldn't be used in a new code.  Otherwise, we'd return the same old
behaviour as before.

However, now I want to bring the subject to LKML (it starts to be
interesting) to clarify (and document, etc.) the allocation bahaviour,
when it should be cheched for NULL, whether these are really "nofail"
allocations, etc.

Thanks,

-- 
Vasiliy

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.