From: "Jin, Gordon" This patch fixes 2 cornercases of overflow caused by argument len in sys_mincore(): Case 1: len is so large that will overflow to 0 after page alignment. E.g. len=(size_t)(-1), i.e. 0xff...ff. Expected result: it's overflow and return ENOMEM. Current result: len is aligned to 0, then treated the same as len=0 and return succeed. This cornercase has been fixed in do_mmap_pgoff(), and here sys_mincore() also needs this fix. Case 2: len is a large number but will not overflow after alignment. But start+len will overflow. E.g. len=(size_t)(-PAGE_SIZE), and start>0. Expected result: it's overflow and return ENOMEM. Current result: return EINVAL. Looks like considering len as a non-positive value, probably influenced by manpage. But since the type of len is size_t, i.e. unsigned, it shouldn't be considered as non-positive value. I've also reported this inconsistency to manpage mincore. Signed-off-by: Gordon Jin Signed-off-by: Andrew Morton --- 25-akpm/mm/mincore.c | 10 +++++++--- 1 files changed, 7 insertions(+), 3 deletions(-) diff -puN mm/mincore.c~fix-mincore-cornercases-overflow-caused-by-large-len mm/mincore.c --- 25/mm/mincore.c~fix-mincore-cornercases-overflow-caused-by-large-len 2005-01-25 22:22:20.345211424 -0800 +++ 25-akpm/mm/mincore.c 2005-01-25 22:22:20.349210816 -0800 @@ -97,8 +97,7 @@ static long mincore_vma(struct vm_area_s * return values: * zero - success * -EFAULT - vec points to an illegal address - * -EINVAL - addr is not a multiple of PAGE_CACHE_SIZE, - * or len has a nonpositive value + * -EINVAL - addr is not a multiple of PAGE_CACHE_SIZE * -ENOMEM - Addresses in the range [addr, addr + len] are * invalid for the address space of this process, or * specify one or more pages which are not currently @@ -114,13 +113,18 @@ asmlinkage long sys_mincore(unsigned lon int unmapped_error = 0; long error = -EINVAL; + if (!len) + return 0; + down_read(¤t->mm->mmap_sem); if (start & ~PAGE_CACHE_MASK) goto out; + + error = -ENOMEM; len = (len + ~PAGE_CACHE_MASK) & PAGE_CACHE_MASK; end = start + len; - if (end < start) + if (end <= start) /* overflow */ goto out; error = -EFAULT; _