[klibc] calloc: Fail if multiplication overflows

calloc() multiplies its 2 arguments together and passes the result to
malloc().  Since the factors and product both have type size_t, this
can result in an integer overflow and subsequent buffer overflow.
Check for this and fail if it happens.

CVE-2021-31870

Signed-off-by: Ben Hutchings <ben@decadent.org.uk>

1 files changed, 8 insertions, 3 deletions

diff --git a/usr/klibc/calloc.c b/usr/klibc/calloc.c
index 53dcc6b2f6bf63..4a81cda15e1ce2 100644
--- a/usr/klibc/calloc.c
+++ b/usr/klibc/calloc.c
@@ -2,12 +2,17 @@
  * calloc.c
  */
 
+#include <errno.h>
 #include <stdlib.h>
 #include <string.h>
 
-/* FIXME: This should look for multiplication overflow */
-
 void *calloc(size_t nmemb, size_t size)
 {
-	return zalloc(nmemb * size);
+	unsigned long prod;
+
+	if (__builtin_umull_overflow(nmemb, size, &prod)) {
+		errno = ENOMEM;
+		return NULL;
+	}
+	return zalloc(prod);
 }