High CVSS 9.8 firefox Integer Overflow 🔧 Commit mapped

Overview

High
Severity
9.8
CVSS
No
Exploited ITW
Fixed
Fix Status
Impacthigh
DescriptionAn integer overflow was present in <code>OrderedHashTable</code> used by the JavaScript engine
ComponentSpiderMonkey
Bug ClassInteger Overflow
Tracker1970095
Fix commitd578778fe18e (firefox) +22/-13
CWECWE-190 (Integer overflow)
CVSS vectorCVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H
CISA KEVNot listed
CreditedShaheen Fazim
Disclosed2025-06-10

Background

`hashShift` / `kHashNumberBits`
the table sizes itself from a hashShift; the number of buckets is 1 << (kHashNumberBits - hashShift). A lower hashShift means more buckets and thus a larger table. kHashNumberBits is 32 here (so MinHashShift = 8 ⇒ MaxHashBuckets = 2^24 = 16777216).
`FillFactor`
ratio of data-array capacity to bucket count (~8/3), so data capacity ≈ buckets × FillFactor.
`allocateBuffer`
allocates the combined Data + bucket + scrambler buffer from dataCapacity and buckets; calcAllocSize computes the byte size.
`rehash`/grow path
when the table grows, it picks a smaller newHashShift, derives newHashBuckets and newCapacity, and allocates a new buffer.

Root Cause Analysis

On the grow path, the code derived the new table size by hand: newHashBuckets = uint32_t(1) << (js::kHashNumberBits - newHashShift) and newCapacity = uint32_t(newHashBuckets * FillFactor). The overflow guard was an indirect bound expressed via maxCapacityLog2 = FloorLog2<size_t(INT32_MAX / FillFactor)> and the test if (newHashShift < (kHashNumberBits - maxCapacityLog2)) ReportAllocationOverflow. This hand-rolled check was the integer-overflow hazard (CWE-190): the uint32_t(newHashBuckets * FillFactor) multiplication (and the downstream byte-size math) could overflow/produce a capacity that did not safely fit when the table was pushed to extreme sizes, because the guard bounded a log2 of a float-divided constant rather than directly bounding newHashBuckets and newCapacity against the real allocation limits.

The patch replaces the ad-hoc arithmetic and guard with explicit, named bounds and assertions. It adds constexpr helpers hashShiftToNumHashBuckets(hashShift) = 1 << (kHashNumberBits - hashShift) and numHashBucketsToDataCapacity(numHashBuckets) = uint32_t(numHashBuckets * FillFactor), and defines hard caps: MinHashShift = 8, MaxHashBuckets = hashShiftToNumHashBuckets(8) = 16777216, and MaxDataCapacity = numHashBucketsToDataCapacity(MaxHashBuckets) = 44739242. The grow path now rejects any newHashShift < MinHashShift (directly bounding the maximum table size) and computes newHashBuckets/newCapacity through the shared helpers. allocateBuffer gains MOZ_ASSERT(dataCapacity <= MaxDataCapacity) and MOZ_ASSERT(buckets <= MaxHashBuckets), and hashBuckets() is routed through the same helper so the whole table uses one consistent, bounded size computation. This removes the possibility of the capacity multiplication overflowing and yielding an undersized allocation for a table that then indexes beyond it. The advisory’s IntegerOverflow/CWE-190 label matches the diff exactly.

Key insight
The hash-table grow path computed newCapacity = newHashBuckets * FillFactor with only an indirect log2-based overflow guard, so an extreme newHashShift could overflow the capacity/byte-size arithmetic and produce an undersized allocation; the fix imposes explicit MinHashShift/MaxHashBuckets/MaxDataCapacity caps and asserts them at allocation time.

Attack Path

  1. Attacker script runs JavaScript that drives a Map/Set (or other OrderedHashTable consumer) to a very large number of entries, forcing repeated growth toward the maximum table size.
  2. Grow computes oversized shift the grow path reaches a newHashShift small enough that newHashBuckets × FillFactor overflows the uint32_t/byte-size computation (pre-fix the indirect guard failed to catch it).
  3. Undersized allocation allocateBuffer/calcAllocSize allocates a buffer smaller than the logical capacity implied by newHashBuckets.
  4. Out-of-bounds access subsequent hash/data indexing uses the inflated bucket/capacity values against the undersized buffer, reading/writing out of bounds.
  5. Primitive a heap out-of-bounds read/write in the JS engine driven from script; the advisory rates this as a critical memory-safety bug (controlled corruption/RCE is inferred, not shown by the diff).

Impact Assessment

This is a critical (CVSS 9.8, C/I/A:H) integer-overflow in a core SpiderMonkey data structure reachable from any web JavaScript (Map/Set), running in the content/renderer process. The consequence of the overflow is an undersized allocation leading to out-of-bounds access on a fundamental engine structure, a strong corruption primitive; however, the specific exploitation into code execution is the advisory’s assessment and is inferred rather than demonstrated by the diff. Reaching the overflow requires driving the table to near-maximal size (large memory/time), but needs no special privilege. This is pure integer arithmetic — GC rooting is not implicated.

Files Changed

  • js/src/builtin/OrderedHashTableObject.h

Audit Directions

  • Other size/capacity arithmetic in the engine
    audit js/src/builtin and ds/ hash/array containers for x * FillFactor / shift-based capacity math lacking a direct cap against the allocation limit.
  • `calcAllocSize` callers
    review every caller that passes a dataCapacity/buckets pair to confirm the new MaxDataCapacity/MaxHashBuckets invariants hold before allocation (the asserts are debug-only).
  • Assumed `kHashNumberBits`
    verify the MinHashShift = 8 cap is correct for all builds/architectures where kHashNumberBits might differ, so MaxHashBuckets/MaxDataCapacity remain safe.
diff --git a/js/src/builtin/OrderedHashTableObject.h b/js/src/builtin/OrderedHashTableObject.h
index 5fbb14309af..c971b308404 100644
--- a/js/src/builtin/OrderedHashTableObject.h
+++ b/js/src/builtin/OrderedHashTableObject.h
@@ -450,6 +450,14 @@ class MOZ_STACK_CLASS OrderedHashTableImpl {
     obj->setReservedSlotPrivateUnbarriered(Slots::HashCodeScramblerSlot, hcs);
   }
 
+  static constexpr uint32_t hashShiftToNumHashBuckets(uint32_t hashShift) {
+    return 1 << (js::kHashNumberBits - hashShift);
+  }
+  static constexpr uint32_t numHashBucketsToDataCapacity(
+      uint32_t numHashBuckets) {
+    return uint32_t(numHashBuckets * FillFactor);
+  }
+
   // Logarithm base 2 of the number of buckets in the hash table initially.
   static constexpr uint32_t InitialBucketsLog2 = 1;
   static constexpr uint32_t InitialBuckets = 1 << InitialBucketsLog2;
@@ -470,6 +478,13 @@ class MOZ_STACK_CLASS OrderedHashTableImpl {
   // If that ratio drops below this value, we shrink the table.
   static constexpr double MinDataFill = 0.25;
 
+  // Note: a lower hash shift results in a higher capacity.
+  static constexpr uint32_t MinHashShift = 8;
+  static constexpr uint32_t MaxHashBuckets =
+      hashShiftToNumHashBuckets(MinHashShift);  // 16777216
+  static constexpr uint32_t MaxDataCapacity =
+      numHashBucketsToDataCapacity(MaxHashBuckets);  // 44739242
+
   template <typename F>
   void forEachIterator(F&& f) {
     TableIteratorObject* next;
@@ -507,6 +522,9 @@ class MOZ_STACK_CLASS OrderedHashTableImpl {
       std::tuple<Data*, Data**, HashCodeScrambler*, size_t>;
   AllocationResult allocateBuffer(JSContext* cx, uint32_t dataCapacity,
                                   uint32_t buckets) {
+    MOZ_ASSERT(dataCapacity <= MaxDataCapacity);
+    MOZ_ASSERT(buckets <= MaxHashBuckets);
+
     size_t numBytes = 0;
     if (MOZ_UNLIKELY(!calcAllocSize(dataCapacity, buckets, &numBytes))) {
       ReportAllocationOverflow(cx);
@@ -1064,7 +1082,7 @@ class MOZ_STACK_CLASS OrderedHashTableImpl {
 
   /* The size of the hash table, in elements. Always a power of two. */
   uint32_t hashBuckets() const {
-    return 1 << (js::kHashNumberBits - getHashShift());
+    return hashShiftToNumHashBuckets(getHashShift());
   }
 
   void destroyData(Data* data, uint32_t length) {
@@ -1196,22 +1214,13 @@ class MOZ_STACK_CLASS OrderedHashTableImpl {
       return true;
     }
 
-    // Ensure the new capacity fits into INT32_MAX.
-    constexpr size_t maxCapacityLog2 =
-        mozilla::tl::FloorLog2<size_t(INT32_MAX / FillFactor)>::value;
-    static_assert(maxCapacityLog2 < kHashNumberBits);
-
-    // Fail if |(js::kHashNumberBits - newHashShift) > maxCapacityLog2|.
-    //
-    // Reorder |kHashNumberBits| so both constants are on the right-hand side.
-    if (MOZ_UNLIKELY(newHashShift < (js::kHashNumberBits - maxCapacityLog2))) {
+    if (MOZ_UNLIKELY(newHashShift < MinHashShift)) {
       ReportAllocationOverflow(cx);
       return false;
     }
 
-    uint32_t newHashBuckets = uint32_t(1)
-                              << (js::kHashNumberBits - newHashShift);
-    uint32_t newCapacity = uint32_t(newHashBuckets * FillFactor);
+    uint32_t newHashBuckets = hashShiftToNumHashBuckets(newHashShift);
+    uint32_t newCapacity = numHashBucketsToDataCapacity(newHashBuckets);
 
     auto [newData, newHashTable, newHcs, numBytes] =
         allocateBuffer(cx, newCapacity, newHashBuckets);
Loading diff…