MB-24055: Reduce HashTable::defaultNumBuckets from 1531 to 3

The variable HashTable::defaultNumBuckets configures the number of
HashTable slots to use by default; assuming ns_server doesn't specify
a value in the bucket config. This was previously 1531. Given we
normally have 1024 vBuckets, this creates 1.51M slots in all the
HashTables (assuming no value from ns_server).

For small buckets this adds a lot of unnecessary overhead -
particulary comparing Ephemeral to memcache buckets, where their empty
size (single node) is:

    Bucket             RSS (KB)
    <none>             40096
    memcache           40260
    Ephemeral          87504

This first needs to be changed in ns_server (see linked patch) so it
no longer specifies the default (previously 3079); however once that
is done we still need to reduce the ep_engine default.

This patch reduces it down to the minimum of 3. Note that's still 3072
slots by default per bucket; and:

a) This doesn't limit how many Docuements can be stored, on Hash
   collision we simply use internal chaining.

b) The HashTableResizer task runs periodically to select the the
   'optimum' size, so we'll quickly adjust to larger sizes.

After this change, the empty size of an Ephemeral bucket is reduced by
30MB:

    Bucket             RSS (KB)
    Ephemeral(size 3)  57800

Change-Id: I94ce68cf3fbc2dfe70690fe8b18fc0dbada0848d
Reviewed-on: http://review.couchbase.org/77266
Reviewed-by: Manu Dhundi <manu@couchbase.com>
Tested-by: Build Bot <build@couchbase.com>

diff --git a/src/hash_table.cc b/src/hash_table.cc
index 5e8c400..8bc1431 100644 (file)
--- a/src/hash_table.cc
+++ b/src/hash_table.cc
@@ -21,11 +21,10 @@
 
 #include <cstring>
 
-#ifndef DEFAULT_HT_SIZE
-#define DEFAULT_HT_SIZE 1531
-#endif
-
-size_t HashTable::defaultNumBuckets = DEFAULT_HT_SIZE;
+// Given we default to 1024 vBuckets, set default HT buckets to lowest
+// prime in our table - this still gives space for 3072 HT slots but
+// minimizes fixed overheads.
+size_t HashTable::defaultNumBuckets = 3;
 size_t HashTable::defaultNumLocks = 193;
 
 static ssize_t prime_size_table[] = {

diff --git a/src/vbucket.cc b/src/vbucket.cc
index 85c9094..dc3b6d9 100644 (file)
--- a/src/vbucket.cc
+++ b/src/vbucket.cc
@@ -1834,6 +1834,7 @@ void VBucket::_addStats(bool details, ADD_STAT add_stat, const void* c) {
         addStat("ht_memory", ht.memorySize(), add_stat, c);
         addStat("ht_item_memory", ht.getItemMemory(), add_stat, c);
         addStat("ht_cache_size", ht.cacheSize.load(), add_stat, c);
+        addStat("ht_size", ht.getSize(), add_stat, c);
         addStat("num_ejects", ht.getNumEjects(), add_stat, c);
         addStat("ops_create", opsCreate.load(), add_stat, c);
         addStat("ops_update", opsUpdate.load(), add_stat, c);

diff --git a/tests/ep_testsuite.cc b/tests/ep_testsuite.cc
index 3f61630..98be806 100644 (file)
--- a/tests/ep_testsuite.cc
+++ b/tests/ep_testsuite.cc
@@ -6319,6 +6319,7 @@ static enum test_result test_mb19687_fixed(ENGINE_HANDLE* h,
                 "vb_0:ht_cache_size",
                 "vb_0:ht_item_memory",
                 "vb_0:ht_memory",
+                "vb_0:ht_size",
                 "vb_0:logical_clock_ticks",
                 "vb_0:max_cas",
                 "vb_0:max_cas_str",