[BP] MB-18580: Wait for VB state to be persisted before starting tests
Intermittent test failures (across multiple tests) have been seen
where we fail to read the number of items in vbucket disk file:
terminate called after throwing an instance of 'std::invalid_argument'
what(): CouchKVStore::getDbFileInfo: Failed to open database file for vBucket = 1 rev = 1 with error:no such file
The issue is that we do not correctly wait for the vBucket files to be
created before starting a test. We /attempt/ to wait in test_setup,
waiting for ep_vb_snapshot_total to be non-zero, however this stat is
not updated when vBuckets are written to disk, instead only when the
vb state snapshot occurs.
To fix this, create a new histogram stat - ep_vb_persist_state_total -
which records how long the actual persist takes (and counts then at
the same time). Change test_setup to check for this stat becoming 1
before continuing.
Results in two new stats:
* disk_persist_vbstate - timing histogram of how long vbState
operations took.
* ep_persist_vbstate_total - count of how many VBStatePersists have
occurred.
Change-Id: Ic24e6cdb51a98ea6fa65005158242bfcf44225d0
Reviewed-on: http://review.couchbase.org/65298
Reviewed-by: Dave Rigby <daver@couchbase.com>
Well-Formed: buildbot <build@couchbase.com>
Tested-by: buildbot <build@couchbase.com>