MB-18453: Give all tasks their own stats and priority 53/65253/10
authorJim Walker <jim@couchbase.com>
Sun, 26 Jun 2016 19:52:10 +0000 (20:52 +0100)
committerDave Rigby <daver@couchbase.com>
Mon, 4 Jul 2016 09:03:47 +0000 (09:03 +0000)
commit6403bc0e8bbd7e94bb03672f505d99ff68d56c36
tree090d3ac2ee646245ea4eb8bec494ecbbe2229bf1
parent7d9dc5d4bb21cc80cc91b5496a6c30a30c0a5f08
MB-18453: Give all tasks their own stats and priority

MB-18453 identified that tasks have copied & pasted
constructors which leads to some tasks all having the
same Priority object.

The fallout of this is that many tasks now all contribute
to the same histogram for runtime and scheduling waittime.
When debugging issues which lead to MB-18453 it is near
impossible at times to know which real task was delayed
as the stats can be attributed to many tasks.

This commit introduces makes all tasks have their own ID
and thus their own histograms and also makes it easier
for new tasks to be created without forgetting to create
a new Priority instance.

tasks.defs.h is a new file that captures every sub-class
of GlobalTask and shows the priority of all tasks.

TASK macros are now used to generate various switch
statements and enums used in task accounting.

The new system is not strict, MyTask could still be
assigned the priority/id of OldTask, however this
flexibility can be useful in some circumstances.

Note this patch has changed ep_testsuite test_item_pager
to increase the max_size value in the test config. This
is because this patch increases the baseline heap usage
of a bucket as we've increased the number of Histogram
object allocated by EventuallyPersistentStore.

Prior to this patch 28 were allocated, with this patch
51 are allocated (1 per task). Each Histogram<hrtime_t
is approx 1568 bytes (on OSX clang build).

The new max_size is 2.5MiB

Change-Id: I209c67945b964023615af37a12f83ca97142ce53
Reviewed-on: http://review.couchbase.org/65253
Well-Formed: buildbot <build@couchbase.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: buildbot <build@couchbase.com>
34 files changed:
CMakeLists.txt
README.md [new file with mode: 0644]
src/access_scanner.h
src/backfill.cc
src/backfill.h
src/bgfetcher.cc
src/checkpoint_remover.cc
src/checkpoint_remover.h
src/connmap.cc
src/dcp-backfill-manager.cc
src/dcp-consumer.cc
src/dcp-consumer.h
src/dcp-stream.h
src/defragmenter.cc
src/ep.cc
src/ep.h
src/ep_engine.cc
src/executorthread.cc
src/flusher.cc
src/htresizer.cc
src/htresizer.h
src/item_pager.cc
src/item_pager.h
src/priority.cc [deleted file]
src/priority.h [deleted file]
src/tapconnection.cc
src/tapconnection.h
src/tasks.cc
src/tasks.def.h [new file with mode: 0644]
src/tasks.h
src/warmup.cc
src/warmup.h
tests/ep_testsuite.cc
tests/module_tests/priority_test.cc [deleted file]