6 years agoMB-12924: [Observe] Do not return keyStats for a temp item 65/44365/6
abhinavdangeti [Tue, 16 Dec 2014 21:10:55 +0000 (13:10 -0800)]
MB-12924: [Observe] Do not return keyStats for a temp item

This caused an issue with observe during bidirectional
XDCR on deletes, where observe over one of the keys
caused the client to return a KeyExistsError as the
cas was not what was expected for a temp item (that
was deleted already).

Change-Id: Iaad9d7a4eea507362c6dd7cdde0febca0d9fc0cf
Reviewed-on: http://review.couchbase.org/44365
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: abhinav dangeti <abhinav@couchbase.com>
6 years agoMB-12706: clear temporary item id on an add operation 83/43383/2
Sriram Ganesan [Tue, 18 Nov 2014 22:16:11 +0000 (14:16 -0800)]
MB-12706: clear temporary item id on an add operation

When an add operation is performed on a temporary item, the id needs
to be cleared so that it isn't temporary item anymore.

Change-Id: Iefde4c74ec5c836cdd67404331358f3bd062ee2e
Reviewed-on: http://review.couchbase.org/43383
Tested-by: Sriram Ganesan <sriram@couchbase.com>
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
6 years ago[BP] Fix memory leak in vbucket deletion 03/43303/2
Mike Wiederhold [Mon, 22 Sep 2014 21:34:51 +0000 (14:34 -0700)]
[BP] Fix memory leak in vbucket deletion

If there are any bg fetch items then we must remember to explicitly
delete the value from them otherwise we will leak memory.

Change-Id: I2220b2f735024da4469664c0c0ab9e8145aa0b1e
Reviewed-on: http://review.couchbase.org/43303
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>
6 years ago[BP] MB-10406 Unlock a locked item on replica vbucket for replication 46/43146/3
Chiyoung Seo [Tue, 25 Mar 2014 20:01:33 +0000 (13:01 -0700)]
[BP] MB-10406 Unlock a locked item on replica vbucket for replication

GetAndLock causes the replication or rebalance operation to fail
in the following scenario:

1) Rebalance starts and is in progress
2) Active vbucket 0 is moving from Node 1 to Node 2
3) Item A on active vbucket 0 on Node 1 is locked through
   getLocked() API while vbucket 0 is still being transferring.
4) Active vbucket 0 is transferred to Node 2. Node 1 changes
   vbucket 0 state to replica.
5) Item A on active vbucket 0 on Node 2 is updated through
   Set() API.
6) Node 2 tries to replicate the updated Item A to the node 1,
   but receives an error code (ENGINE_KEY_EEXISTS) from Node 1
   because Item A is still locked due to the step 3).
7) Node 2 closes the TAP connection to Node 1 because of the
   above error code, which is a bug in the engine side.

To resolve the above issue, we should unlock a locked item on
replica vbucket.

Change-Id: I5300b9f8f1864ed8ff5388dc7a621c205c589047
Reviewed-on: http://review.couchbase.org/43146
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>
6 years agoCumulative hotfix for Includes changes from the following branches: 82/41382/2
Chris Hillery [Fri, 12 Sep 2014 00:45:42 +0000 (17:45 -0700)]
Cumulative hotfix for Includes changes from the following branches:

specifically comprising the following commits:

commit 94987ab3d101e967fcbd1236a1bb8fd37847cf41
Author: Mike Wiederhold <mike@couchbase.com>
Date:   Thu Sep 11 11:46:00 2014 -0700

    CBSE-1350:  Fix deadlock in checkpoint persistence command

    We need to release the hpChkMutex before notifying memcached in
    order to prevent a deadlock. The deadlock occurs when the flusher
    trys to notify memcached of checkpoint persistence at the same
    time a memcached worker thread is trying to add a new checkpoint
    persistence request.

commit a8be76ab4e237280cc5ad1249ddd895e6f0c394a
Author: Trond Norbye <trond.norbye@gmail.com>
Date:   Wed Jul 30 07:42:05 2014 +0200

    Fix log message for remaining bytes to send to mccouch

    In addition to improving the log message, backoff trying
    to send the data to allow the other side to drain their

commit c694d0e3043e2517569ca4276d496a4612bec9f5
Author: Chiyoung Seo <chiyoung.seo@gmail.com>
Date:   Fri Apr 25 17:37:29 2014 -0700

    MB-10962 Don't snooze the TAP connection notifier unnecessarily.

    There is a race in pausing and resuming the TAP connection
    notifier upon receiving a new mutation. To address this issue,
    This change adds a flag that indicates if there is a pending
    notification event for any TAP connections.

commit f9b9a8948cc6d6489b4f6b0fe4569be39c0cf456
Author: Jim Walker <jim@couchbase.com>
Date:   Thu Jun 12 13:55:35 2014 +0100

    MB-11411 Warmup with an access log always sets the loaded document's rev-id to 1.

Change-Id: If108875143a7b1cde5a4c5b80dadab6e71a12784
Reviewed-on: http://review.couchbase.org/41382
Reviewed-by: Philip Labee <philip@couchbase.com>
Tested-by: Philip Labee <philip@couchbase.com>
6 years agoMB-10277 Fix to the memory corruption bug in TOUCH command 48/33848/3 v2.5.1
Chiyoung Seo [Sat, 22 Feb 2014 00:17:27 +0000 (16:17 -0800)]
MB-10277 Fix to the memory corruption bug in TOUCH command

If TOUCH command is issued with the same expiration time for
a non-resident item, it causes the segmentation fault due to
the NULL pointer to an Item instance.
This change makes sure that we create an Item instance in the
above case because we still need to send the item's metadata
to the client.

Change-Id: Ic1efe24add1f0a3d2da2804c106b61c26bb275fc
Reviewed-on: http://review.couchbase.org/33848
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>
6 years agoMB-10059 Set a backfilled QueuedItem's rev_seq from hashtable. 03/33503/3
Chiyoung Seo [Thu, 13 Feb 2014 20:32:00 +0000 (12:32 -0800)]
MB-10059 Set a backfilled QueuedItem's rev_seq from hashtable.

Change-Id: Idfd3eb5a69147c657705232b2c0b6cf0a38ab902
Reviewed-on: http://review.couchbase.org/33503
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>
6 years agoMB-10059 Remove the unnecessary check in getting a rev_seq value 02/33502/3
Chiyoung Seo [Thu, 13 Feb 2014 19:10:52 +0000 (11:10 -0800)]
MB-10059 Remove the unnecessary check in getting a rev_seq value

Change-Id: I1757b5b0f8c2d58474faeabeb046e9363df3f346
Reviewed-on: http://review.couchbase.org/33502
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>
6 years agoMB-10059 Don't create a new rev_seq for a new item on replica. 01/33501/3
Chiyoung Seo [Thu, 13 Feb 2014 19:01:36 +0000 (11:01 -0800)]
MB-10059 Don't create a new rev_seq for a new item on replica.

Replica vbucket currently ignores rev_seq values of new items
from its active vbucket, but instead generates new rev_seq
values for those new items. This can cause the XDCR to be in
inconsistent state, especially when the rebalance or failover
happens. This change makes sure that the replica vbucket uses
a rev_seq value from its active vbucket for a new item.

Change-Id: I634d0296a610b88ffb5e1b97d20606a01d41dc23
Reviewed-on: http://review.couchbase.org/33501
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>
6 years agoMB-9953 Replace the old queued_item instance with the new one. 16/32816/2
Chiyoung Seo [Fri, 24 Jan 2014 19:04:40 +0000 (11:04 -0800)]
MB-9953 Replace the old queued_item instance with the new one.

When a new mutation arrives into a checkpoint, we should replace
the old queued_item instance with a new instance for the same
key. Otherwise, some fields won't be correctly updated (e.g.,
rev_seqno is not updated correctly).

Change-Id: I9d7c67be675c2297c84e253c47941149dd76d406
Reviewed-on: http://review.couchbase.org/32816
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Reviewed-by: Sundararaman Sridharan <sundar.sridharan@gmail.com>
Tested-by: Sundararaman Sridharan <sundar.sridharan@gmail.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>
6 years agoMB-9953 Correct a rev_seqno type to uint64_t 75/32775/2
Chiyoung Seo [Thu, 23 Jan 2014 21:53:25 +0000 (13:53 -0800)]
MB-9953 Correct a rev_seqno type to uint64_t

Change-Id: I46afadda87a83bf5f1b01dcb007b2a0feb6a7e27
Reviewed-on: http://review.couchbase.org/32775
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>
6 years ago[BP] MB-9692 Handle MutationLog lseek errors gracefully 97/32697/3
Sundar Sridharan [Tue, 21 Jan 2014 22:26:40 +0000 (14:26 -0800)]
[BP] MB-9692 Handle MutationLog lseek errors gracefully

Instead of asserting throw appropriate exception with logs.

Change-Id: Ib64b5bea506e2524cada118287e9e2d300b970e0
Reviewed-on: http://review.couchbase.org/32697
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>
7 years agoMB-9800 Always respond to CHECKPOINT_PERSISTENCE through flusher 10/32610/2
Chiyoung Seo [Fri, 17 Jan 2014 23:17:50 +0000 (15:17 -0800)]
MB-9800 Always respond to CHECKPOINT_PERSISTENCE through flusher

Even if a requested checkpoint is already persisted, let the
flusher respond to CHECKPOINT_PERSISTENCE command to prevent any
race conditions between the database storage engine and
view indexer.

Change-Id: I26a4d77d97c4c1d93957621f746e09bee2e2ebb4
Reviewed-on: http://review.couchbase.org/32610
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>
7 years agoMB-9938 Fix to the deadlock in doing the bg fetch job. 61/32561/2
Chiyoung Seo [Thu, 16 Jan 2014 20:56:45 +0000 (12:56 -0800)]
MB-9938 Fix to the deadlock in doing the bg fetch job.

Change-Id: Ia4cf7a98441812b0e703308bf04125026c4fd7e5
Reviewed-on: http://review.couchbase.org/32561
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>
7 years agoMB-9693 Return ENGINE_SUCCESS if an item is updated during bg fetch 72/32472/2
Chiyoung Seo [Wed, 15 Jan 2014 22:50:40 +0000 (14:50 -0800)]
MB-9693 Return ENGINE_SUCCESS if an item is updated during bg fetch

If an item (that might not exist before) is inserted while we are
processing a bg fetch request for its metadata, we should not
return any error codes, but instead ENGNINE_SUCCESS, so that the
memcached worker thread can return that item's metadata to the

Change-Id: I2f9749d422aef305764783d363d11f9921f186dd
Reviewed-on: http://review.couchbase.org/32472
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>
7 years agoMB-9693 Check the vbucket existence only upon a bg fetch completion 71/32471/2
Chiyoung Seo [Wed, 15 Jan 2014 22:44:58 +0000 (14:44 -0800)]
MB-9693 Check the vbucket existence only upon a bg fetch completion

As there is always a race in vbucket state changes, we should check
the vbucket existence only upon completing a bg fetch and then let
a memcached worker thread visit the vbucket again to complete GET

Change-Id: I9cdc76897c5e18f4c02a75606ec75a3f4dcee063
Reviewed-on: http://review.couchbase.org/32471
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>
7 years agoMB-9693 Don't evict a temp item whose state is in TEMP_INIT. 70/32470/2
Chiyoung Seo [Wed, 15 Jan 2014 22:39:08 +0000 (14:39 -0800)]
MB-9693 Don't evict a temp item whose state is in TEMP_INIT.

If a temp item's state is TEMP_INIT, it means that we spawned a
background fetch to figure out if an item was deleted before or
never exists before. If we allow the item pager to remove this
temp item, getMeta() request can't be processed correctly and will
return an incorrect status to the client.

Change-Id: I1b025c09d22660d70dea22cb92ae728a0bb8709e
Reviewed-on: http://review.couchbase.org/32470
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>
7 years agoMB-9693: Re-map enoent to not my vbucket for failed bg fetch 32/31832/5
Mike Wiederhold [Fri, 10 Jan 2014 20:28:41 +0000 (12:28 -0800)]
MB-9693: Re-map enoent to not my vbucket for failed bg fetch

If a VBucket is deleted in the middle of a background fetch then
the result of the disk read is an enoent. When processing this
error we need to check to see if the VBucket is still valid and
also that it is in active state before returning enoent. If it's
not then we should re-map the return code to not my vbucket since
this is much more descriptive of the actual cause of the operation

Change-Id: I1131b9657a2ee7ca7990416afcd483cd9f54c4e0
Reviewed-on: http://review.couchbase.org/31832
Reviewed-by: Sundararaman Sridharan <sundar.sridharan@gmail.com>
Reviewed-by: Junyi Xie <junyi.couchbase@gmail.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
7 years agoMB-9864: Reset access log if less then header size 03/31803/3
Sundar Sridharan [Fri, 10 Jan 2014 22:14:37 +0000 (14:14 -0800)]
MB-9864: Reset access log if less then header size

sometimes an incomplete write can result in access
log header being less than header size.
Do not throw Short Read exception, rather
simply recovery by removing and recreating the access log
Change-Id: I8d1250eb1e78dd71f57b7a30ed8d90defedde446
Reviewed-on: http://review.couchbase.org/31803
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Reviewed-by: Sundararaman Sridharan <sundar.sridharan@gmail.com>
Tested-by: Sundararaman Sridharan <sundar.sridharan@gmail.com>
7 years agoMB-9852 Notify pending connection when a pending vb becomes active 03/31703/2
Chiyoung Seo [Wed, 8 Jan 2014 02:32:46 +0000 (11:32 +0900)]
MB-9852 Notify pending connection when a pending vb becomes active

This change fixes a bug in notifying pending connections
when a vbucket state is transitioned from pending to active.

Change-Id: I5dd77f0f0c8d543808be3613d3d0336b4db01006
Reviewed-on: http://review.couchbase.org/31703
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>
7 years agoMB-9423: Add a 30 second timeout to cbepctl and cbstats 73/31273/3
Mike Wiederhold [Fri, 20 Dec 2013 20:04:01 +0000 (12:04 -0800)]
MB-9423: Add a 30 second timeout to cbepctl and cbstats

Change-Id: I392a9efab924de99e9ddbf4171b368b02327453d
Reviewed-on: http://review.couchbase.org/31273
Reviewed-by: abhinav dangeti <abhinav@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
7 years agoMB-9161 Reset engine_specific to NULL when chk_persistence fails 77/30677/2
Chiyoung Seo [Wed, 27 Nov 2013 23:59:27 +0000 (15:59 -0800)]
MB-9161 Reset engine_specific to NULL when chk_persistence fails

If a checkpoint_persistence request fails due to the timeout,
we should then reset the engine_specific value to NULL, so that
the next checkpoint persistence request should work correctly.

Change-Id: Iab0ab3c41fe39dc7060c3d094ec04cb9cfadcb73
Reviewed-on: http://review.couchbase.org/30677
Reviewed-by: Sundararaman Sridharan <sundar.sridharan@gmail.com>
Tested-by: Sundararaman Sridharan <sundar.sridharan@gmail.com>
7 years agoMB-9618 Release engine handle before CCCP callback 92/30492/4
Sundar Sridharan [Sat, 23 Nov 2013 01:16:18 +0000 (17:16 -0800)]
MB-9618 Release engine handle before CCCP callback

As the memcached CCCP callback can allocate more memory for the
sending buffer, we should release the engine handle before
calling the callback function, so that we can maintain the
memory accounting correctly.
Change-Id: I2899f65acf3b40db9cd3cf7602e5bb4b874750bf
Reviewed-on: http://review.couchbase.org/30492
Reviewed-by: Sundararaman Sridharan <sundar.sridharan@gmail.com>
Tested-by: Sundararaman Sridharan <sundar.sridharan@gmail.com>
7 years agoMB-9311: Notify all the pending connections on vbucket delete 53/30353/3
Mike Wiederhold [Sat, 16 Nov 2013 01:12:10 +0000 (17:12 -0800)]
MB-9311: Notify all the pending connections on vbucket delete

If a given vbucket is deleted, we should notify it to all of its
pending connections, so that the connections are released
immediately and the error messages should be sent to the clients.

Change-Id: Ic230a803f0b26944f98e84ed4d21f8d834e3a561
Reviewed-on: http://review.couchbase.org/30353
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
7 years agoMB-9549 Fix to the memory leak in GetAndUpdateTTL 03/30303/5
Chiyoung Seo [Thu, 14 Nov 2013 02:56:42 +0000 (18:56 -0800)]
MB-9549 Fix to the memory leak in GetAndUpdateTTL

This is a regression from 2.2.0, which causes a memory leak
when we update the TTL for a non-resident item.

Change-Id: I3dfd9dcd09a0acfdcd2cd6623f75ca54916d1116
Reviewed-on: http://review.couchbase.org/30303
Reviewed-by: Sundararaman Sridharan <sundar.sridharan@gmail.com>
Tested-by: Sundararaman Sridharan <sundar.sridharan@gmail.com>
7 years agoMB-9203 Always reset the open checkpoint upon resetting a vbucket 02/30302/3
Chiyoung Seo [Thu, 14 Nov 2013 02:35:10 +0000 (18:35 -0800)]
MB-9203 Always reset the open checkpoint upon resetting a vbucket

When a replica vbucket is reset, its open checkpoint id is not
reset to zero when the vbucket has zero items. This causes a bug
in updating the persisted checkpoint id, which consequently
incurs the rebalance to be stuck.

This fix always resets the open checkpoint id of a replica
vbucket to zero upon resetting the vbucket.

Change-Id: If6b5e2ca9b37dd533818e8d3397ec2c695345295
Reviewed-on: http://review.couchbase.org/30302
Reviewed-by: Sundararaman Sridharan <sundar.sridharan@gmail.com>
Tested-by: Sundararaman Sridharan <sundar.sridharan@gmail.com>
7 years agoMB-9532 Find a new compacted file upon receiving a temp failure 01/30301/2
Chiyoung Seo [Thu, 14 Nov 2013 01:03:05 +0000 (17:03 -0800)]
MB-9532 Find a new compacted file upon receiving a temp failure

When the flusher receives a temp failure from the mccouch, it
should check if a new compacted file exists or not before
re-writing dirty items. This is required especially when the
deletion of the old file fails due to some external issues.
Otherwise, the flusher will keep writing dirty items into the
old file and receiving a temp failure over and over.

Change-Id: Ifd91eea1acfe82f5ea43ff93ca650b7928844b33
Reviewed-on: http://review.couchbase.org/30301
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Reviewed-by: Sundararaman Sridharan <sundar.sridharan@gmail.com>
Tested-by: Sundararaman Sridharan <sundar.sridharan@gmail.com>
7 years agoMB-9434: prevent deadlock between flusher & stats 00/29900/3
Sundar Sridharan [Wed, 30 Oct 2013 22:03:06 +0000 (15:03 -0700)]
MB-9434: prevent deadlock between flusher & stats

If the memcached stats thread which is holding the thread
lock tries to acquire the high priority checkpoint list
lock just to read its size, it can deadlock on the
flusher thread which may be holding the list lock & trying
to notify the same memcached thread doing the stats

To avoid this deadlock maintain the size of the high
priority checkpoints in a separate variable and allow for
dirty read on the size

Change-Id: I6f0991c22deb2cc7762e5967496b99c3baed2c12
Reviewed-on: http://review.couchbase.org/29900
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>
7 years agoMB-8727: Cluster Configuration Carrier Publication 84/29684/19
abhinavdangeti [Wed, 30 Oct 2013 21:57:56 +0000 (14:57 -0700)]
MB-8727: Cluster Configuration Carrier Publication

Addresses MB-8727:

Addresses MB-8728:

- Cluster config Blob will be saved as it is received
and will be returned as it is, with assumption that
blob will be of JSON type, and that SDK will need to
check the type of the document to differentiate
between older and new version of couchbase.

Addresses MB-8729:
- NOT_MY_VBUCKET to return cluster config blob
instead of the "not my vbucket" message.
- Memcached
change: http://review.couchbase.org/#/c/29844/
For memcached to retrieve Cluster Configuration from
ep_engine in case of ENGINE_NOT_MY_VBUCKET response.
- bucket_engine
change: http://review.couchbase.org/#/c/29845/

Change-Id: I20cf4aa145139d19524f252d3444dd0f52d0965d
Reviewed-on: http://review.couchbase.org/29684
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>
7 years agoMB-8187: chk_items estimate only for MUTATION/DELs 82/29882/5
abhinavdangeti [Wed, 30 Oct 2013 17:49:17 +0000 (10:49 -0700)]
MB-8187: chk_items estimate only for MUTATION/DELs

- During stat collection, account for chk_items (the estimate)
only during TAP_MUTATION and TAP_DELETIONS, and not at checkpoint
start and end messages.
- This way docsTotal and docsTransferred stats will show as zero
during a rebalance if the bucket were empty.

Change-Id: I5df5d19f16b7dd7abe7465d356bb748d56e20d8e
Reviewed-on: http://review.couchbase.org/29882
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>
7 years agoMB-8244: Unit test fix in fetching-worker-stats 56/29856/4
abhinavdangeti [Wed, 30 Oct 2013 17:51:50 +0000 (10:51 -0700)]
MB-8244: Unit test fix in fetching-worker-stats

Change-Id: I0831bf56410f76b0c9849fffab0f8fcbfaff4b6e
Reviewed-on: http://review.couchbase.org/29856
Reviewed-by: abhinav dangeti <abhinav@couchbase.com>
Tested-by: abhinav dangeti <abhinav@couchbase.com>
7 years agoMB-9147: lseek can return 0 if the file has no len 43/29843/2
Sundar Sridharan [Tue, 29 Oct 2013 02:28:39 +0000 (19:28 -0700)]
MB-9147: lseek can return 0 if the file has no len

This will avoid asserting incorrectly.
Backporting change from master branch into 2.5.0

Change-Id: Ibca0d69782f753e0dbb16652d5591a21e514c9f9
Reviewed-on: http://review.couchbase.org/29048
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Reviewed-on: http://review.couchbase.org/29843
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>
7 years agoMB-8024: Fix to the BackfillRemaining stat overflow 36/29836/5
dliao [Mon, 28 Oct 2013 22:52:29 +0000 (15:52 -0700)]
MB-8024: Fix to the BackfillRemaining stat overflow

Note that stats used in estimating the number of backfill items
are atomic individually, but not atomic when they use together.
Therefore, there is still a small time window that can cause the
backfill estimation counter to be overflow or underflow.

This change adds a check to prevent adding a negative number to
the backfill estimation counter.

Change-Id: I4752445e680c36b387aa80ffa1a985a7926bc9f2
Reviewed-on: http://review.couchbase.org/29836
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>
7 years agofor MB-7230: state why access log was seen corrupt 68/29768/2
Sundar Sridharan [Fri, 25 Oct 2013 18:46:09 +0000 (11:46 -0700)]
for MB-7230: state why access log was seen corrupt

Basic debugging whenever we hit a read error on
scanning the access log

Change-Id: I4b02a9484e7d0b56c0691e062717729d9d7ed4d5
Reviewed-on: http://review.couchbase.org/29768
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>
7 years agoMB-9306 Fix v->isDirty assertion due to race 99/29699/3
Sundar Sridharan [Fri, 11 Oct 2013 23:51:53 +0000 (16:51 -0700)]
MB-9306 Fix v->isDirty assertion due to race

BGFetcher must Validate CAS values before queuing dirty
item to disk when exptime is updated

Relax assertion since downside is just another set operation

Backport from

Change-Id: Icfcce79371bb867052f12c8a8574c8a318d71ace
Reviewed-on: http://review.couchbase.org/29461
Reviewed-by: Chiyoung Seo <chiyoung@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung@couchbase.com>
Reviewed-on: http://review.couchbase.org/29699
Reviewed-by: abhinav dangeti <abhinav.dangeti@gmail.com>
Tested-by: abhinav dangeti <abhinav.dangeti@gmail.com>
7 years agoMB-9098: Acquire lock to retrieve list size 87/29687/2
abhinavdangeti [Tue, 22 Oct 2013 18:57:35 +0000 (11:57 -0700)]
MB-9098: Acquire lock to retrieve list size

- Need for synchronizing access to shared data
to avoid segmentation faults and possible
memory corruption.

- Backporting from master:

Change-Id: Id24770dbc72c082967959aee30c8e21b9556989e
Reviewed-on: http://review.couchbase.org/29687
Reviewed-by: Sundararaman Sridharan <sundar.sridharan@gmail.com>
Tested-by: Sundararaman Sridharan <sundar.sridharan@gmail.com>
7 years agoMB-9044: fixup check for stats.json existence 84/28784/2
Trond Norbye [Wed, 4 Sep 2013 06:19:12 +0000 (06:19 +0000)]
MB-9044: fixup check for stats.json existence

Change-Id: I1a02ed27eda5d2bd46a3b7431952c45d3c2dc0ef
Reviewed-on: http://review.couchbase.org/28784
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-9000 Retry open_db operation with the create option. 17/28717/3
Chiyoung Seo [Sat, 31 Aug 2013 02:10:54 +0000 (19:10 -0700)]
MB-9000 Retry open_db operation with the create option.

When an open_db operation without a create option fails
with FILE_NOT_FOUND error constantly, we should open a database
file with a create option to support subsequent read / write

Change-Id: I30dac3f0d7dda1c32c99bdebb4a99aef09c8a577
Reviewed-on: http://review.couchbase.org/28717
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-9015 Return the items CAS value from touch 69/28569/3
Trond Norbye [Mon, 26 Aug 2013 10:29:15 +0000 (12:29 +0200)]
MB-9015 Return the items CAS value from touch

Change-Id: Ie9a5d3f8a520c7f7553cfa990270b1464534ce78
Reviewed-on: http://review.couchbase.org/28569
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Reviewed-by: Sergey Avseyev <sergey.avseyev@gmail.com>
7 years agoMB-8530 Non-blocking checkpoint stats call for all vbuckets 54/28554/4
Chiyoung Seo [Sat, 24 Aug 2013 02:05:47 +0000 (19:05 -0700)]
MB-8530 Non-blocking checkpoint stats call for all vbuckets

We saw that the timings of checkpoint stats call for all vbuckets
was ranged from 16ms to 262ms, which seems quite high. These
checkpoint stats are frequently requested by the cluster manager
during cluster rebalance. This change supports the non-blocking
checkpoint stats call for all vbuckets, so that the worker threads
can process other requests while the stats call is being processed
by the NON-IO dispatcher.

Change-Id: Ie472597c31aa13db1d9ae49855fb29a91314857d
Reviewed-on: http://review.couchbase.org/28554
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-8992: Unlock a locked key when deleting with the proper cas 45/28645/2
Mike Wiederhold [Wed, 28 Aug 2013 23:06:05 +0000 (16:06 -0700)]
MB-8992: Unlock a locked key when deleting with the proper cas

Change-Id: I057c290c40301fb4cd8667d7752abf2cfd778690
Reviewed-on: http://review.couchbase.org/28645
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-8960 New operational stats for setWithMeta ops 88/28588/6
Sundar Sridharan [Tue, 27 Aug 2013 02:00:53 +0000 (19:00 -0700)]
MB-8960 New operational stats for setWithMeta ops

1) ep_num_ops_get_meta_on_set_meta
 Number of get_meta ops spawn due to set_with_meta operations

2) set_with_meta
 Latency timing histogram for set_with_meta operations

Change-Id: I75a22c1ffd479da8a0bdad8a1881b93896a32235
Reviewed-on: http://review.couchbase.org/28588
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-8955 Prevent ExTask starvation: order by taskId 50/28550/4
Sundar Sridharan [Fri, 23 Aug 2013 23:49:31 +0000 (16:49 -0700)]
MB-8955 Prevent ExTask starvation: order by taskId

Change-Id: Ib75d92a1e64ca85843e688ebe23579841c23f4d7
Reviewed-on: http://review.couchbase.org/28550
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-100 Fix compilation on gcc 4.8. 23/28523/2
Aliaksey Artamonau [Fri, 23 Aug 2013 02:37:42 +0000 (19:37 -0700)]
MB-100 Fix compilation on gcc 4.8.

Here's the error it produces:

src/atomic.hh: In instantiation of 'void ThreadLocalPtr<T>::operator=(T*) [with T = AtomicPtr<std::queue<SingleThreadedRCPtr<TapConnection>, std::deque<SingleThreadedRCPtr<TapConnection>, std::allocator<SingleThreadedRCPtr<TapConnection> > > > >]':
src/atomic.hh:606:21:   required from 'AtomicPtr<std::queue<T> >* AtomicQueue<T>::initialize() [with T = SingleThreadedRCPtr<TapConnection>]'
src/atomic.hh:613:31:   required from 'std::queue<T>* AtomicQueue<T>::swapQueue(std::queue<T>*) [with T = SingleThreadedRCPtr<TapConnection>]'
src/atomic.hh:514:38:   required from 'void AtomicQueue<T>::push(T&) [with T = SingleThreadedRCPtr<TapConnection>]'
src/tapconnmap.cc:286:45:   required from here
src/atomic.hh:91:21: error: 'set' was not declared in this scope, and no declarations were found by argument-dependent lookup at the point of instantiation [-fpermissive]
src/atomic.hh:91:21: note: declarations in dependent base 'ThreadLocal<AtomicPtr<std::queue<SingleThreadedRCPtr<TapConnection>, std::deque<SingleThreadedRCPtr<TapConnection>, std::allocator<SingleThreadedRCPtr<TapConnection> > > > >*>' are not found by unqualified lookup
src/atomic.hh:91:21: note: use 'this->set' instead

Change-Id: Ia4dbc0026f954c7f5d799903261abc239b8da6e4
Reviewed-on: http://review.couchbase.org/28523
Tested-by: Aliaksey Artamonau <aliaksiej.artamonau@gmail.com>
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-8930 Stat for max number of bg fetches in the queue seen so far 90/28490/3
Chiyoung Seo [Thu, 22 Aug 2013 22:12:58 +0000 (15:12 -0700)]
MB-8930 Stat for max number of bg fetches in the queue seen so far

Change-Id: Ice13a5cccf538b01350ef538d5d09fc6215d1760
Reviewed-on: http://review.couchbase.org/28490
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-8869 Use atomic queue to push notifications for TAP producers 64/28464/3
Chiyoung Seo [Thu, 22 Aug 2013 06:48:58 +0000 (23:48 -0700)]
MB-8869 Use atomic queue to push notifications for TAP producers

We saw the frontend ops/sec performance regression from the
previous fix to the OBSERVE replication latency issue. To address
this issue, this change uses the atomic queue to push
notification events for TAP producers, and schedules the
daemon task to grab those events and notify them to the
corresponding TAP producers.

Change-Id: I4bda29d50d446c338cff78014f20f2cef9f7bd64
Reviewed-on: http://review.couchbase.org/28464
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-8890 Fix to the incorrect check for deleted/non-existent items 28/28428/3
Chiyoung Seo [Wed, 21 Aug 2013 08:50:14 +0000 (01:50 -0700)]
MB-8890 Fix to the incorrect check for deleted/non-existent items

Change-Id: I3669419ba83b6624e58d9feebc0f695ca6212824
Reviewed-on: http://review.couchbase.org/28428
Reviewed-by: Sundararaman Sridharan <sundar.sridharan@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-8530 Set TAP paused flag upon returning TAP_PAUSE response. 24/28424/2
Chiyoung Seo [Wed, 21 Aug 2013 04:29:30 +0000 (21:29 -0700)]
MB-8530 Set TAP paused flag upon returning TAP_PAUSE response.

We should set the TAP paused flag to true only if there are no
items to be sent to the destination. Otherwise, we invoke
too many unnecessary notify_io_complete calls.

Change-Id: Icc1da864b4202406ff84f29172ade09fe395cce4
Reviewed-on: http://review.couchbase.org/28424
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-8890: Remove assertion for deleted/non-existent tmp item 03/28403/3
Mike Wiederhold [Tue, 20 Aug 2013 22:19:31 +0000 (15:19 -0700)]
MB-8890: Remove assertion for deleted/non-existent tmp item

This assertion is invalid since it is possible that a background
fetch for an item is updated through a front end operation and
then evicted from memory before the background fetch completes.

Change-Id: I3d7638c10bd88721f3a30c7aa4e143fa476b0a14
Reviewed-on: http://review.couchbase.org/28403
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-8825 Use seq num and cas only for delete conflict resolution 98/28398/2
Chiyoung Seo [Tue, 20 Aug 2013 19:40:21 +0000 (12:40 -0700)]
MB-8825 Use seq num and cas only for delete conflict resolution

Recently, we decided to use an expiration field for tombstone
purging, which can break the conflict resolution for the case
where the source and destination have the same seq number and
cas value. To resolve this issue, the change allow us to use
seq number and cas value only for delete conflict resolutions.

Change-Id: I51ac6eecc7e9f4499bd0ac7285f9021381fdaacd
Reviewed-on: http://review.couchbase.org/28398
Tested-by: Junyi Xie <junyi.couchbase@gmail.com>
Reviewed-by: Junyi Xie <junyi.couchbase@gmail.com>
7 years agoMB-8861 Increase the number of vbucket / tap producer map locks 83/28383/2
Chiyoung Seo [Tue, 20 Aug 2013 02:07:58 +0000 (19:07 -0700)]
MB-8861 Increase the number of vbucket / tap producer map locks

Change-Id: I858029642f69a8e6dae8349f8cd2a169396bcdb4
Reviewed-on: http://review.couchbase.org/28383
Reviewed-by: abhinav dangeti <abhinav.dangeti@gmail.com>
Tested-by: abhinav dangeti <abhinav.dangeti@gmail.com>
7 years agoMB-100 Correct the tap notification task name 82/28382/2
Chiyoung Seo [Mon, 19 Aug 2013 22:58:37 +0000 (15:58 -0700)]
MB-100 Correct the tap notification task name

Change-Id: I809783bfa1f6a060faaf5bd95c2c6b4622356d71
Reviewed-on: http://review.couchbase.org/28382
Reviewed-by: abhinav dangeti <abhinav.dangeti@gmail.com>
Tested-by: abhinav dangeti <abhinav.dangeti@gmail.com>
7 years agoMB-8861 Schedule a tap connection notification through dispatcher 34/28334/4
Chiyoung Seo [Sat, 17 Aug 2013 21:54:17 +0000 (14:54 -0700)]
MB-8861 Schedule a tap connection notification through dispatcher

This change allows us to schedule a notification event for a
given TAP connection through the non-IO dispatcher.

Change-Id: I3ef0ef3c9bcd86293f0114355c064069946c5977
Reviewed-on: http://review.couchbase.org/28334
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-8657 Fix the number of workers to be matched for mix loads 25/28325/3
Sundar Sridharan [Sat, 17 Aug 2013 02:40:06 +0000 (19:40 -0700)]
MB-8657 Fix the number of workers to be matched for mix loads

This change fixes the issue for a case where the number of
actual worker threads is not matched with the number of readers
and writers given by the user.

Change-Id: I1c6b1284bd4677af715603d01d1829b6611a87f0
Reviewed-on: http://review.couchbase.org/28325
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-8861: Release bucket lock before queueing a dirty item 19/28319/9
Mike Wiederhold [Fri, 16 Aug 2013 21:41:57 +0000 (14:41 -0700)]
MB-8861: Release bucket lock before queueing a dirty item

When we queue a dirty item we notify any tap connections that
are streaming items from the vbucket the item was added to. If we
don't release the lock then we can cause a deadlock on the
notifySync and queueLock variables in the tap replicator.

Change-Id: Ia8fc31f5c0ac33cf6a79ef4dd4337f8a6459d6ef
Reviewed-on: http://review.couchbase.org/28319
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
7 years agoMB-8825: Add stat to keep track of conflict resolution fails 71/28271/2
Mike Wiederhold [Thu, 15 Aug 2013 23:51:22 +0000 (16:51 -0700)]
MB-8825: Add stat to keep track of conflict resolution fails

This stat will help with xdcr debugging

Change-Id: Ie2ea77f51f40581ab60c823cf40c5276b4e27845
Reviewed-on: http://review.couchbase.org/28271
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-8828: Check memory condition before returning enomem for withmeta ops 69/28269/2
Mike Wiederhold [Thu, 15 Aug 2013 23:07:11 +0000 (16:07 -0700)]
MB-8828: Check memory condition before returning enomem for withmeta ops

If we run into a memory condition when we are trying to put something
into the hashtable then we return enomem even though we might actually
be able to free memory. This change makes sure we check the memory
condition before actually returning an error code.

Change-Id: I3d10641764653c971aaeb9e8089fb21fe891a699
Reviewed-on: http://review.couchbase.org/28269
Reviewed-by: David Liao <david.liao@couchbase.com>
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
7 years agoMB-8846 Use std::set instead of unordered::set. 57/28257/2
Chiyoung Seo [Thu, 15 Aug 2013 18:27:31 +0000 (11:27 -0700)]
MB-8846 Use std::set instead of unordered::set.

The current mingw build env doesn't support unordered::set.

Change-Id: Ib4c6e635015ff464442430e138169d1fb067b063
Reviewed-on: http://review.couchbase.org/28257
Tested-by: Bin Cui <bin.cui@gmail.com>
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-8798 Remove dead conns by iterating the conn list one time. 07/28107/5
Chiyoung Seo [Mon, 12 Aug 2013 06:31:19 +0000 (23:31 -0700)]
MB-8798 Remove dead conns by iterating the conn list one time.

Change-Id: I04a9e64462391b7b93249828d1ecbef3dfd60175
Reviewed-on: http://review.couchbase.org/28107
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-8453 Maintain the TAP connection list for each vbucket 72/27972/8
abhinavdangeti [Thu, 8 Aug 2013 21:13:12 +0000 (14:13 -0700)]
MB-8453 Maintain the TAP connection list for each vbucket

When we receive a mutation event for a given vbucket, we
should check if any of its replicator TAP connections is
in a paused state, and notify that event to the paused
TAP connections, so that they can immediately send it
to their replicas.

To address the above issue, this change maintains a
separate list of TAP connections for each vbucket.
The list gets updated when an TAP connection is closed or
an TAP connection's vbucket list is changed.

Change-Id: I5ca49fe17fbeb406da3ebcc046285ddc164735eb
Reviewed-on: http://review.couchbase.org/27972
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-8679 Add pendingVbs queue to bgfetcher 78/27178/5
Jin Lim [Thu, 27 Jun 2013 20:13:46 +0000 (13:13 -0700)]
MB-8679 Add pendingVbs queue to bgfetcher

This queue allows bgfetcher to visit only those vbuckets
in the given shard that have pending bgfetch requests.
This optmizes scheduling bgfetch requests in more efficient
fifo way.

Rebalance should benefit the above enhancement most, and initial
test shows below performance increase.


                2.1.0-718-rel ||  2.1.1-with-this-fix

Time hrs        4.48          ||  3.83

Time (sec)      3095          ||  2578

Sets (90-95-99)
(ms)            11 – 17 – 53  ||  6 – 9 – 18

Gets (90-95-99)
(ms)            13 – 20 – 208 ||  9 – 14 – 164

Ops (avg)       2.40k         ||  1.48k

Change-Id: Iadbbcb069701da947aac2a8cbc0d8b30762a13a4
Reviewed-on: http://review.couchbase.org/27178
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoRevert "Revert "MB-8581 Continue warmup after loading access_log."" 52/28152/2
Sundar Sridharan [Tue, 13 Aug 2013 01:01:40 +0000 (18:01 -0700)]
Revert "Revert "MB-8581 Continue warmup after loading access_log.""

This reverts commit b74fdcc82e895d36f974e8e5ac3554532a3cdb90.


Change-Id: If7aae2ee894911038410eee82aa6ec5a1474fbc0
Reviewed-on: http://review.couchbase.org/28152
Tested-by: Bin Cui <bin.cui@gmail.com>
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-100: Fix build error on windows. 49/28149/4
Bin Cui [Tue, 13 Aug 2013 00:05:39 +0000 (17:05 -0700)]
MB-100: Fix build error on windows.

On windows, std::tr1 doesn't have definition for

Change-Id: If978d165bd55833fde34e03a4e650eeb94166db5
Reviewed-on: http://review.couchbase.org/28149
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoRevert "MB-8581 Continue warmup after loading access_log." 41/28141/2
Phil Labee [Mon, 12 Aug 2013 23:28:04 +0000 (16:28 -0700)]
Revert "MB-8581 Continue warmup after loading access_log."

This reverts commit 06c086ccb69109fb1a89c17a8bf6c509ea03a9c8.

Change-Id: I2354ff0e7e66fff46f0c78ecab6b150a0e03d706
Reviewed-on: http://review.couchbase.org/28141
Tested-by: Philip Labee <philip@couchbase.com>
Reviewed-by: Philip Labee <philip@couchbase.com>
7 years agoMB-8657 fix total number of threads allocated for MRW 96/28096/3
Jin Lim [Fri, 9 Aug 2013 23:04:02 +0000 (16:04 -0700)]
MB-8657 fix total number of threads allocated for MRW

Upon this fix the total number of threads allocated for MRW
workers matches the max thread number specified by users via

Certain setting such as 4, 5, 7 may break the workload ratio (2:1)
btw numbers of readers and writers but will conform to the user
input thus avoid any confusion.

Change-Id: I3e45ebc38903d37df6746a9f20b9a1e8f6919e03
Reviewed-on: http://review.couchbase.org/28096
Tested-by: Jin Lim <jin@couchbase.com>
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-8581 Continue warmup after loading access_log. 38/28038/3
David Liao [Thu, 8 Aug 2013 19:56:35 +0000 (12:56 -0700)]
MB-8581 Continue warmup after loading access_log.

Added a loadingData step after loadingAccessLog to meet
warmup watermarks. In order to avoid loading items already
loaded earlier in loadingAccessLog phase, we check if an
item is resident or not during loadingData phase.
A method "isLoaded" is added to LoadingStorageKVPairCallback
to perform the check.

Added a test case "warmup_accesslog" for this feature.

Change-Id: I5eeb86b43b4e05a87df97b103c253a2db79db5ed
Reviewed-on: http://review.couchbase.org/28038
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoCheck for the existence of the file before opening it 22/27922/2
Trond Norbye [Mon, 5 Aug 2013 18:53:00 +0000 (20:53 +0200)]
Check for the existence of the file before opening it

I had to add the following test to avoid ep-engine to
dump core on SmartOS.

Change-Id: I139acd69de51547e71ea0f787e4c6f903f5e4ba4
Reviewed-on: http://review.couchbase.org/27922
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-8012 Fix to unit test failures in set/add/del/WithMeta ops. 98/27698/2
Chiyoung Seo [Fri, 26 Jul 2013 22:52:47 +0000 (15:52 -0700)]
MB-8012 Fix to unit test failures in set/add/del/WithMeta ops.

We recently made a change in persisting a deletion so that we
put the current epoch timestamp of the deletion in the
expiration field of the key's meta data.

This change fixes the unit test failures due to the above change.

Change-Id: I15ffa887c5464c7185176173a68ba86a6a9833df
Reviewed-on: http://review.couchbase.org/27698
Reviewed-by: Jin Lim <jin@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-8012: store deletion timestamp in correct byteorder 21/27721/2
Aliaksey Kandratsenka [Mon, 29 Jul 2013 13:03:50 +0000 (16:03 +0300)]
MB-8012: store deletion timestamp in correct byteorder

Change-Id: Ia2d3f1257cc530e7f81fbe610f44595bd96544b4
Reviewed-on: http://review.couchbase.org/27721
Tested-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Reviewed-by: Aaron Miller <apage43@ninjawhale.com>
7 years agoMB-8698: Fixed conditional check bug in warmup/loadingAccessLog 85/27685/2
david liao [Thu, 25 Jul 2013 23:08:10 +0000 (16:08 -0700)]
MB-8698: Fixed conditional check bug in warmup/loadingAccessLog

When loading access log during warmup, make sure we correctly
check for duplicate items and skip them. The bug caused all
items to be skipped.

Change-Id: I26f4452a9862ea69ab496b3d43758bb824412559
Reviewed-on: http://review.couchbase.org/27685
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-100 make ep-engine compile 21/27521/4
Junyi Xie [Wed, 17 Jul 2013 22:00:14 +0000 (18:00 -0400)]
MB-100 make ep-engine compile

Change-Id: I72a1c766415c1e458d625f4d1f58fa56048ed10d
Reviewed-on: http://review.couchbase.org/27521
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-8012 Store time of deletion in expiry field 27/27427/7
Aaron Miller [Mon, 15 Jul 2013 17:59:14 +0000 (10:59 -0700)]
MB-8012 Store time of deletion in expiry field

Using this timestamp the compactor can purge delete records once they
reach a certain age.

Change-Id: Ib541de9e1a5c14674eaabbea6c03629f9f713870
Reviewed-on: http://review.couchbase.org/27427
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-8486 Create a new CAS for an item with corrupted CAS in warmup 76/27376/3
Chiyoung Seo [Wed, 10 Jul 2013 23:13:33 +0000 (16:13 -0700)]
MB-8486 Create a new CAS for an item with corrupted CAS in warmup

If the CAS value of a given item is corrupted during warmup, we
can simply create a new CAS value for that item instead of
aborting the process.

Change-Id: I02b4889c6f71d27f98946aeeb9529f90bdbff53b
Reviewed-on: http://review.couchbase.org/27376
Reviewed-by: Jin Lim <jin@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-8600: Add all args to log message to prevent possible segfault
Mike Wiederhold [Mon, 15 Jul 2013 21:46:02 +0000 (14:46 -0700)]
MB-8600: Add all args to log message to prevent possible segfault

Change-Id: I4a60431fd775670ef1353e7cf36884e7f1436b4c
Reviewed-on: http://review.couchbase.org/27431
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
7 years agoMB-8578: Add stats uuid
Trond Norbye [Wed, 10 Jul 2013 06:17:05 +0000 (08:17 +0200)]
MB-8578: Add stats uuid

Change-Id: I348f644e810711297700b5c94de075e3ee3d154d
Reviewed-on: http://review.couchbase.org/27356
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Trond Norbye <trond.norbye@gmail.com>
7 years agoMB-8244: Add dispatcher, worker thread stats
abhinavdangeti [Tue, 2 Jul 2013 19:41:32 +0000 (12:41 -0700)]
MB-8244: Add dispatcher, worker thread stats

- Enables dispatcher stats
- Prints reader-writer thread stats
- Add condition to return immediately without
printing stats if engine in shutdown mode
- Changes based on previous comments included
- Adding a unittest: test_worker_stats
- Formated dispatcher stats (sample)

    runtime: 6ms
    state:   running
    task:    Running a flusher loop

- Add stats' details in doc/stats.org

Change-Id: I034dc6ea04df0e7e018849e1ebe8c1f6a02d9246
Reviewed-on: http://review.couchbase.org/27121
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: abhinav dangeti <abhinav.dangeti@gmail.com>
7 years agoMB-8467 Reset notifySent flag upon vbucket filter changes.
Chiyoung Seo [Sat, 15 Jun 2013 04:17:02 +0000 (21:17 -0700)]
MB-8467 Reset notifySent flag upon vbucket filter changes.

When the TAP producer receives the vbucket filter change
request from the cluster manager, it should set notifySent
flag to false, so that the TAP producer can send the TAP
OPAQUE response to the cluster manager immediately in order to
notify that the filter change request was successfully applied.

Otherwise, there can be 5 seconds unnecessary delay in sending
the reponse to the cluster manager.

Change-Id: Ie9e7720d6d2ff8a71040a95963b2065a4a58686e
Reviewed-on: http://review.couchbase.org/27156
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-8464: Fix crash in test_checkpoint_persistence
abhinavdangeti [Wed, 26 Jun 2013 22:48:36 +0000 (15:48 -0700)]
MB-8464: Fix crash in test_checkpoint_persistence

- test_checkpoint_persistence seen to be failing
sometimes only in Linux due to a race condition
- adjusting the checkpoint_flush_timeout before
notifying_IO_completion during notify_checkpoint_
persisted seems to have stopped this error from

Change-Id: I6804cb9dc566e9f0d2ed831c694feda19491762a
Reviewed-on: http://review.couchbase.org/27155
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
7 years agoMB-8244 add currentTask to track currently executing task
Jin Lim [Thu, 20 Jun 2013 23:09:38 +0000 (16:09 -0700)]
MB-8244 add currentTask to track currently executing task

Change-Id: I5d65ab1e2a959232d0a43abd410290344e343a6f
Reviewed-on: http://review.couchbase.org/27060
Reviewed-by: Jin Lim <jin@couchbase.com>
Tested-by: Jin Lim <jin@couchbase.com>
Jin Lim [Wed, 19 Jun 2013 22:15:09 +0000 (15:15 -0700)]

This new states will be useful for indicating which
exucutor thread (readers/writers) is waiting. This
will be available as part of readers/writers stat.

Change-Id: Id6179855ca2d430bc7d1356b40e231a1506be2b5
Reviewed-on: http://review.couchbase.org/27038
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Jin Lim <jin@couchbase.com>
7 years agoCBD-963: empty commit to create 2.1.1 branch 2.1.1r
Phil Labee [Wed, 19 Jun 2013 00:32:29 +0000 (17:32 -0700)]
CBD-963:  empty commit to create 2.1.1 branch

Change-Id: I26b122a697dc93e20347b79992c39f6233c162ca
Reviewed-on: http://review.couchbase.org/27016
Reviewed-by: Philip Labee <philip@couchbase.com>
Tested-by: Philip Labee <philip@couchbase.com>
7 years agoMB-8439 add decrDiskQueueSize method instead of undercount assert
Jin Lim [Wed, 12 Jun 2013 01:20:43 +0000 (18:20 -0700)]
MB-8439 add decrDiskQueueSize method instead of undercount assert

DiskQueueSuze is no longer a critical variable for flusher and
checkpoint manager's decision making logics. It is simply a stat
indicating current pending disk write.

At any given point if this stat is undercounted we should just log
a warning and continue instead of assert. Otherwise, the assert
could cause a serious data loss for the crashed node.

Change-Id: I97d7b0ed6e6348578c3f35c373de779a21f469ed
Reviewed-on: http://review.couchbase.org/26897
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-8415 change num_shards parameter to num_workers
Jin Lim [Thu, 6 Jun 2013 06:20:47 +0000 (23:20 -0700)]
MB-8415 change num_shards parameter to num_workers

The parameter is really for setting an approximate
total number of worker threads (num_workers) not
a total number of shards for given bucket (num_shards).

Change-Id: Ica52b0b03f4da4106f46a0a4bee3bee4957f9e01
Reviewed-on: http://review.couchbase.org/26772
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Reviewed-by: Jin Lim <jin@couchbase.com>
Tested-by: Jin Lim <jin@couchbase.com>
7 years agoMB-8413 Release CouchNotifier instances if not referenced
Chiyoung Seo [Sat, 8 Jun 2013 04:20:41 +0000 (21:20 -0700)]
MB-8413 Release CouchNotifier instances if not referenced

Change-Id: I88766f6e6da14107c0bb712b4d732448f5cc742d
Reviewed-on: http://review.couchbase.org/26845
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-8413 Release KVShard instances upon a bucket deletion.
Chiyoung Seo [Sat, 8 Jun 2013 01:04:12 +0000 (18:04 -0700)]
MB-8413 Release KVShard instances upon a bucket deletion.

Change-Id: I91db6eb20c30a37df98b23b7ac8799d0cbff7e64
Reviewed-on: http://review.couchbase.org/26846
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-8292: Switch engine onto snaphot stats add_stat callback
Mike Wiederhold [Thu, 6 Jun 2013 23:50:47 +0000 (16:50 -0700)]
MB-8292: Switch engine onto snaphot stats add_stat callback

The add_casted_stat function will switch the current engine off
of the thread so we need to switch it back on when entering the
snapshot stats add_stat callback.

Change-Id: I2208f37c0d028908a803176b2efabd867f8a06f3
Reviewed-on: http://review.couchbase.org/26811
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
7 years agoMB-8418 Handle stats key exceptions according to memcached errors
Chiyoung Seo [Thu, 6 Jun 2013 23:43:40 +0000 (16:43 -0700)]
MB-8418 Handle stats key exceptions according to memcached errors

Change-Id: Ibfe2c046b20f6913184dbc8a251d71451f5345d7
Reviewed-on: http://review.couchbase.org/26808
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-8370 add WorkLoadPolicy class 2.1.0r
Jin Lim [Mon, 3 Jun 2013 09:14:41 +0000 (02:14 -0700)]
MB-8370 add WorkLoadPolicy class

This new class defines and applies user defined workload
access pattern while global io manager launching worker
threads for a newly registered bucket.

Change-Id: I1905786fc67b9004118f11d83196d76f95dfb136
Reviewed-on: http://review.couchbase.org/26723
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Jin Lim <jin@couchbase.com>
7 years agoMB-8214 replace wake() with snooze()
Jin Lim [Sat, 25 May 2013 08:06:34 +0000 (01:06 -0700)]
MB-8214 replace wake() with snooze()

Calling IOManager::wake() method from running task is
unnecessary. This only causes redundant resorting of
already sorted futureQ of running thread.

Change-Id: Ife0afcd0dc7f977af314d40124e62237e3def6d6
Reviewed-on: http://review.couchbase.org/26720
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Jin Lim <jin@couchbase.com>
7 years agoMB-8214 remove unnecessary mutex notification
Jin Lim [Sat, 25 May 2013 08:18:39 +0000 (01:18 -0700)]
MB-8214 remove unnecessary mutex notification

ExecutorThread::reschedule() method gets called very
often. This does not need to call mutex notification
at all since the running thread itself calls the method
during its runtime.

Change-Id: I66b0fe60c82c604d173fdab8090bd7461fe2cece
Reviewed-on: http://review.couchbase.org/26721
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Jin Lim <jin@couchbase.com>
7 years agoMB-7735: Fix to a race in notifying / releasing TAP connections
Chiyoung Seo [Thu, 14 Mar 2013 21:05:24 +0000 (14:05 -0700)]
MB-7735: Fix to a race in notifying / releasing TAP connections

Releasing and notifying an TAP conneciton is now synchronized
through a separate lock.

Change-Id: I36deeeaab98235500d9ba22c213c413eac1fbc04
Reviewed-on: http://review.couchbase.org/26705
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years ago2.1.0 branch at build 2.0.2-824
Phil Labee [Tue, 4 Jun 2013 00:44:57 +0000 (17:44 -0700)]
2.1.0 branch at build 2.0.2-824

Change-Id: I1a7e793d1b5ff9b7b4c9e1a9212e3259dd155fd3
Reviewed-on: http://review.couchbase.org/26696
Reviewed-by: Philip Labee <philip@couchbase.com>
Tested-by: Philip Labee <philip@couchbase.com>
7 years agoMB-8383: Add stats for returning meta operations
Mike Wiederhold [Fri, 31 May 2013 17:43:44 +0000 (10:43 -0700)]
MB-8383: Add stats for returning meta operations

Change-Id: I53089da790c2f50303861ab3780d32ee8cf0c9f8
Reviewed-on: http://review.couchbase.org/26645
Reviewed-by: Jin Lim <jin@couchbase.com>
Tested-by: Michael Wiederhold <mike@couchbase.com>
7 years agoMB-8370 set number of writer to 1 when num_shard is 1
Jin Lim [Thu, 30 May 2013 22:27:38 +0000 (15:27 -0700)]
MB-8370 set number of writer to 1 when num_shard is 1

Without this fix, the number of writer would be set to 0 if
num_shard is 1

Change-Id: I5022ca77ba476019daa6eec3d4bd60dd1bf36532
Reviewed-on: http://review.couchbase.org/26650
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Jin Lim <jin@couchbase.com>
7 years agoMB-8380 Update the vbucket flush stats after commit
Chiyoung Seo [Fri, 31 May 2013 04:42:27 +0000 (21:42 -0700)]
MB-8380 Update the vbucket flush stats after commit

Otherwise, the disk drain stats keep incremented even if there
are some errors that cause dirty items to be requeued.

Change-Id: I4db7e54b898e9881298d53019ac3999df6ef109d
Reviewed-on: http://review.couchbase.org/26632
Reviewed-by: Jin Lim <jin@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-8214 & MB-8323 allocate iomanager threads per bucket
Jin Lim [Tue, 28 May 2013 19:16:13 +0000 (12:16 -0700)]
MB-8214 & MB-8323 allocate iomanager threads per bucket

Based on the latest litmus tests still the 2:1 ratio of
readers and writers threads renders the best result
without previously measured get perfroamce regression.

This commit include changes that allocate ExecutorThreads
per bucket while still maintaning single global IOManager
instance. Also these changes allow a better way of
handling of thread termination after the bucket shutdown,
which in turn eliminates the crash observed in MB-8323.

Change-Id: I01d045277911cc461f04ef34df5251ccfb397124
Reviewed-on: http://review.couchbase.org/26560
Tested-by: Jin Lim <jin@couchbase.com>
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
7 years agoMB-8378 Clean up all the TAP connections within a destroy thread
Chiyoung Seo [Thu, 30 May 2013 23:42:14 +0000 (16:42 -0700)]
MB-8378 Clean up all the TAP connections within a destroy thread

As a separate dedicated thread performs destroying an engine
instance, we can simply clean up all the TAP connections within
that destroy thread. In this way, it doesn't need to wait for
NON-IO dispather to complete cleaning up TAP connections.

Change-Id: I39f3373b2867a2357a4f802bdc4e6ba77f4f5c82
Reviewed-on: http://review.couchbase.org/26625
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-8259 Reflect checkpoint collasping impact on disk queue size.
Chiyoung Seo [Thu, 30 May 2013 19:58:35 +0000 (12:58 -0700)]
MB-8259 Reflect checkpoint collasping impact on disk queue size.

Multiple checkpoints on a replica vbucket can be collapsed into
a single checkpoint to reduce the memory overhead. We should
reflect the checkpoint collapsing impact on the disk queue size

Change-Id: I1b4d3fcd0055da160126dacf1a7e12bace2c550c
Reviewed-on: http://review.couchbase.org/26621
Reviewed-by: Jin Lim <jin@couchbase.com>
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-8214: Add xdcr conflict resolution to ep-engine 2.0.2r
Mike Wiederhold [Thu, 16 May 2013 23:39:37 +0000 (16:39 -0700)]
MB-8214: Add xdcr conflict resolution to ep-engine

This change adds the xdcr conflict resolution mechanism
to ep-engine and allows setMeta, addMeta, and delMeta to
do background fetches in order to decide whether or not we
should persist and item sent ot us by xdcr.

Change-Id: I8a94432b1179511a632a03e9cb73ccba10389119
Reviewed-on: http://review.couchbase.org/26422
Reviewed-by: Chiyoung Seo <chiyoung.seo@gmail.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>
7 years agoMB-8066 Check checkpoint persistence completion after flushing.
Chiyoung Seo [Wed, 22 May 2013 23:20:10 +0000 (16:20 -0700)]
MB-8066 Check checkpoint persistence completion after flushing.

If there are any checkpoint persistence requests for a given
vbucket, we should check its completion after flushing that
vbucket. Otherwise, it will mostly require another round of a
flusher cycle to check its completion.

Change-Id: Ic6359a89ec2d26d7c8d97d668edffa3ba960c613
Reviewed-on: http://review.couchbase.org/26482
Reviewed-by: Michael Wiederhold <mike@couchbase.com>
Reviewed-by: Jin Lim <jin@couchbase.com>
Tested-by: Chiyoung Seo <chiyoung.seo@gmail.com>