2014-07-21_16:43:42.28788 18:43:42.287 [info] Application lager started on node 'couchdb@couch3-testing.[TLD]'
2014-07-21_16:43:42.28790 18:43:42.287 [info] Application couch_log started on node 'couchdb@couch3-testing.[TLD]'
2014-07-21_16:43:42.29300 Apache CouchDB 8675de9 is starting.
2014-07-21_16:43:42.29491 18:43:42.294 [info] Starting couch_sup
2014-07-21_16:43:42.57452 18:43:42.574 [error] Failed to open crash log file log/crash.log with error: permission denied
2014-07-21_16:43:42.57877 18:43:42.578 [info] needed 180.801 ms to open new _users
2014-07-21_16:43:42.65086 18:43:42.650 [info] needed 44.74 ms to open new _replicator
2014-07-21_16:43:42.66147 Apache CouchDB has started. Time to relax.
2014-07-21_16:43:42.66174 18:43:42.661 [info] Apache CouchDB has started on http://127.0.0.1:5986/
2014-07-21_16:43:42.66186 18:43:42.661 [info] Apache CouchDB has started on https://127.0.0.1:6984/
2014-07-21_16:43:42.66219 18:43:42.661 [info] Application couch started on node 'couchdb@couch3-testing.[TLD]'
2014-07-21_16:43:42.66234 18:43:42.662 [info] Application ets_lru started on node 'couchdb@couch3-testing.[TLD]'
2014-07-21_16:43:42.68357 18:43:42.683 [info] Application rexi started on node 'couchdb@couch3-testing.[TLD]'
2014-07-21_16:43:42.73420 18:43:42.733 [info] needed 43.13 ms to open new nodes
2014-07-21_16:43:42.76451 18:43:42.764 [info] Application mem3 started on node 'couchdb@couch3-testing.[TLD]'
2014-07-21_16:43:42.76482 18:43:42.764 [info] Application fabric started on node 'couchdb@couch3-testing.[TLD]'
2014-07-21_16:43:43.00932 18:43:43.008 [info] needed 245.347 ms to open new dbs
2014-07-21_16:43:43.24089 18:43:43.240 [info] Application chttpd started on node 'couchdb@couch3-testing.[TLD]'
2014-07-21_16:43:43.47555 18:43:43.240 [info] Application couch_replicator started on node 'couchdb@couch3-testing.[TLD]'
2014-07-21_16:43:43.49874 18:43:43.498 [info] Application ddoc_cache started on node 'couchdb@couch3-testing.[TLD]'
2014-07-21_16:43:43.51102 18:43:43.510 [info] Application runtime_tools started on node 'couchdb@couch3-testing.[TLD]'
2014-07-21_16:43:43.51317 18:43:43.510 [info] Application snappy started on node 'couchdb@couch3-testing.[TLD]'
2014-07-21_16:44:09.38547 18:44:09.385 [notice] undefined - - GET / 200
2014-07-21_16:44:09.39234 18:44:09.392 [notice] e059716e [XXXX] undefined GET / 200 ok 24
2014-07-21_16:45:26.05557 18:45:26.054 [info] needed 15.117 ms to open new shards/40000000-5fffffff/eumetsat.1405961126
2014-07-21_16:45:26.05581 18:45:26.054 [info] needed 15.116 ms to open new shards/00000000-1fffffff/eumetsat.1405961126
2014-07-21_16:45:26.05644 18:45:26.054 [info] needed 15.259 ms to open new shards/20000000-3fffffff/eumetsat.1405961126
2014-07-21_16:45:26.06192 18:45:26.055 [info] needed 15.963 ms to open new shards/60000000-7fffffff/eumetsat.1405961126
2014-07-21_16:45:26.06217 18:45:26.055 [info] needed 17.152 ms to open new shards/80000000-9fffffff/eumetsat.1405961126
2014-07-21_16:45:26.06240 18:45:26.055 [info] needed 19.076 ms to open new shards/c0000000-dfffffff/eumetsat.1405961126
2014-07-21_16:45:26.06273 18:45:26.055 [info] needed 18.165 ms to open new shards/a0000000-bfffffff/eumetsat.1405961126
2014-07-21_16:45:26.06295 18:45:26.055 [info] needed 20.405 ms to open new shards/e0000000-ffffffff/eumetsat.1405961126
2014-07-21_16:45:26.35673 18:45:26.355 [error] gen_server couch_server terminated with reason: {key_exists,{1405,961126,354645}} in gb_trees:insert_1/4 line 324
2014-07-21_16:45:26.35742 18:45:26.355 [error] CRASH REPORT Process couch_server with 0 neighbours exited with reason: {key_exists,{1405,961126,354645}} in gb_trees:insert_1/4 line 324 in gen_server:terminate/6 line 744
2014-07-21_16:45:26.35788 18:45:26.355 [error] Supervisor couch_primary_services had child couch_server started with couch_server:sup_start_link() at <0.124.0> exit with reason {key_exists,{1405,961126,354645}} in gb_trees:insert_1/4 line 324 in context child_terminated
2014-07-21_16:45:26.35889 18:45:26.356 [error] gen_server couch_auth_cache terminated with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78
2014-07-21_16:45:26.35993 18:45:26.356 [error] CRASH REPORT Process couch_auth_cache with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:terminate/6 line 744
2014-07-21_16:45:26.36021 18:45:26.356 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at <0.174.0> exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context child_terminated
2014-07-21_16:45:26.36094 18:45:26.360 [info] needed 3.169 ms to open new _users
2014-07-21_16:45:26.36198 18:45:26.361 [info] needed 0.875 ms to open new shards/00000000-1fffffff/eumetsat.1405961126
2014-07-21_16:45:26.36563 18:45:26.365 [info] needed 2.666 ms to open new shards/40000000-5fffffff/eumetsat.1405961126
2014-07-21_16:45:26.36653 18:45:26.365 [info] needed 2.329 ms to open new shards/80000000-9fffffff/eumetsat.1405961126
2014-07-21_16:45:26.36685 18:45:26.365 [info] needed 2.553 ms to open new shards/c0000000-dfffffff/eumetsat.1405961126
2014-07-21_16:45:26.36707 18:45:26.365 [info] needed 3.093 ms to open new shards/20000000-3fffffff/eumetsat.1405961126
2014-07-21_16:45:26.36725 18:45:26.365 [info] needed 2.688 ms to open new shards/e0000000-ffffffff/eumetsat.1405961126
2014-07-21_16:45:26.36756 18:45:26.365 [error] rexi_server exit:noproc [{couch_db,collect_results,3,[{file,"src/couch_db.erl"},{line,930}]},{couch_db,write_and_commit,4,[{file,"src/couch_db.erl"},{line,942}]},{couch_db,update_docs,4,[{file,"src/couch_db.erl"},{line,881}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,165}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_16:45:26.36781 18:45:26.366 [info] needed 3.149 ms to open new shards/60000000-7fffffff/eumetsat.1405961126
2014-07-21_16:45:26.36797 18:45:26.366 [info] needed 3.265 ms to open new shards/a0000000-bfffffff/eumetsat.1405961126
2014-07-21_16:45:28.59188 18:45:28.591 [info] needed 77.318 ms to open new dbs
2014-07-21_16:47:34.82964 18:47:34.828 [notice] undefined - - GET / 200
2014-07-21_16:47:34.82972 18:47:34.829 [notice] ccd5d84d [XXXX] undefined GET / 200 ok 1
2014-07-21_16:49:36.72147 18:49:36.720 [notice] undefined - - GET / 200
2014-07-21_16:49:36.72171 18:49:36.721 [notice] 2b9cbc40 [XXXX] undefined GET / 200 ok 1
2014-07-21_16:52:59.45376 18:52:59.451 [notice] undefined - - GET / 200
2014-07-21_16:52:59.45421 18:52:59.453 [notice] 951bb438 [XXXX] undefined GET / 200 ok 2
2014-07-21_16:55:03.44132 18:55:03.440 [notice] undefined - - GET / 200
2014-07-21_16:55:03.44244 18:55:03.440 [notice] f0ba5e3b [XXXX] undefined GET / 200 ok 1
2014-07-21_16:58:28.25708 18:58:28.255 [notice] undefined - - GET / 200
2014-07-21_16:58:28.25731 18:58:28.257 [notice] 41686e67 [XXXX] undefined GET / 200 ok 2
2014-07-21_17:00:19.19973 19:00:19.194 [error] Error in process <0.4826.0> on node 'couchdb@couch3-testing.[TLD]' with exit value: {{rexi_EXIT,{shutdown,[{couch_db,collect_results,3,[{file,"src/couch_db.erl"},{line,930}]},{couch_db,write_and_commit,4,[{file,"src/couch_db.erl"},{line,942}]},{couch_db,update_docs,4,[{file,"src/couch_db.erl"},{line,811}]},{fabric_rpc... 
2014-07-21_17:00:19.19976 
2014-07-21_17:00:19.19976 
2014-07-21_17:00:19.20101 19:00:19.194 [warning] mem3_sync shards/60000000-7fffffff/eumetsat.1405961126 couchdb@couch1-testing.[TLD] {{rexi_EXIT,{shutdown,[{couch_db,collect_results,3,[{file,[115,114,99,47,99,111,117,99,104,95,100,98,46,101,114,108]},{line,930}]},{couch_db,write_and_commit,4,[{file,[115,114,99,47,99,111,117,99,104,95,100,98,46,101,114,108]},{line,942}]},{couch_db,update_docs,4,[{file,[115,114,99,47,99,111,117,99,104,95,100,98,46,101,114,108]},{line,811}]},{fabric_rpc,with_db,3,[{file,[115,114,99,47,102,97,98,114,105,99,95,114,112,99,46,101,114,108]},{line,165}]},{rexi_server,init_p,3,[{file,[115,114,99,47,114,101,120,105,95,115,101,114,118,101,114,46,101,114,108]},{line,151}]}]}},[{mem3_rep,rexi_call,2,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,193}]},{mem3_rep,save_on_target,3,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,171}]},{mem3_rep,replicate_batch,1,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,148}]},{mem3_rep,repl,2,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,93}]},{mem3_rep,go,1,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,69}]},{mem3_sync,'-start_push_replication/1-fun-0-',2,[{file,[115,114,99,47,109,101,109,51,95,115,121,110,99,46,101,114,108]},{line,209}]}]}
2014-07-21_17:00:19.20459 19:00:19.204 [warning] mem3_sync shards/60000000-7fffffff/eumetsat.1405961126 couchdb@couch2-testing.[TLD] {{rexi_EXIT,{shutdown,[{couch_db,collect_results,3,[{file,[115,114,99,47,99,111,117,99,104,95,100,98,46,101,114,108]},{line,930}]},{couch_db,write_and_commit,4,[{file,[115,114,99,47,99,111,117,99,104,95,100,98,46,101,114,108]},{line,942}]},{couch_db,update_docs,4,[{file,[115,114,99,47,99,111,117,99,104,95,100,98,46,101,114,108]},{line,881}]},{fabric_rpc,with_db,3,[{file,[115,114,99,47,102,97,98,114,105,99,95,114,112,99,46,101,114,108]},{line,165}]},{rexi_server,init_p,3,[{file,[115,114,99,47,114,101,120,105,95,115,101,114,118,101,114,46,101,114,108]},{line,151}]}]}},[{mem3_rep,rexi_call,2,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,193}]},{mem3_rep,replicate_batch,1,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,150}]},{mem3_rep,repl,2,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,93}]},{mem3_rep,go,1,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,69}]},{mem3_sync,'-start_push_replication/1-fun-0-',2,[{file,[115,114,99,47,109,101,109,51,95,115,121,110,99,46,101,114,108]},{line,209}]}]}
2014-07-21_17:00:19.20530 19:00:19.204 [error] Error in process <0.4827.0> on node 'couchdb@couch3-testing.[TLD]' with exit value: {{rexi_EXIT,{shutdown,[{couch_db,collect_results,3,[{file,"src/couch_db.erl"},{line,930}]},{couch_db,write_and_commit,4,[{file,"src/couch_db.erl"},{line,942}]},{couch_db,update_docs,4,[{file,"src/couch_db.erl"},{line,881}]},{fabric_rpc... 
2014-07-21_17:00:19.20532 
2014-07-21_17:00:19.20533 
2014-07-21_17:00:20.13382 19:00:20.132 [error] Error in process <0.4985.0> on node 'couchdb@couch3-testing.[TLD]' with exit value: {{rexi_EXIT,{shutdown,[{couch_db,collect_results,3,[{file,"src/couch_db.erl"},{line,930}]},{couch_db,write_and_commit,4,[{file,"src/couch_db.erl"},{line,942}]},{couch_db,update_docs,4,[{file,"src/couch_db.erl"},{line,881}]},{fabric_rpc... 
2014-07-21_17:00:20.13384 
2014-07-21_17:00:20.13385 
2014-07-21_17:00:20.13405 19:00:20.133 [warning] mem3_sync shards/20000000-3fffffff/eumetsat.1405961126 couchdb@couch1-testing.[TLD] {{rexi_EXIT,{shutdown,[{couch_db,collect_results,3,[{file,[115,114,99,47,99,111,117,99,104,95,100,98,46,101,114,108]},{line,930}]},{couch_db,write_and_commit,4,[{file,[115,114,99,47,99,111,117,99,104,95,100,98,46,101,114,108]},{line,942}]},{couch_db,update_docs,4,[{file,[115,114,99,47,99,111,117,99,104,95,100,98,46,101,114,108]},{line,881}]},{fabric_rpc,with_db,3,[{file,[115,114,99,47,102,97,98,114,105,99,95,114,112,99,46,101,114,108]},{line,165}]},{rexi_server,init_p,3,[{file,[115,114,99,47,114,101,120,105,95,115,101,114,118,101,114,46,101,114,108]},{line,151}]}]}},[{mem3_rep,rexi_call,2,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,193}]},{mem3_rep,replicate_batch,1,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,150}]},{mem3_rep,repl,2,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,93}]},{mem3_rep,go,1,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,69}]},{mem3_sync,'-start_push_replication/1-fun-0-',2,[{file,[115,114,99,47,109,101,109,51,95,115,121,110,99,46,101,114,108]},{line,209}]}]}
2014-07-21_17:00:24.30773 19:00:24.306 [error] Error in process <0.5777.0> on node 'couchdb@couch3-testing.[TLD]' with exit value: {{rexi_EXIT,{shutdown,[{couch_db,collect_results,3,[{file,"src/couch_db.erl"},{line,930}]},{couch_db,write_and_commit,4,[{file,"src/couch_db.erl"},{line,942}]},{couch_db,update_docs,4,[{file,"src/couch_db.erl"},{line,881}]},{fabric_rpc... 
2014-07-21_17:00:24.30773 
2014-07-21_17:00:24.30773 
2014-07-21_17:00:24.30808 19:00:24.307 [warning] mem3_sync shards/60000000-7fffffff/eumetsat.1405961126 couchdb@couch1-testing.[TLD] {{rexi_EXIT,{shutdown,[{couch_db,collect_results,3,[{file,[115,114,99,47,99,111,117,99,104,95,100,98,46,101,114,108]},{line,930}]},{couch_db,write_and_commit,4,[{file,[115,114,99,47,99,111,117,99,104,95,100,98,46,101,114,108]},{line,942}]},{couch_db,update_docs,4,[{file,[115,114,99,47,99,111,117,99,104,95,100,98,46,101,114,108]},{line,881}]},{fabric_rpc,with_db,3,[{file,[115,114,99,47,102,97,98,114,105,99,95,114,112,99,46,101,114,108]},{line,165}]},{rexi_server,init_p,3,[{file,[115,114,99,47,114,101,120,105,95,115,101,114,118,101,114,46,101,114,108]},{line,151}]}]}},[{mem3_rep,rexi_call,2,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,193}]},{mem3_rep,replicate_batch,1,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,150}]},{mem3_rep,repl,2,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,93}]},{mem3_rep,go,1,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,69}]},{mem3_sync,'-start_push_replication/1-fun-0-',2,[{file,[115,114,99,47,109,101,109,51,95,115,121,110,99,46,101,114,108]},{line,209}]}]}
2014-07-21_17:00:24.30953 19:00:24.308 [error] Error in process <0.5796.0> on node 'couchdb@couch3-testing.[TLD]' with exit value: {{rexi_EXIT,{badarg,[{ets,lookup,[couch_dbs,<<44 bytes>>],[]},{couch_server,open,2,[{file,"src/couch_server.erl"},{line,78}]},{fabric_rpc,get_or_create_db,2,[{file,"src/fabric_rpc.erl"},{line,178}]},{fabric_rpc,get_missing_revs,3,[{file... 
2014-07-21_17:00:24.30953 
2014-07-21_17:00:24.30953 
2014-07-21_17:00:24.30998 19:00:24.309 [warning] mem3_sync shards/c0000000-dfffffff/eumetsat.1405961126 couchdb@couch1-testing.[TLD] {{rexi_EXIT,{badarg,[{ets,lookup,[couch_dbs,<<115,104,97,114,100,115,47,99,48,48,48,48,48,48,48,45,100,102,102,102,102,102,102,102,47,101,117,109,101,116,115,97,116,46,49,52,48,53,57,54,49,49,50,54>>],[]},{couch_server,open,2,[{file,[115,114,99,47,99,111,117,99,104,95,115,101,114,118,101,114,46,101,114,108]},{line,78}]},{fabric_rpc,get_or_create_db,2,[{file,[115,114,99,47,102,97,98,114,105,99,95,114,112,99,46,101,114,108]},{line,178}]},{fabric_rpc,get_missing_revs,3,[{file,[115,114,99,47,102,97,98,114,105,99,95,114,112,99,46,101,114,108]},{line,119}]},{rexi_server,init_p,3,[{file,[115,114,99,47,114,101,120,105,95,115,101,114,118,101,114,46,101,114,108]},{line,151}]}]}},[{mem3_rep,rexi_call,2,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,193}]},{mem3_rep,replicate_batch,1,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,144}]},{mem3_rep,repl,2,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,93}]},{mem3_rep,go,1,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,69}]},{mem3_sync,'-start_push_replication/1-fun-0-',2,[{file,[115,114,99,47,109,101,109,51,95,115,121,110,99,46,101,114,108]},{line,209}]}]}
2014-07-21_17:00:25.17400 19:00:25.173 [warning] mem3_sync shards/80000000-9fffffff/eumetsat.1405961126 couchdb@couch1-testing.[TLD] {{rexi_EXIT,{badarg,[{ets,lookup,[couch_dbs,<<115,104,97,114,100,115,47,56,48,48,48,48,48,48,48,45,57,102,102,102,102,102,102,102,47,101,117,109,101,116,115,97,116,46,49,52,48,53,57,54,49,49,50,54>>],[]},{couch_server,open,2,[{file,[115,114,99,47,99,111,117,99,104,95,115,101,114,118,101,114,46,101,114,108]},{line,78}]},{fabric_rpc,get_or_create_db,2,[{file,[115,114,99,47,102,97,98,114,105,99,95,114,112,99,46,101,114,108]},{line,178}]},{fabric_rpc,with_db,3,[{file,[115,114,99,47,102,97,98,114,105,99,95,114,112,99,46,101,114,108]},{line,162}]},{rexi_server,init_p,3,[{file,[115,114,99,47,114,101,120,105,95,115,101,114,118,101,114,46,101,114,108]},{line,151}]}]}},[{mem3_rep,rexi_call,2,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,193}]},{mem3_rep,replicate_batch,1,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,150}]},{mem3_rep,repl,2,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,93}]},{mem3_rep,go,1,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,69}]},{mem3_sync,'-start_push_replication/1-fun-0-',2,[{file,[115,114,99,47,109,101,109,51,95,115,121,110,99,46,101,114,108]},{line,209}]}]}
2014-07-21_17:00:25.17487 19:00:25.174 [warning] mem3_sync shards/20000000-3fffffff/eumetsat.1405961126 couchdb@couch1-testing.[TLD] {{rexi_EXIT,{badarg,[{ets,lookup,[couch_dbs,<<115,104,97,114,100,115,47,50,48,48,48,48,48,48,48,45,51,102,102,102,102,102,102,102,47,101,117,109,101,116,115,97,116,46,49,52,48,53,57,54,49,49,50,54>>],[]},{couch_server,open,2,[{file,[115,114,99,47,99,111,117,99,104,95,115,101,114,118,101,114,46,101,114,108]},{line,78}]},{fabric_rpc,get_or_create_db,2,[{file,[115,114,99,47,102,97,98,114,105,99,95,114,112,99,46,101,114,108]},{line,178}]},{fabric_rpc,get_missing_revs,3,[{file,[115,114,99,47,102,97,98,114,105,99,95,114,112,99,46,101,114,108]},{line,119}]},{rexi_server,init_p,3,[{file,[115,114,99,47,114,101,120,105,95,115,101,114,118,101,114,46,101,114,108]},{line,151}]}]}},[{mem3_rep,rexi_call,2,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,193}]},{mem3_rep,replicate_batch,1,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,144}]},{mem3_rep,repl,2,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,93}]},{mem3_rep,go,1,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,69}]},{mem3_sync,'-start_push_replication/1-fun-0-',2,[{file,[115,114,99,47,109,101,109,51,95,115,121,110,99,46,101,114,108]},{line,209}]}]}
2014-07-21_17:00:25.17554 19:00:25.175 [error] Error in process <0.5906.0> on node 'couchdb@couch3-testing.[TLD]' with exit value: {{rexi_EXIT,{badarg,[{ets,lookup,[couch_dbs,<<44 bytes>>],[]},{couch_server,open,2,[{file,"src/couch_server.erl"},{line,78}]},{fabric_rpc,get_or_create_db,2,[{file,"src/fabric_rpc.erl"},{line,178}]},{fabric_rpc,with_db,3,[{file,"src/fab... 
2014-07-21_17:00:25.17554 
2014-07-21_17:00:25.17554 
2014-07-21_17:00:25.17600 19:00:25.175 [error] Error in process <0.5930.0> on node 'couchdb@couch3-testing.[TLD]' with exit value: {{rexi_EXIT,{badarg,[{ets,lookup,[couch_dbs,<<44 bytes>>],[]},{couch_server,open,2,[{file,"src/couch_server.erl"},{line,78}]},{fabric_rpc,get_or_create_db,2,[{file,"src/fabric_rpc.erl"},{line,178}]},{fabric_rpc,get_missing_revs,3,[{file... 
2014-07-21_17:00:25.17603 
2014-07-21_17:00:25.17603 
2014-07-21_17:00:28.60008 19:00:28.598 [error] Failed to get security object on {'couchdb@couch1-testing.[TLD]',<<"shards/80000000-9fffffff/eumetsat.1405961126">>} :: {rexi_EXIT,{badarg,[{ets,lookup,[couch_dbs,<<"shards/80000000-9fffffff/eumetsat.1405961126">>],[]},{couch_server,open,2,[{file,"src/couch_server.erl"},{line,78}]},{fabric_rpc,get_or_create_db,2,[{file,"src/fabric_rpc.erl"},{line,178}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,162}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]}}
2014-07-21_17:00:28.60072 19:00:28.598 [error] Error checking security objects for eumetsat :: {error,no_majority}
2014-07-21_17:00:28.60096 19:00:28.599 [error] Error in process <0.6451.0> on node 'couchdb@couch3-testing.[TLD]' with exit value: {{rexi_EXIT,{badarg,[{ets,lookup,[couch_dbs,<<44 bytes>>],[]},{couch_server,open,2,[{file,"src/couch_server.erl"},{line,78}]},{fabric_rpc,get_or_create_db,2,[{file,"src/fabric_rpc.erl"},{line,178}]},{fabric_rpc,with_db,3,[{file,"src/fab... 
2014-07-21_17:00:28.60096 
2014-07-21_17:00:28.60096 
2014-07-21_17:00:28.60130 19:00:28.599 [warning] mem3_sync shards/80000000-9fffffff/eumetsat.1405961126 couchdb@couch1-testing.[TLD] {{rexi_EXIT,{badarg,[{ets,lookup,[couch_dbs,<<115,104,97,114,100,115,47,56,48,48,48,48,48,48,48,45,57,102,102,102,102,102,102,102,47,101,117,109,101,116,115,97,116,46,49,52,48,53,57,54,49,49,50,54>>],[]},{couch_server,open,2,[{file,[115,114,99,47,99,111,117,99,104,95,115,101,114,118,101,114,46,101,114,108]},{line,78}]},{fabric_rpc,get_or_create_db,2,[{file,[115,114,99,47,102,97,98,114,105,99,95,114,112,99,46,101,114,108]},{line,178}]},{fabric_rpc,with_db,3,[{file,[115,114,99,47,102,97,98,114,105,99,95,114,112,99,46,101,114,108]},{line,162}]},{rexi_server,init_p,3,[{file,[115,114,99,47,114,101,120,105,95,115,101,114,118,101,114,46,101,114,108]},{line,151}]}]}},[{mem3_rep,rexi_call,2,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,193}]},{mem3_rep,calculate_start_seq,3,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,207}]},{mem3_rep,repl,2,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,89}]},{mem3_rep,go,1,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,69}]},{mem3_sync,'-start_push_replication/1-fun-0-',2,[{file,[115,114,99,47,109,101,109,51,95,115,121,110,99,46,101,114,108]},{line,209}]}]}
2014-07-21_17:00:28.60167 19:00:28.599 [error] Error in process <0.6435.0> on node 'couchdb@couch3-testing.[TLD]' with exit value: {{rexi_EXIT,{badarg,[{ets,lookup,[couch_dbs,<<44 bytes>>],[]},{couch_server,open,2,[{file,"src/couch_server.erl"},{line,78}]},{fabric_rpc,get_or_create_db,2,[{file,"src/fabric_rpc.erl"},{line,178}]},{fabric_rpc,with_db,3,[{file,"src/fab... 
2014-07-21_17:00:28.60168 
2014-07-21_17:00:28.60169 
2014-07-21_17:00:28.60183 19:00:28.600 [warning] mem3_sync shards/e0000000-ffffffff/eumetsat.1405961126 couchdb@couch1-testing.[TLD] {{rexi_EXIT,{badarg,[{ets,lookup,[couch_dbs,<<115,104,97,114,100,115,47,101,48,48,48,48,48,48,48,45,102,102,102,102,102,102,102,102,47,101,117,109,101,116,115,97,116,46,49,52,48,53,57,54,49,49,50,54>>],[]},{couch_server,open,2,[{file,[115,114,99,47,99,111,117,99,104,95,115,101,114,118,101,114,46,101,114,108]},{line,78}]},{fabric_rpc,get_or_create_db,2,[{file,[115,114,99,47,102,97,98,114,105,99,95,114,112,99,46,101,114,108]},{line,178}]},{fabric_rpc,with_db,3,[{file,[115,114,99,47,102,97,98,114,105,99,95,114,112,99,46,101,114,108]},{line,162}]},{rexi_server,init_p,3,[{file,[115,114,99,47,114,101,120,105,95,115,101,114,118,101,114,46,101,114,108]},{line,151}]}]}},[{mem3_rep,rexi_call,2,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,193}]},{mem3_rep,save_on_target,3,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,171}]},{mem3_rep,replicate_batch,1,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,148}]},{mem3_rep,repl,2,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,93}]},{mem3_rep,go,1,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,69}]},{mem3_sync,'-start_push_replication/1-fun-0-',2,[{file,[115,114,99,47,109,101,109,51,95,115,121,110,99,46,101,114,108]},{line,209}]}]}
2014-07-21_17:00:29.45897 19:00:29.457 [warning] mem3_sync shards/c0000000-dfffffff/eumetsat.1405961126 couchdb@couch2-testing.[TLD] shutdown
2014-07-21_17:00:29.46121 19:00:29.457 [error] rexi_server exit:shutdown [{couch_db,collect_results,3,[{file,"src/couch_db.erl"},{line,930}]},{couch_db,write_and_commit,4,[{file,"src/couch_db.erl"},{line,942}]},{couch_db,update_docs,4,[{file,"src/couch_db.erl"},{line,881}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,165}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:29.46185 19:00:29.458 [error] gen_server couch_server terminated with reason: {key_exists,{1405,962029,457537}} in gb_trees:insert_1/4 line 324
2014-07-21_17:00:29.46254 19:00:29.458 [error] CRASH REPORT Process couch_server with 0 neighbours exited with reason: {key_exists,{1405,962029,457537}} in gb_trees:insert_1/4 line 324 in gen_server:terminate/6 line 744
2014-07-21_17:00:29.47062 19:00:29.459 [error] Supervisor couch_primary_services had child couch_server started with couch_server:sup_start_link() at <0.520.0> exit with reason {key_exists,{1405,962029,457537}} in gb_trees:insert_1/4 line 324 in context child_terminated
2014-07-21_17:00:29.47130 19:00:29.459 [error] gen_server couch_auth_cache terminated with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78
2014-07-21_17:00:29.47174 19:00:29.459 [error] CRASH REPORT Process couch_auth_cache with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:terminate/6 line 744
2014-07-21_17:00:29.47212 19:00:29.459 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at <0.521.0> exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context child_terminated
2014-07-21_17:00:29.47271 19:00:29.459 [error] rexi_server error:badarg [{ets,lookup,[couch_dbs,<<"shards/c0000000-dfffffff/eumetsat.1405961126">>],[]},{couch_server,open,2,[{file,"src/couch_server.erl"},{line,78}]},{fabric_rpc,get_or_create_db,2,[{file,"src/fabric_rpc.erl"},{line,178}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,162}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:29.47326 19:00:29.460 [error] rexi_server error:badarg [{ets,lookup,[couch_dbs,<<"shards/c0000000-dfffffff/eumetsat.1405961126">>],[]},{couch_server,open,2,[{file,"src/couch_server.erl"},{line,78}]},{fabric_rpc,get_or_create_db,2,[{file,"src/fabric_rpc.erl"},{line,178}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,162}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:29.47934 19:00:29.460 [error] rexi_server error:badarg [{ets,lookup,[couch_dbs,<<"shards/c0000000-dfffffff/eumetsat.1405961126">>],[]},{couch_server,open,2,[{file,"src/couch_server.erl"},{line,78}]},{fabric_rpc,get_or_create_db,2,[{file,"src/fabric_rpc.erl"},{line,178}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,162}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:29.47997 19:00:29.460 [error] Error in process <0.6614.0> on node 'couchdb@couch3-testing.[TLD]' with exit value: {badarg,[{ets,lookup,[couch_dbs,<<44 bytes>>],[]},{couch_server,open,2,[{file,"src/couch_server.erl"},{line,78}]},{couch_db,open,2,[{file,"src/couch_db.erl"},{line,81}]},{mem3_rep,go,1,[{file,"src/mem3_rep.erl"},{line,65}]},{mem3_sync,'-start_push_replication/1-fun-0-'... 
2014-07-21_17:00:29.47999 
2014-07-21_17:00:29.47999 
2014-07-21_17:00:29.48031 19:00:29.460 [warning] mem3_sync shards/c0000000-dfffffff/eumetsat.1405961126 couchdb@couch1-testing.[TLD] {badarg,[{ets,lookup,[couch_dbs,<<115,104,97,114,100,115,47,99,48,48,48,48,48,48,48,45,100,102,102,102,102,102,102,102,47,101,117,109,101,116,115,97,116,46,49,52,48,53,57,54,49,49,50,54>>],[]},{couch_server,open,2,[{file,[115,114,99,47,99,111,117,99,104,95,115,101,114,118,101,114,46,101,114,108]},{line,78}]},{couch_db,open,2,[{file,[115,114,99,47,99,111,117,99,104,95,100,98,46,101,114,108]},{line,81}]},{mem3_rep,go,1,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,65}]},{mem3_sync,'-start_push_replication/1-fun-0-',2,[{file,[115,114,99,47,109,101,109,51,95,115,121,110,99,46,101,114,108]},{line,209}]}]}
2014-07-21_17:00:29.48086 19:00:29.460 [error] Failed to get security object on {'couchdb@couch3-testing.[TLD]',<<"shards/c0000000-dfffffff/eumetsat.1405961126">>} :: {rexi_EXIT,{badarg,[{ets,lookup,[couch_dbs,<<"shards/c0000000-dfffffff/eumetsat.1405961126">>],[]},{couch_server,open,2,[{file,"src/couch_server.erl"},{line,78}]},{fabric_rpc,get_or_create_db,2,[{file,"src/fabric_rpc.erl"},{line,178}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,162}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]}}
2014-07-21_17:00:29.48127 19:00:29.461 [error] Error checking security objects for eumetsat :: {error,no_majority}
2014-07-21_17:00:29.48150 19:00:29.462 [info] needed 0.994 ms to open new _users
2014-07-21_17:00:29.48173 19:00:29.471 [info] needed 0.94 ms to open new shards/c0000000-dfffffff/eumetsat.1405961126
2014-07-21_17:00:29.48196 19:00:29.479 [warning] mem3_sync shards/c0000000-dfffffff/eumetsat.1405961126 couchdb@couch2-testing.[TLD] noproc
2014-07-21_17:00:30.17707 19:00:30.176 [info] needed 0.961 ms to open new shards/80000000-9fffffff/eumetsat.1405961126
2014-07-21_17:00:30.18008 19:00:30.179 [info] needed 3.842 ms to open new shards/20000000-3fffffff/eumetsat.1405961126
2014-07-21_17:00:30.27272 19:00:30.271 [info] needed 0.477 ms to open new shards/40000000-5fffffff/eumetsat.1405961126
2014-07-21_17:00:32.60201 19:00:32.601 [info] needed 49.607 ms to open new shards/60000000-7fffffff/eumetsat.1405961126
2014-07-21_17:00:32.60241 19:00:32.601 [info] needed 49.732 ms to open new shards/00000000-1fffffff/eumetsat.1405961126
2014-07-21_17:00:32.74522 19:00:32.744 [info] needed 192.763 ms to open new shards/a0000000-bfffffff/eumetsat.1405961126
2014-07-21_17:00:32.74573 19:00:32.744 [info] needed 192.825 ms to open new shards/e0000000-ffffffff/eumetsat.1405961126
2014-07-21_17:00:33.60330 19:00:33.601 [error] rexi_server exit:shutdown [{couch_db,collect_results,3,[{file,"src/couch_db.erl"},{line,930}]},{couch_db,write_and_commit,4,[{file,"src/couch_db.erl"},{line,942}]},{couch_db,update_docs,4,[{file,"src/couch_db.erl"},{line,881}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,165}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:33.60371 19:00:33.601 [warning] mem3_sync shards/60000000-7fffffff/eumetsat.1405961126 couchdb@couch2-testing.[TLD] shutdown
2014-07-21_17:00:33.60426 19:00:33.602 [warning] mem3_sync shards/60000000-7fffffff/eumetsat.1405961126 couchdb@couch1-testing.[TLD] shutdown
2014-07-21_17:00:33.62147 19:00:33.607 [error] gen_server couch_server terminated with reason: {key_exists,{1405,962033,601781}} in gb_trees:insert_1/4 line 324
2014-07-21_17:00:33.62287 19:00:33.607 [error] CRASH REPORT Process couch_server with 0 neighbours exited with reason: {key_exists,{1405,962033,601781}} in gb_trees:insert_1/4 line 324 in gen_server:terminate/6 line 744
2014-07-21_17:00:33.62337 19:00:33.608 [error] Supervisor couch_primary_services had child couch_server started with couch_server:sup_start_link() at <0.6609.0> exit with reason {key_exists,{1405,962033,601781}} in gb_trees:insert_1/4 line 324 in context child_terminated
2014-07-21_17:00:33.62421 19:00:33.609 [error] gen_server couch_auth_cache terminated with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78
2014-07-21_17:00:33.62463 19:00:33.609 [error] CRASH REPORT Process couch_auth_cache with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:terminate/6 line 744
2014-07-21_17:00:33.62547 19:00:33.609 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at <0.6611.0> exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context child_terminated
2014-07-21_17:00:33.62595 19:00:33.610 [error] CRASH REPORT Process <0.6800.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:33.62712 19:00:33.610 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at <0.6611.0> exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:33.62764 19:00:33.610 [error] CRASH REPORT Process <0.6803.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:33.62807 19:00:33.610 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.6611.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:33.62864 19:00:33.611 [error] CRASH REPORT Process <0.6807.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:33.62901 19:00:33.611 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.6611.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:33.62936 19:00:33.611 [error] CRASH REPORT Process <0.6810.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:33.62962 19:00:33.611 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.6611.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:33.63033 19:00:33.611 [error] CRASH REPORT Process <0.6813.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:33.63080 19:00:33.612 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.6611.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:33.63137 19:00:33.612 [error] CRASH REPORT Process <0.6816.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:33.63948 19:00:33.612 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.6611.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:33.63990 19:00:33.612 [error] CRASH REPORT Process <0.6819.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:33.64024 19:00:33.612 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.6611.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:33.64068 19:00:33.613 [error] CRASH REPORT Process <0.6822.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:33.64105 19:00:33.613 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.6611.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:33.64149 19:00:33.613 [error] CRASH REPORT Process <0.6825.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:33.64196 19:00:33.613 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.6611.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:33.64232 19:00:33.614 [error] CRASH REPORT Process <0.6828.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:33.64265 19:00:33.614 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.6611.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:33.64313 19:00:33.614 [error] CRASH REPORT Process <0.6831.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:33.64376 19:00:33.614 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.6611.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:33.64456 19:00:33.614 [error] CRASH REPORT Process <0.6834.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:33.64550 19:00:33.615 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.6611.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:33.64604 19:00:33.615 [error] CRASH REPORT Process <0.6837.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:33.64638 19:00:33.615 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.6611.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:33.64686 19:00:33.616 [error] CRASH REPORT Process <0.6840.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:33.64741 19:00:33.616 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.6611.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:33.64780 19:00:33.616 [error] CRASH REPORT Process <0.6843.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:33.64819 19:00:33.616 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.6611.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:33.64853 19:00:33.616 [error] CRASH REPORT Process <0.6846.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:33.64960 19:00:33.617 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.6611.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:33.65060 19:00:33.617 [error] CRASH REPORT Process <0.6849.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:33.65107 19:00:33.617 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.6611.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:33.65144 19:00:33.617 [error] CRASH REPORT Process <0.6852.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:33.65198 19:00:33.617 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.6611.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:33.65309 19:00:33.618 [error] CRASH REPORT Process <0.6855.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:33.65442 19:00:33.618 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.6611.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:33.65546 19:00:33.618 [error] CRASH REPORT Process <0.6858.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:33.65648 19:00:33.618 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.6611.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:33.65776 19:00:33.622 [info] needed 1.207 ms to open new shards/80000000-9fffffff/eumetsat.1405961126
2014-07-21_17:00:33.65965 19:00:33.622 [warning] mem3_sync shards/80000000-9fffffff/eumetsat.1405961126 couchdb@couch1-testing.[TLD] noproc
2014-07-21_17:00:33.66036 19:00:33.657 [info] needed 36.812 ms to open new _users
2014-07-21_17:00:33.66059 19:00:33.658 [info] needed 35.903 ms to open new shards/60000000-7fffffff/eumetsat.1405961126
2014-07-21_17:00:33.66087 19:00:33.659 [warning] mem3_sync shards/e0000000-ffffffff/eumetsat.1405961126 couchdb@couch1-testing.[TLD] noproc
2014-07-21_17:00:33.66118 19:00:33.659 [warning] mem3_sync shards/60000000-7fffffff/eumetsat.1405961126 couchdb@couch2-testing.[TLD] noproc
2014-07-21_17:00:33.66137 19:00:33.660 [warning] mem3_sync shards/60000000-7fffffff/eumetsat.1405961126 couchdb@couch1-testing.[TLD] noproc
2014-07-21_17:00:33.69172 19:00:33.691 [info] needed 0.355 ms to open new shards/e0000000-ffffffff/eumetsat.1405961126
2014-07-21_17:00:33.71576 19:00:33.715 [info] needed 0.852 ms to open new shards/40000000-5fffffff/eumetsat.1405961126
2014-07-21_17:00:33.77649 19:00:33.776 [info] needed 0.795 ms to open new dbs
2014-07-21_17:00:33.79796 19:00:33.795 [error] rexi_server exit:shutdown [{couch_db,collect_results,3,[{file,"src/couch_db.erl"},{line,930}]},{couch_db,write_and_commit,4,[{file,"src/couch_db.erl"},{line,942}]},{couch_db,update_docs,4,[{file,"src/couch_db.erl"},{line,881}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,165}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:33.79925 19:00:33.795 [error] rexi_server exit:shutdown [{couch_db,collect_results,3,[{file,"src/couch_db.erl"},{line,930}]},{couch_db,write_and_commit,4,[{file,"src/couch_db.erl"},{line,942}]},{couch_db,update_docs,4,[{file,"src/couch_db.erl"},{line,811}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,165}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:33.80016 19:00:33.795 [error] rexi_server exit:shutdown [{couch_db,collect_results,3,[{file,"src/couch_db.erl"},{line,930}]},{couch_db,write_and_commit,4,[{file,"src/couch_db.erl"},{line,942}]},{couch_db,update_docs,4,[{file,"src/couch_db.erl"},{line,881}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,165}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:33.80111 19:00:33.795 [error] gen_server couch_server terminated with reason: {key_exists,{1405,962033,794896}} in gb_trees:insert_1/4 line 324
2014-07-21_17:00:33.80160 19:00:33.796 [error] CRASH REPORT Process couch_server with 0 neighbours exited with reason: {key_exists,{1405,962033,794896}} in gb_trees:insert_1/4 line 324 in gen_server:terminate/6 line 744
2014-07-21_17:00:33.80232 19:00:33.796 [error] Supervisor couch_primary_services had child couch_server started with couch_server:sup_start_link() at <0.6799.0> exit with reason {key_exists,{1405,962033,794896}} in gb_trees:insert_1/4 line 324 in context child_terminated
2014-07-21_17:00:33.80280 19:00:33.796 [error] rexi_server error:badarg [{ets,lookup,[couch_dbs,<<"shards/60000000-7fffffff/eumetsat.1405961126">>],[]},{couch_server,open,2,[{file,"src/couch_server.erl"},{line,78}]},{fabric_rpc,get_or_create_db,2,[{file,"src/fabric_rpc.erl"},{line,178}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,162}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:33.80313 19:00:33.796 [error] gen_server couch_auth_cache terminated with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78
2014-07-21_17:00:33.80398 19:00:33.796 [error] rexi_server error:badarg [{ets,lookup,[couch_dbs,<<"shards/60000000-7fffffff/eumetsat.1405961126">>],[]},{couch_server,open,2,[{file,"src/couch_server.erl"},{line,78}]},{fabric_rpc,get_or_create_db,2,[{file,"src/fabric_rpc.erl"},{line,178}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,162}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:33.80481 19:00:33.797 [error] CRASH REPORT Process couch_auth_cache with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:terminate/6 line 744
2014-07-21_17:00:33.81066 19:00:33.797 [error] rexi_server error:badarg [{ets,lookup,[couch_dbs,<<"shards/40000000-5fffffff/eumetsat.1405961126">>],[]},{couch_server,open,2,[{file,"src/couch_server.erl"},{line,78}]},{fabric_rpc,get_or_create_db,2,[{file,"src/fabric_rpc.erl"},{line,178}]},{fabric_rpc,get_missing_revs,3,[{file,"src/fabric_rpc.erl"},{line,119}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:33.81122 19:00:33.797 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at <0.6861.0> exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context child_terminated
2014-07-21_17:00:33.81171 19:00:33.797 [error] CRASH REPORT Process <0.6907.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:33.81217 19:00:33.797 [error] rexi_server error:badarg [{ets,lookup,[couch_dbs,<<"shards/40000000-5fffffff/eumetsat.1405961126">>],[]},{couch_server,open,2,[{file,"src/couch_server.erl"},{line,78}]},{fabric_rpc,get_or_create_db,2,[{file,"src/fabric_rpc.erl"},{line,178}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,162}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:33.81405 19:00:33.797 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at <0.6861.0> exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:33.81457 19:00:33.798 [error] CRASH REPORT Process <0.6912.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:33.81506 19:00:33.798 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.6861.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:33.81532 19:00:33.799 [info] needed 0.867 ms to open new _users
2014-07-21_17:00:33.81558 19:00:33.799 [info] needed 0.833 ms to open new shards/40000000-5fffffff/eumetsat.1405961126
2014-07-21_17:00:33.85910 19:00:33.858 [info] needed 0.921 ms to open new shards/e0000000-ffffffff/eumetsat.1405961126
2014-07-21_17:00:33.98149 19:00:33.981 [info] needed 1.298 ms to open new shards/20000000-3fffffff/eumetsat.1405961126
2014-07-21_17:00:34.27732 19:00:34.276 [info] needed 0.907 ms to open new shards/a0000000-bfffffff/eumetsat.1405961126
2014-07-21_17:00:34.46013 19:00:34.459 [info] needed 0.451 ms to open new shards/c0000000-dfffffff/eumetsat.1405961126
2014-07-21_17:00:34.58557 19:00:34.584 [info] needed 0.641 ms to open new shards/00000000-1fffffff/eumetsat.1405961126
2014-07-21_17:00:34.75112 19:00:34.750 [warning] mem3_sync shards/00000000-1fffffff/eumetsat.1405961126 couchdb@couch1-testing.[TLD] shutdown
2014-07-21_17:00:34.75261 19:00:34.750 [error] rexi_server exit:shutdown [{couch_db,collect_results,3,[{file,"src/couch_db.erl"},{line,930}]},{couch_db,write_and_commit,4,[{file,"src/couch_db.erl"},{line,942}]},{couch_db,update_docs,4,[{file,"src/couch_db.erl"},{line,811}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,165}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:34.75306 19:00:34.750 [warning] mem3_sync shards/00000000-1fffffff/eumetsat.1405961126 couchdb@couch2-testing.[TLD] shutdown
2014-07-21_17:00:34.75345 19:00:34.750 [error] rexi_server exit:shutdown [{couch_db,collect_results,3,[{file,"src/couch_db.erl"},{line,930}]},{couch_db,write_and_commit,4,[{file,"src/couch_db.erl"},{line,942}]},{couch_db,update_docs,4,[{file,"src/couch_db.erl"},{line,811}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,165}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:34.75404 19:00:34.750 [error] rexi_server exit:shutdown [{couch_db,collect_results,3,[{file,"src/couch_db.erl"},{line,930}]},{couch_db,write_and_commit,4,[{file,"src/couch_db.erl"},{line,942}]},{couch_db,update_docs,4,[{file,"src/couch_db.erl"},{line,881}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,165}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:34.75490 19:00:34.750 [error] rexi_server exit:shutdown [{couch_db,collect_results,3,[{file,"src/couch_db.erl"},{line,930}]},{couch_db,write_and_commit,4,[{file,"src/couch_db.erl"},{line,942}]},{couch_db,update_docs,4,[{file,"src/couch_db.erl"},{line,881}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,165}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:34.75534 19:00:34.751 [error] rexi_server exit:shutdown [{couch_db,collect_results,3,[{file,"src/couch_db.erl"},{line,930}]},{couch_db,write_and_commit,4,[{file,"src/couch_db.erl"},{line,942}]},{couch_db,update_docs,4,[{file,"src/couch_db.erl"},{line,881}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,165}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:34.75634 19:00:34.751 [error] gen_server couch_server terminated with reason: {key_exists,{1405,962034,750262}} in gb_trees:insert_1/4 line 324
2014-07-21_17:00:34.75682 19:00:34.751 [error] CRASH REPORT Process couch_server with 0 neighbours exited with reason: {key_exists,{1405,962034,750262}} in gb_trees:insert_1/4 line 324 in gen_server:terminate/6 line 744
2014-07-21_17:00:34.75716 19:00:34.751 [error] Supervisor couch_primary_services had child couch_server started with couch_server:sup_start_link() at <0.6905.0> exit with reason {key_exists,{1405,962034,750262}} in gb_trees:insert_1/4 line 324 in context child_terminated
2014-07-21_17:00:34.75774 19:00:34.752 [error] gen_server couch_auth_cache terminated with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78
2014-07-21_17:00:34.75814 19:00:34.752 [error] CRASH REPORT Process couch_auth_cache with 0 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:terminate/6 line 744
2014-07-21_17:00:34.75852 19:00:34.752 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at <0.6916.0> exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context child_terminated
2014-07-21_17:00:34.75874 19:00:34.754 [info] needed 1.192 ms to open new _users
2014-07-21_17:00:34.78614 19:00:34.785 [error] rexi_server exit:noproc [{couch_db,collect_results,3,[{file,"src/couch_db.erl"},{line,930}]},{couch_db,write_and_commit,4,[{file,"src/couch_db.erl"},{line,942}]},{couch_db,update_docs,4,[{file,"src/couch_db.erl"},{line,881}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,165}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:34.87387 19:00:34.873 [info] needed 80.562 ms to open new shards/00000000-1fffffff/eumetsat.1405961126
2014-07-21_17:00:34.94514 19:00:34.944 [info] needed 1.174 ms to open new shards/a0000000-bfffffff/eumetsat.1405961126
2014-07-21_17:00:34.94681 19:00:34.945 [error] Error in process <0.7172.0> on node 'couchdb@couch3-testing.[TLD]' with exit value: {{rexi_EXIT,{badarg,[{ets,lookup,[couch_dbs,<<44 bytes>>],[]},{couch_server,open,2,[{file,"src/couch_server.erl"},{line,78}]},{fabric_rpc,get_or_create_db,2,[{file,"src/fabric_rpc.erl"},{line,178}]},{fabric_rpc,with_db,3,[{file,"src/fab... 
2014-07-21_17:00:34.94682 
2014-07-21_17:00:34.94683 
2014-07-21_17:00:34.94720 19:00:34.945 [warning] mem3_sync shards/00000000-1fffffff/eumetsat.1405961126 couchdb@couch1-testing.[TLD] {{rexi_EXIT,{badarg,[{ets,lookup,[couch_dbs,<<115,104,97,114,100,115,47,48,48,48,48,48,48,48,48,45,49,102,102,102,102,102,102,102,47,101,117,109,101,116,115,97,116,46,49,52,48,53,57,54,49,49,50,54>>],[]},{couch_server,open,2,[{file,[115,114,99,47,99,111,117,99,104,95,115,101,114,118,101,114,46,101,114,108]},{line,78}]},{fabric_rpc,get_or_create_db,2,[{file,[115,114,99,47,102,97,98,114,105,99,95,114,112,99,46,101,114,108]},{line,178}]},{fabric_rpc,with_db,3,[{file,[115,114,99,47,102,97,98,114,105,99,95,114,112,99,46,101,114,108]},{line,162}]},{rexi_server,init_p,3,[{file,[115,114,99,47,114,101,120,105,95,115,101,114,118,101,114,46,101,114,108]},{line,151}]}]}},[{mem3_rep,rexi_call,2,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,193}]},{mem3_rep,save_on_target,3,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,171}]},{mem3_rep,replicate_batch,1,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,148}]},{mem3_rep,repl,2,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,93}]},{mem3_rep,go,1,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,69}]},{mem3_sync,'-start_push_replication/1-fun-0-',2,[{file,[115,114,99,47,109,101,109,51,95,115,121,110,99,46,101,114,108]},{line,209}]}]}
2014-07-21_17:00:35.88786 19:00:35.887 [info] needed 3.537 ms to open new shards/60000000-7fffffff/eumetsat.1405961126
2014-07-21_17:00:36.59928 19:00:36.598 [warning] mem3_sync shards/60000000-7fffffff/eumetsat.1405961126 couchdb@couch2-testing.[TLD] shutdown
2014-07-21_17:00:36.60173 19:00:36.598 [error] rexi_server exit:shutdown [{couch_db,collect_results,3,[{file,"src/couch_db.erl"},{line,930}]},{couch_db,write_and_commit,4,[{file,"src/couch_db.erl"},{line,942}]},{couch_db,update_docs,4,[{file,"src/couch_db.erl"},{line,811}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,165}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:36.60302 19:00:36.599 [error] gen_server couch_server terminated with reason: {key_exists,{1405,962036,598577}} in gb_trees:insert_1/4 line 324
2014-07-21_17:00:36.60393 19:00:36.599 [error] rexi_server error:badarg [{ets,lookup,[couch_dbs,<<"shards/00000000-1fffffff/eumetsat.1405961126">>],[]},{couch_server,open,2,[{file,"src/couch_server.erl"},{line,78}]},{fabric_rpc,get_or_create_db,2,[{file,"src/fabric_rpc.erl"},{line,178}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,162}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:36.60436 19:00:36.600 [error] CRASH REPORT Process couch_server with 0 neighbours exited with reason: {key_exists,{1405,962036,598577}} in gb_trees:insert_1/4 line 324 in gen_server:terminate/6 line 744
2014-07-21_17:00:36.60471 19:00:36.600 [error] Failed to get security object on {'couchdb@couch3-testing.[TLD]',<<"shards/00000000-1fffffff/eumetsat.1405961126">>} :: {rexi_EXIT,{badarg,[{ets,lookup,[couch_dbs,<<"shards/00000000-1fffffff/eumetsat.1405961126">>],[]},{couch_server,open,2,[{file,"src/couch_server.erl"},{line,78}]},{fabric_rpc,get_or_create_db,2,[{file,"src/fabric_rpc.erl"},{line,178}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,162}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]}}
2014-07-21_17:00:36.60513 19:00:36.600 [error] Supervisor couch_primary_services had child couch_server started with couch_server:sup_start_link() at <0.7152.0> exit with reason {key_exists,{1405,962036,598577}} in gb_trees:insert_1/4 line 324 in context child_terminated
2014-07-21_17:00:36.60543 19:00:36.600 [error] gen_server couch_auth_cache terminated with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78
2014-07-21_17:00:36.60580 19:00:36.600 [error] Error checking security objects for eumetsat :: {error,no_majority}
2014-07-21_17:00:36.60654 19:00:36.600 [error] CRASH REPORT Process couch_auth_cache with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:terminate/6 line 744
2014-07-21_17:00:36.60668 19:00:36.601 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at <0.7153.0> exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context child_terminated
2014-07-21_17:00:36.61568 19:00:36.601 [error] CRASH REPORT Process <0.7354.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:36.61704 19:00:36.601 [error] rexi_server error:badarg [{ets,lookup,[couch_dbs,<<"shards/00000000-1fffffff/eumetsat.1405961126">>],[]},{couch_server,open,2,[{file,"src/couch_server.erl"},{line,78}]},{fabric_rpc,get_or_create_db,2,[{file,"src/fabric_rpc.erl"},{line,178}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,162}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:36.61745 19:00:36.601 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at <0.7153.0> exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:36.61792 19:00:36.601 [error] rexi_server error:badarg [{ets,lookup,[couch_dbs,<<"shards/00000000-1fffffff/eumetsat.1405961126">>],[]},{couch_server,open,2,[{file,"src/couch_server.erl"},{line,78}]},{fabric_rpc,get_or_create_db,2,[{file,"src/fabric_rpc.erl"},{line,178}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,162}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:36.61852 19:00:36.603 [info] needed 0.883 ms to open new _users
2014-07-21_17:00:36.61886 19:00:36.604 [warning] mem3_sync shards/00000000-1fffffff/eumetsat.1405961126 couchdb@couch2-testing.[TLD] noproc
2014-07-21_17:00:36.61990 19:00:36.616 [warning] mem3_sync shards/00000000-1fffffff/eumetsat.1405961126 couchdb@couch1-testing.[TLD] noproc
2014-07-21_17:00:36.62000 19:00:36.616 [info] needed 0.957 ms to open new shards/00000000-1fffffff/eumetsat.1405961126
2014-07-21_17:00:36.62145 19:00:36.621 [warning] mem3_sync shards/60000000-7fffffff/eumetsat.1405961126 couchdb@couch1-testing.[TLD] noproc
2014-07-21_17:00:36.62221 19:00:36.621 [info] needed 1.054 ms to open new shards/60000000-7fffffff/eumetsat.1405961126
2014-07-21_17:00:36.73519 19:00:36.734 [error] Error in process <0.7397.0> on node 'couchdb@couch3-testing.[TLD]' with exit value: {{rexi_EXIT,{badarg,[{ets,lookup,[couch_dbs,<<44 bytes>>],[]},{couch_server,open,2,[{file,"src/couch_server.erl"},{line,78}]},{fabric_rpc,get_or_create_db,2,[{file,"src/fabric_rpc.erl"},{line,178}]},{fabric_rpc,get_missing_revs,3,[{file... 
2014-07-21_17:00:36.73522 
2014-07-21_17:00:36.73523 
2014-07-21_17:00:36.73574 19:00:36.735 [warning] mem3_sync shards/00000000-1fffffff/eumetsat.1405961126 couchdb@couch1-testing.[TLD] {{rexi_EXIT,{badarg,[{ets,lookup,[couch_dbs,<<115,104,97,114,100,115,47,48,48,48,48,48,48,48,48,45,49,102,102,102,102,102,102,102,47,101,117,109,101,116,115,97,116,46,49,52,48,53,57,54,49,49,50,54>>],[]},{couch_server,open,2,[{file,[115,114,99,47,99,111,117,99,104,95,115,101,114,118,101,114,46,101,114,108]},{line,78}]},{fabric_rpc,get_or_create_db,2,[{file,[115,114,99,47,102,97,98,114,105,99,95,114,112,99,46,101,114,108]},{line,178}]},{fabric_rpc,get_missing_revs,3,[{file,[115,114,99,47,102,97,98,114,105,99,95,114,112,99,46,101,114,108]},{line,119}]},{rexi_server,init_p,3,[{file,[115,114,99,47,114,101,120,105,95,115,101,114,118,101,114,46,101,114,108]},{line,151}]}]}},[{mem3_rep,rexi_call,2,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,193}]},{mem3_rep,replicate_batch,1,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,144}]},{mem3_rep,repl,2,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,93}]},{mem3_rep,go,1,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,69}]},{mem3_sync,'-start_push_replication/1-fun-0-',2,[{file,[115,114,99,47,109,101,109,51,95,115,121,110,99,46,101,114,108]},{line,209}]}]}
2014-07-21_17:00:36.77612 19:00:36.776 [info] needed 15.483 ms to open new shards/e0000000-ffffffff/eumetsat.1405961126
2014-07-21_17:00:36.78061 19:00:36.778 [info] needed 14.457 ms to open new shards/80000000-9fffffff/eumetsat.1405961126
2014-07-21_17:00:36.78916 19:00:36.778 [info] needed 15.473 ms to open new shards/a0000000-bfffffff/eumetsat.1405961126
2014-07-21_17:00:36.78983 19:00:36.778 [error] gen_server couch_server terminated with reason: {key_exists,{1405,962036,778073}} in gb_trees:insert_1/4 line 324
2014-07-21_17:00:36.79039 19:00:36.779 [error] rexi_server exit:{{{key_exists,{1405,962036,778073}},[{gb_trees,insert_1,4,[{file,"gb_trees.erl"},{line,324}]},{gb_trees,insert_1,4,[{file,"gb_trees.erl"},{line,303}]},{gb_trees,insert_1,4,[{file,"gb_trees.erl"},{line,303}]},{gb_trees,insert,3,[{file,"gb_trees.erl"},{line,283}]},{couch_lru,insert,2,[{file,"src/couch_lru.erl"},{line,23}]},{couch_server,handle_call,3,[{file,"src/couch_server.erl"},{line,347}]},{gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,585}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]},{gen_server,call,[couch_server,{open,<<"shards/c0000000-dfffffff/eumetsat.1405961126">>,[{user_ctx,{user_ctx,null,[<<"_admin">>],undefined}}]},infinity]}} [{gen_server,call,3,[{file,"gen_server.erl"},{line,188}]},{couch_server,open,2,[{file,"src/couch_server.erl"},{line,84}]},{fabric_rpc,get_or_create_db,2,[{file,"src/fabric_rpc.erl"},{line,178}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,162}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:36.79097 19:00:36.779 [error] CRASH REPORT Process couch_server with 0 neighbours exited with reason: {key_exists,{1405,962036,778073}} in gb_trees:insert_1/4 line 324 in gen_server:terminate/6 line 744
2014-07-21_17:00:36.79242 19:00:36.780 [error] Supervisor couch_primary_services had child couch_server started with couch_server:sup_start_link() at <0.7353.0> exit with reason {key_exists,{1405,962036,778073}} in gb_trees:insert_1/4 line 324 in context child_terminated
2014-07-21_17:00:36.79312 19:00:36.780 [error] gen_server couch_auth_cache terminated with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78
2014-07-21_17:00:36.79834 19:00:36.780 [error] CRASH REPORT Process couch_auth_cache with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:terminate/6 line 744
2014-07-21_17:00:36.79877 19:00:36.780 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at <0.7359.0> exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context child_terminated
2014-07-21_17:00:36.79914 19:00:36.780 [error] CRASH REPORT Process <0.7432.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:36.79961 19:00:36.781 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at <0.7359.0> exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:36.80001 19:00:36.781 [error] CRASH REPORT Process <0.7435.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:36.80036 19:00:36.781 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.7359.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:36.80080 19:00:36.781 [error] CRASH REPORT Process <0.7438.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:36.80117 19:00:36.782 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.7359.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:36.80152 19:00:36.782 [error] CRASH REPORT Process <0.7441.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:36.80204 19:00:36.782 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.7359.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:36.80336 19:00:36.782 [error] CRASH REPORT Process <0.7444.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:36.80427 19:00:36.782 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.7359.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:36.80457 19:00:36.783 [error] CRASH REPORT Process <0.7447.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:36.80490 19:00:36.783 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.7359.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:36.80527 19:00:36.783 [error] CRASH REPORT Process <0.7450.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:36.80557 19:00:36.783 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.7359.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:36.80588 19:00:36.784 [error] CRASH REPORT Process <0.7453.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:36.80629 19:00:36.784 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.7359.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:36.80754 19:00:36.784 [error] CRASH REPORT Process <0.7456.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:36.80867 19:00:36.784 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.7359.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:36.80929 19:00:36.784 [error] CRASH REPORT Process <0.7459.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:36.80971 19:00:36.785 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.7359.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:36.81002 19:00:36.785 [error] CRASH REPORT Process <0.7462.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:36.81030 19:00:36.785 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.7359.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:36.81069 19:00:36.785 [error] CRASH REPORT Process <0.7465.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:36.81102 19:00:36.786 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.7359.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:36.81148 19:00:36.786 [error] CRASH REPORT Process <0.7468.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:36.81177 19:00:36.786 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.7359.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:36.81222 19:00:36.786 [error] CRASH REPORT Process <0.7471.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:36.81256 19:00:36.786 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.7359.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:36.81317 19:00:36.787 [error] CRASH REPORT Process <0.7474.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:36.81353 19:00:36.787 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.7359.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:36.81404 19:00:36.787 [error] CRASH REPORT Process <0.7477.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:36.81444 19:00:36.788 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.7359.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:36.81491 19:00:36.788 [error] CRASH REPORT Process <0.7480.0> with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:init_it/6 line 328
2014-07-21_17:00:36.81519 19:00:36.788 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at {restarting,<0.7359.0>} exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context start_error
2014-07-21_17:00:36.81541 19:00:36.790 [info] needed 0.699 ms to open new _users
2014-07-21_17:00:36.81762 19:00:36.816 [info] needed 1.31 ms to open new shards/00000000-1fffffff/eumetsat.1405961126
2014-07-21_17:00:36.81807 19:00:36.816 [info] needed 1.327 ms to open new shards/a0000000-bfffffff/eumetsat.1405961126
2014-07-21_17:00:36.81848 19:00:36.817 [info] needed 1.427 ms to open new shards/60000000-7fffffff/eumetsat.1405961126
2014-07-21_17:00:36.81936 19:00:36.817 [info] needed 1.695 ms to open new shards/e0000000-ffffffff/eumetsat.1405961126
2014-07-21_17:00:36.87093 19:00:36.870 [info] needed 1.103 ms to open new shards/c0000000-dfffffff/eumetsat.1405961126
2014-07-21_17:00:36.87196 19:00:36.871 [info] needed 1.494 ms to open new shards/80000000-9fffffff/eumetsat.1405961126
2014-07-21_17:00:36.93862 19:00:36.937 [error] gen_server couch_server terminated with reason: {key_exists,{1405,962036,936453}} in gb_trees:insert_1/4 line 324
2014-07-21_17:00:36.93963 19:00:36.937 [error] CRASH REPORT Process couch_server with 0 neighbours exited with reason: {key_exists,{1405,962036,936453}} in gb_trees:insert_1/4 line 324 in gen_server:terminate/6 line 744
2014-07-21_17:00:36.94066 19:00:36.937 [error] Supervisor couch_primary_services had child couch_server started with couch_server:sup_start_link() at <0.7431.0> exit with reason {key_exists,{1405,962036,936453}} in gb_trees:insert_1/4 line 324 in context child_terminated
2014-07-21_17:00:36.94178 19:00:36.937 [error] gen_server couch_auth_cache terminated with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78
2014-07-21_17:00:36.94263 19:00:36.938 [error] CRASH REPORT Process couch_auth_cache with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:terminate/6 line 744
2014-07-21_17:00:36.94921 19:00:36.938 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at <0.7483.0> exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context child_terminated
2014-07-21_17:00:36.94948 19:00:36.939 [info] needed 1.329 ms to open new shards/a0000000-bfffffff/eumetsat.1405961126
2014-07-21_17:00:36.94977 19:00:36.939 [info] needed 1.563 ms to open new shards/80000000-9fffffff/eumetsat.1405961126
2014-07-21_17:00:36.95008 19:00:36.940 [info] needed 1.058 ms to open new _users
2014-07-21_17:00:36.98977 19:00:36.989 [info] needed 0.823 ms to open new shards/c0000000-dfffffff/eumetsat.1405961126
2014-07-21_17:00:37.03277 19:00:37.032 [info] needed 0.359 ms to open new shards/e0000000-ffffffff/eumetsat.1405961126
2014-07-21_17:00:37.05646 19:00:37.056 [info] needed 18.501 ms to open new shards/20000000-3fffffff/eumetsat.1405961126
2014-07-21_17:00:37.05675 19:00:37.056 [info] needed 25.805 ms to open new shards/60000000-7fffffff/eumetsat.1405961126
2014-07-21_17:00:37.05714 19:00:37.056 [info] needed 22.572 ms to open new shards/40000000-5fffffff/eumetsat.1405961126
2014-07-21_17:00:37.05746 19:00:37.056 [info] needed 18.138 ms to open new shards/00000000-1fffffff/eumetsat.1405961126
2014-07-21_17:00:37.08014 19:00:37.078 [error] Failed to get security object on {'couchdb@couch1-testing.[TLD]',<<"shards/c0000000-dfffffff/eumetsat.1405961126">>} :: {rexi_EXIT,{{{{key_exists,{1405,962037,234734}},[{gb_trees,insert_1,4,[{file,"gb_trees.erl"},{line,324}]},{gb_trees,insert,3,[{file,"gb_trees.erl"},{line,283}]},{couch_lru,update,2,[{file,"src/couch_lru.erl"},{line,29}]},{couch_server,handle_cast,2,[{file,"src/couch_server.erl"},{line,474}]},{gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,604}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]},{gen_server,call,[couch_server,{open,<<"shards/c0000000-dfffffff/eumetsat.1405961126">>,[{user_ctx,{user_ctx,null,[<<"_admin">>],undefined}}]},infinity]}},[{gen_server,call,3,[{file,"gen_server.erl"},{line,188}]},{couch_server,open,2,[{file,"src/couch_server.erl"},{line,84}]},{fabric_rpc,get_or_create_db,2,[{file,"src/fabric_rpc.erl"},{line,178}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,162}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]}}
2014-07-21_17:00:37.08093 19:00:37.078 [error] Error checking security objects for eumetsat :: {error,no_majority}
2014-07-21_17:00:37.08135 19:00:37.079 [error] Error in process <0.7574.0> on node 'couchdb@couch3-testing.[TLD]' with exit value: {{rexi_EXIT,{{{{key_exists,{1405,962037,234734}},[{gb_trees,insert_1,4,[{file,"gb_trees.erl"},{line,324}]},{gb_trees,insert,3,[{file,"gb_trees.erl"},{line,283}]},{couch_lru,update,2,[{file,"src/couch_lru.erl"},{line,29}]},{couch_server,handle_cast... 
2014-07-21_17:00:37.08136 
2014-07-21_17:00:37.08136 
2014-07-21_17:00:37.08164 19:00:37.080 [warning] mem3_sync shards/c0000000-dfffffff/eumetsat.1405961126 couchdb@couch1-testing.[TLD] {{rexi_EXIT,{{{{key_exists,{1405,962037,234734}},[{gb_trees,insert_1,4,[{file,[103,98,95,116,114,101,101,115,46,101,114,108]},{line,324}]},{gb_trees,insert,3,[{file,[103,98,95,116,114,101,101,115,46,101,114,108]},{line,283}]},{couch_lru,update,2,[{file,[115,114,99,47,99,111,117,99,104,95,108,114,117,46,101,114,108]},{line,29}]},{couch_server,handle_cast,2,[{file,[115,114,99,47,99,111,117,99,104,95,115,101,114,118,101,114,46,101,114,108]},{line,474}]},{gen_server,handle_msg,5,[{file,[103,101,110,95,115,101,114,118,101,114,46,101,114,108]},{line,604}]},{proc_lib,init_p_do_apply,3,[{file,[112,114,111,99,95,108,105,98,46,101,114,108]},{line,239}]}]},{gen_server,call,[couch_server,{open,<<115,104,97,114,100,115,47,99,48,48,48,48,48,48,48,45,100,102,102,102,102,102,102,102,47,101,117,109,101,116,115,97,116,46,49,52,48,53,57,54,49,49,50,54>>,[{user_ctx,{user_ctx,null,[<<95,97,100,109,105,110>>],undefined}},{io_priority,{internal_repl,<<115,104,97,114,100,115,47,99,48,48,48,48,48,48,48,45,100,102,102,102,102,102,102,102,47,101,117,109,101,116,115,97,116,46,49,52,48,53,57,54,49,49,50,54>>}}]},infinity]}},[{gen_server,call,3,[{file,[103,101,110,95,115,101,114,118,101,114,46,101,114,108]},{line,188}]},{couch_server,open,2,[{file,[115,114,99,47,99,111,117,99,104,95,115,101,114,118,101,114,46,101,114,108]},{line,84}]},{fabric_rpc,get_or_create_db,2,[{file,[115,114,99,47,102,97,98,114,105,99,95,114,112,99,46,101,114,108]},{line,178}]},{fabric_rpc,with_db,3,[{file,[115,114,99,47,102,97,98,114,105,99,95,114,112,99,46,101,114,108]},{line,162}]},{rexi_server,init_p,3,[{file,[115,114,99,47,114,101,120,105,95,115,101,114,118,101,114,46,101,114,108]},{line,151}]}]}},[{mem3_rep,rexi_call,2,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,193}]},{mem3_rep,calculate_start_seq,3,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,207}]},{mem3_rep,repl,2,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,89}]},{mem3_rep,go,1,[{file,[115,114,99,47,109,101,109,51,95,114,101,112,46,101,114,108]},{line,69}]},{mem3_sync,'-start_push_replication/1-fun-0-',2,[{file,[115,114,99,47,109,101,109,51,95,115,121,110,99,46,101,114,108]},{line,209}]}]}
2014-07-21_17:00:37.09161 19:00:37.090 [error] gen_server couch_server terminated with reason: {key_exists,{1405,962037,89421}} in gb_trees:insert_1/4 line 324
2014-07-21_17:00:37.09217 19:00:37.090 [error] CRASH REPORT Process couch_server with 0 neighbours exited with reason: {key_exists,{1405,962037,89421}} in gb_trees:insert_1/4 line 324 in gen_server:terminate/6 line 744
2014-07-21_17:00:37.09285 19:00:37.090 [error] Supervisor couch_primary_services had child couch_server started with couch_server:sup_start_link() at <0.7532.0> exit with reason {key_exists,{1405,962037,89421}} in gb_trees:insert_1/4 line 324 in context child_terminated
2014-07-21_17:00:37.09331 19:00:37.090 [error] gen_server couch_auth_cache terminated with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78
2014-07-21_17:00:37.09368 19:00:37.090 [error] CRASH REPORT Process couch_auth_cache with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:terminate/6 line 744
2014-07-21_17:00:37.09383 19:00:37.090 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at <0.7536.0> exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context child_terminated
2014-07-21_17:00:37.09453 19:00:37.090 [error] rexi_server error:badarg [{ets,lookup,[couch_dbs,<<"shards/80000000-9fffffff/eumetsat.1405961126">>],[]},{couch_server,open,2,[{file,"src/couch_server.erl"},{line,78}]},{fabric_rpc,get_or_create_db,2,[{file,"src/fabric_rpc.erl"},{line,178}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,162}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:37.09513 19:00:37.091 [error] rexi_server error:badarg [{ets,lookup,[couch_dbs,<<"shards/60000000-7fffffff/eumetsat.1405961126">>],[]},{couch_server,open,2,[{file,"src/couch_server.erl"},{line,78}]},{fabric_rpc,get_or_create_db,2,[{file,"src/fabric_rpc.erl"},{line,178}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,162}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:37.09533 19:00:37.092 [info] needed 0.87 ms to open new _users
2014-07-21_17:00:37.09548 19:00:37.094 [info] needed 1.058 ms to open new shards/40000000-5fffffff/eumetsat.1405961126
2014-07-21_17:00:37.21936 19:00:37.218 [info] needed 1.199 ms to open new shards/a0000000-bfffffff/eumetsat.1405961126
2014-07-21_17:00:37.22151 19:00:37.221 [info] needed 1.622 ms to open new shards/c0000000-dfffffff/eumetsat.1405961126
2014-07-21_17:00:37.23482 19:00:37.234 [info] needed 0.822 ms to open new shards/e0000000-ffffffff/eumetsat.1405961126
2014-07-21_17:00:37.23766 19:00:37.236 [info] needed 1.087 ms to open new shards/20000000-3fffffff/eumetsat.1405961126
2014-07-21_17:00:37.23867 19:00:37.238 [info] needed 0.624 ms to open new shards/00000000-1fffffff/eumetsat.1405961126
2014-07-21_17:00:37.24174 19:00:37.241 [error] rexi_server exit:shutdown [{couch_db,collect_results,3,[{file,"src/couch_db.erl"},{line,930}]},{couch_db,write_and_commit,4,[{file,"src/couch_db.erl"},{line,942}]},{couch_db,update_docs,4,[{file,"src/couch_db.erl"},{line,881}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,165}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:37.24262 19:00:37.241 [error] rexi_server exit:shutdown [{couch_db,collect_results,3,[{file,"src/couch_db.erl"},{line,930}]},{couch_db,write_and_commit,4,[{file,"src/couch_db.erl"},{line,942}]},{couch_db,update_docs,4,[{file,"src/couch_db.erl"},{line,881}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,165}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:37.24372 19:00:37.241 [error] rexi_server exit:shutdown [{couch_db,collect_results,3,[{file,"src/couch_db.erl"},{line,930}]},{couch_db,write_and_commit,4,[{file,"src/couch_db.erl"},{line,942}]},{couch_db,update_docs,4,[{file,"src/couch_db.erl"},{line,881}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,165}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:37.24427 19:00:37.242 [error] gen_server couch_server terminated with reason: {key_exists,{1405,962037,240775}} in gb_trees:insert_1/4 line 324
2014-07-21_17:00:37.34776 19:00:37.242 [error] CRASH REPORT Process couch_server with 0 neighbours exited with reason: {key_exists,{1405,962037,240775}} in gb_trees:insert_1/4 line 324 in gen_server:terminate/6 line 744
2014-07-21_17:00:37.34987 19:00:37.242 [error] Supervisor couch_primary_services had child couch_server started with couch_server:sup_start_link() at <0.7604.0> exit with reason {key_exists,{1405,962037,240775}} in gb_trees:insert_1/4 line 324 in context child_terminated
2014-07-21_17:00:37.35027 19:00:37.242 [error] gen_server couch_auth_cache terminated with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78
2014-07-21_17:00:37.35077 19:00:37.242 [error] CRASH REPORT Process couch_auth_cache with 0 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:terminate/6 line 744
2014-07-21_17:00:37.35110 19:00:37.242 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at <0.7605.0> exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context child_terminated
2014-07-21_17:00:37.35165 19:00:37.243 [error] rexi_server error:badarg [{ets,lookup,[couch_dbs,<<"shards/00000000-1fffffff/eumetsat.1405961126">>],[]},{couch_server,open,2,[{file,"src/couch_server.erl"},{line,78}]},{fabric_rpc,get_or_create_db,2,[{file,"src/fabric_rpc.erl"},{line,178}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,162}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:37.35209 19:00:37.244 [info] needed 0.652 ms to open new _users
2014-07-21_17:00:37.35246 19:00:37.350 [info] needed 1.198 ms to open new shards/a0000000-bfffffff/eumetsat.1405961126
2014-07-21_17:00:37.47140 19:00:37.470 [error] rexi_server exit:noproc [{couch_db,collect_results,3,[{file,"src/couch_db.erl"},{line,930}]},{couch_db,write_and_commit,4,[{file,"src/couch_db.erl"},{line,942}]},{couch_db,update_docs,4,[{file,"src/couch_db.erl"},{line,881}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,165}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:37.55282 19:00:37.552 [info] needed 13.774 ms to open new shards/c0000000-dfffffff/eumetsat.1405961126
2014-07-21_17:00:37.70760 19:00:37.707 [info] needed 1.03 ms to open new shards/60000000-7fffffff/eumetsat.1405961126
2014-07-21_17:00:38.66448 19:00:38.663 [info] needed 2.055 ms to open new shards/80000000-9fffffff/eumetsat.1405961126
2014-07-21_17:00:38.66569 19:00:38.665 [info] needed 3.712 ms to open new shards/e0000000-ffffffff/eumetsat.1405961126
2014-07-21_17:00:38.77999 19:00:38.779 [info] needed 3.033 ms to open new dbs
2014-07-21_17:00:38.81550 19:00:38.815 [info] needed 2.618 ms to open new shards/40000000-5fffffff/eumetsat.1405961126
2014-07-21_17:00:39.01799 19:00:39.017 [info] needed 1.125 ms to open new shards/20000000-3fffffff/eumetsat.1405961126
2014-07-21_17:00:39.75342 19:00:39.752 [info] needed 0.453 ms to open new shards/00000000-1fffffff/eumetsat.1405961126
2014-07-21_17:00:39.98382 19:00:39.982 [error] rexi_server exit:shutdown [{couch_db,collect_results,3,[{file,"src/couch_db.erl"},{line,930}]},{couch_db,write_and_commit,4,[{file,"src/couch_db.erl"},{line,942}]},{couch_db,update_docs,4,[{file,"src/couch_db.erl"},{line,881}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,165}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:39.98413 19:00:39.982 [error] rexi_server exit:shutdown [{couch_db,collect_results,3,[{file,"src/couch_db.erl"},{line,930}]},{couch_db,write_and_commit,4,[{file,"src/couch_db.erl"},{line,942}]},{couch_db,update_docs,4,[{file,"src/couch_db.erl"},{line,881}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,165}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:39.98585 19:00:39.982 [error] rexi_server exit:shutdown [{couch_db,collect_results,3,[{file,"src/couch_db.erl"},{line,930}]},{couch_db,write_and_commit,4,[{file,"src/couch_db.erl"},{line,942}]},{couch_db,update_docs,4,[{file,"src/couch_db.erl"},{line,881}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,165}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:39.98946 19:00:39.983 [error] gen_server couch_server terminated with reason: {key_exists,{1405,962039,982107}} in gb_trees:insert_1/4 line 324
2014-07-21_17:00:39.99129 19:00:39.983 [error] CRASH REPORT Process couch_server with 0 neighbours exited with reason: {key_exists,{1405,962039,982107}} in gb_trees:insert_1/4 line 324 in gen_server:terminate/6 line 744
2014-07-21_17:00:39.99264 19:00:39.983 [error] Supervisor couch_primary_services had child couch_server started with couch_server:sup_start_link() at <0.7648.0> exit with reason {key_exists,{1405,962039,982107}} in gb_trees:insert_1/4 line 324 in context child_terminated
2014-07-21_17:00:39.99402 19:00:39.983 [error] Supervisor couch_primary_services had child couch_server started with couch_server:sup_start_link() at <0.7648.0> exit with reason reached_max_restart_intensity in context shutdown
2014-07-21_17:00:39.99548 19:00:39.984 [error] gen_server couch_auth_cache terminated with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78
2014-07-21_17:00:39.99620 19:00:39.984 [error] rexi_server error:badarg [{ets,lookup,[couch_dbs,<<"shards/00000000-1fffffff/eumetsat.1405961126">>],[]},{couch_server,open,2,[{file,"src/couch_server.erl"},{line,78}]},{fabric_rpc,get_or_create_db,2,[{file,"src/fabric_rpc.erl"},{line,178}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,162}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:39.99718 19:00:39.984 [error] CRASH REPORT Process couch_auth_cache with 1 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in gen_server:terminate/6 line 744
2014-07-21_17:00:39.99781 19:00:39.984 [error] Replication manager received unexpected message {'EXIT',<0.228.0>,killed}
2014-07-21_17:00:39.99834 19:00:39.984 [notice] Stopping all ongoing replications
2014-07-21_17:00:39.99899 19:00:39.985 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at <0.7649.0> exit with reason bad argument in call to ets:lookup(couch_dbs, <<"_users">>) in couch_server:open/2 line 78 in context child_terminated
2014-07-21_17:00:39.99955 19:00:39.985 [error] ddoc_cache_opener evictor died killed
2014-07-21_17:00:39.99977 19:00:39.985 [notice] mem3_nodes changes listener died killed
2014-07-21_17:00:40.00025 19:00:39.985 [error] Supervisor couch_secondary_services had child auth_cache started with couch_auth_cache:start_link() at <0.7649.0> exit with reason reached_max_restart_intensity in context shutdown
2014-07-21_17:00:40.00064 19:00:39.985 [error] gen_server couch_replicator_manager terminated with reason: {unexpected_msg,{'EXIT',<0.228.0>,killed}}
2014-07-21_17:00:40.00107 19:00:39.985 [notice] mem3_shards changes listener died killed
2014-07-21_17:00:40.00148 19:00:39.985 [error] CRASH REPORT Process couch_replicator_manager with 0 neighbours exited with reason: {unexpected_msg,{'EXIT',<0.228.0>,killed}} in gen_server:terminate/6 line 744
2014-07-21_17:00:40.00209 19:00:39.985 [error] rexi_server error:badarg [{ets,lookup,[couch_dbs,<<"shards/00000000-1fffffff/eumetsat.1405961126">>],[]},{couch_server,open,2,[{file,"src/couch_server.erl"},{line,78}]},{fabric_rpc,get_or_create_db,2,[{file,"src/fabric_rpc.erl"},{line,178}]},{fabric_rpc,with_db,3,[{file,"src/fabric_rpc.erl"},{line,162}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,151}]}]
2014-07-21_17:00:40.00250 19:00:39.986 [error] Supervisor couch_secondary_services had child replicator_manager started with couch_replicator_manager:start_link() at <0.183.0> exit with reason {unexpected_msg,{'EXIT',<0.228.0>,killed}} in context shutdown_error
2014-07-21_17:00:40.00286 19:00:39.986 [error] Supervisor couch_sup had child couch_primary_services started with couch_primary_sup:start_link() at <0.121.0> exit with reason shutdown in context child_terminated
2014-07-21_17:00:40.00332 19:00:39.986 [error] gen_server mem3_shards terminated with reason: bad argument in call to ets:lookup(couch_dbs, <<"dbs">>) in couch_server:open/2 line 78
2014-07-21_17:00:40.00353 19:00:39.986 [notice] mem3_shards changes listener died {badarg,[{ets,lookup,[couch_dbs,<<"dbs">>],[]},{couch_server,open,2,[{file,"src/couch_server.erl"},{line,78}]},{couch_db,open,2,[{file,"src/couch_db.erl"},{line,81}]},{mem3_util,ensure_exists,1,[{file,"src/mem3_util.erl"},{line,173}]},{mem3_shards,get_update_seq,0,[{file,"src/mem3_shards.erl"},{line,192}]},{mem3_shards,'-init/1-fun-0-',0,[{file,"src/mem3_shards.erl"},{line,118}]}]}
2014-07-21_17:00:40.00384 19:00:39.986 [error] CRASH REPORT Process mem3_shards with 0 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"dbs">>) in couch_server:open/2 line 78 in gen_server:terminate/6 line 744
2014-07-21_17:00:40.00417 19:00:39.988 [error] Supervisor mem3_sup had child mem3_shards started with mem3_shards:start_link() at <0.245.0> exit with reason bad argument in call to ets:lookup(couch_dbs, <<"dbs">>) in couch_server:open/2 line 78 in context child_terminated
2014-07-21_17:00:40.00453 19:00:39.988 [error] CRASH REPORT Process <0.7888.0> with 0 neighbours exited with reason: no such process or port in call to gen_server:init_it/6 line 328
2014-07-21_17:00:40.00490 19:00:39.988 [error] gen_server ddoc_cache_opener terminated with reason: no match of right hand value {error,noproc} in ddoc_cache_opener:handle_info/2 line 130
2014-07-21_17:00:40.00520 19:00:39.988 [error] CRASH REPORT Process ddoc_cache_opener with 0 neighbours exited with reason: no match of right hand value {error,noproc} in ddoc_cache_opener:handle_info/2 line 130 in gen_server:terminate/6 line 744
2014-07-21_17:00:40.00550 19:00:39.988 [error] Supervisor ddoc_cache_sup had child ddoc_cache_opener started with ddoc_cache_opener:start_link() at <0.303.0> exit with reason no match of right hand value {error,noproc} in ddoc_cache_opener:handle_info/2 line 130 in context child_terminated
2014-07-21_17:00:40.00581 19:00:39.989 [error] CRASH REPORT Process <0.7886.0> with 0 neighbours exited with reason: no such process or port in call to gen_server:init_it/6 line 328
2014-07-21_17:00:40.00610 19:00:39.989 [error] gen_server mem3_sync terminated with reason: no match of right hand value {error,noproc} in mem3_sync:handle_info/2 line 120
2014-07-21_17:00:40.00645 19:00:39.989 [error] CRASH REPORT Process mem3_sync with 0 neighbours exited with reason: no match of right hand value {error,noproc} in mem3_sync:handle_info/2 line 120 in gen_server:terminate/6 line 744
2014-07-21_17:00:40.00678 19:00:39.989 [error] CRASH REPORT Process <0.7892.0> with 0 neighbours exited with reason: no such process or port in call to gen_server:init_it/6 line 328
2014-07-21_17:00:40.00741 19:00:39.990 [notice] mem3_shards changes listener died {badarg,[{ets,lookup,[couch_dbs,<<"dbs">>],[]},{couch_server,open,2,[{file,"src/couch_server.erl"},{line,78}]},{couch_db,open,2,[{file,"src/couch_db.erl"},{line,81}]},{mem3_util,ensure_exists,1,[{file,"src/mem3_util.erl"},{line,173}]},{mem3_shards,get_update_seq,0,[{file,"src/mem3_shards.erl"},{line,192}]},{mem3_shards,'-init/1-fun-0-',0,[{file,"src/mem3_shards.erl"},{line,118}]}]}
2014-07-21_17:00:40.00782 19:00:39.992 [info] needed 1.708 ms to open new dbs
2014-07-21_17:00:40.00905 19:00:40.006 [error] CRASH REPORT Process <0.7891.0> with 0 neighbours exited with reason: no match of right hand value {error,noproc} in ddoc_cache_opener:init/1 line 66 in gen_server:init_it/6 line 328
2014-07-21_17:00:40.01012 19:00:40.007 [error] Supervisor ddoc_cache_sup had child ddoc_cache_opener started with ddoc_cache_opener:start_link() at <0.303.0> exit with reason no match of right hand value {error,noproc} in ddoc_cache_opener:init/1 line 66 in context start_error
2014-07-21_17:00:40.01143 19:00:40.007 [error] Error in process <0.7895.0> on node 'couchdb@couch3-testing.[TLD]' with exit value: {badarg,[{ets,lookup,[couch_dbs,<<3 bytes>>],[]},{couch_server,open,2,[{file,"src/couch_server.erl"},{line,78}]},{couch_db,open,2,[{file,"src/couch_db.erl"},{line,81}]},{mem3_util,ensure_exists,1,[{file,"src/mem3_util.erl"},{line,173}]},{mem3_shards... 
2014-07-21_17:00:40.01144 
2014-07-21_17:00:40.01144 
2014-07-21_17:00:40.01242 19:00:40.007 [error] Supervisor mem3_sup had child mem3_sync started with mem3_sync:start_link() at <0.214.0> exit with reason no match of right hand value {error,noproc} in mem3_sync:handle_info/2 line 120 in context child_terminated
2014-07-21_17:00:40.01339 19:00:40.007 [error] gen_server mem3_shards terminated with reason: bad argument in call to ets:lookup(couch_dbs, <<"dbs">>) in couch_server:open/2 line 78
2014-07-21_17:00:40.01466 19:00:40.007 [error] CRASH REPORT Process mem3_shards with 0 neighbours exited with reason: bad argument in call to ets:lookup(couch_dbs, <<"dbs">>) in couch_server:open/2 line 78 in gen_server:terminate/6 line 744
2014-07-21_17:00:40.01523 19:00:40.007 [error] CRASH REPORT Process <0.7897.0> with 0 neighbours exited with reason: no such process or port in call to gen_server:init_it/6 line 328
2014-07-21_17:00:40.01574 19:00:40.007 [error] CRASH REPORT Process <0.7894.0> with 0 neighbours exited with reason: no match of right hand value {error,noproc} in ddoc_cache_opener:init/1 line 66 in gen_server:init_it/6 line 328
2014-07-21_17:00:40.01657 19:00:40.008 [error] Supervisor ddoc_cache_sup had child ddoc_cache_opener started with ddoc_cache_opener:start_link() at {restarting,<0.303.0>} exit with reason no match of right hand value {error,noproc} in ddoc_cache_opener:init/1 line 66 in context start_error
2014-07-21_17:00:40.01757 19:00:40.008 [error] CRASH REPORT Process <0.7900.0> with 0 neighbours exited with reason: no such process or port in call to gen_server:init_it/6 line 328
2014-07-21_17:00:40.01813 19:00:40.008 [info] needed 16.732 ms to open new _users
2014-07-21_17:00:40.01894 19:00:40.008 [error] CRASH REPORT Process <0.7899.0> with 0 neighbours exited with reason: no match of right hand value {error,noproc} in ddoc_cache_opener:init/1 line 66 in gen_server:init_it/6 line 328
2014-07-21_17:00:40.01942 19:00:40.008 [info] needed 16.877 ms to open new nodes
2014-07-21_17:00:40.02044 19:00:40.008 [error] Supervisor ddoc_cache_sup had child ddoc_cache_opener started with ddoc_cache_opener:start_link() at {restarting,<0.303.0>} exit with reason no match of right hand value {error,noproc} in ddoc_cache_opener:init/1 line 66 in context start_error
2014-07-21_17:00:40.02115 19:00:40.008 [error] CRASH REPORT Process <0.7903.0> with 0 neighbours exited with reason: no such process or port in call to gen_server:init_it/6 line 328
2014-07-21_17:00:40.02165 19:00:40.008 [error] CRASH REPORT Process <0.7902.0> with 0 neighbours exited with reason: no match of right hand value {error,noproc} in ddoc_cache_opener:init/1 line 66 in gen_server:init_it/6 line 328
2014-07-21_17:00:40.02948 19:00:40.008 [error] Supervisor ddoc_cache_sup had child ddoc_cache_opener started with ddoc_cache_opener:start_link() at {restarting,<0.303.0>} exit with reason no match of right hand value {error,noproc} in ddoc_cache_opener:init/1 line 66 in context start_error
2014-07-21_17:00:40.02997 19:00:40.008 [error] CRASH REPORT Process <0.7906.0> with 0 neighbours exited with reason: no such process or port in call to gen_server:init_it/6 line 328
2014-07-21_17:00:40.03045 19:00:40.008 [error] CRASH REPORT Process <0.7905.0> with 0 neighbours exited with reason: no match of right hand value {error,noproc} in ddoc_cache_opener:init/1 line 66 in gen_server:init_it/6 line 328
2014-07-21_17:00:40.03086 19:00:40.009 [error] Supervisor ddoc_cache_sup had child ddoc_cache_opener started with ddoc_cache_opener:start_link() at {restarting,<0.303.0>} exit with reason no match of right hand value {error,noproc} in ddoc_cache_opener:init/1 line 66 in context start_error
2014-07-21_17:00:40.03130 19:00:40.009 [error] Supervisor ddoc_cache_sup had child ddoc_cache_opener started with ddoc_cache_opener:start_link() at {restarting,<0.303.0>} exit with reason reached_max_restart_intensity in context shutdown
2014-07-21_17:00:40.03174 19:00:40.009 [error] CRASH REPORT Process <0.7909.0> with 0 neighbours exited with reason: no such process or port in call to gen_server:init_it/6 line 328
2014-07-21_17:00:40.03212 19:00:40.009 [error] CRASH REPORT Process <0.7896.0> with 0 neighbours exited with reason: no match of right hand value {error,noproc} in mem3_sync:init/1 line 68 in gen_server:init_it/6 line 328
2014-07-21_17:00:40.03261 19:00:40.009 [error] Supervisor mem3_sup had child mem3_sync started with mem3_sync:start_link() at <0.214.0> exit with reason no match of right hand value {error,noproc} in mem3_sync:init/1 line 68 in context start_error
2014-07-21_17:00:40.03306 19:00:40.009 [error] Supervisor mem3_sup had child mem3_shards started with mem3_shards:start_link() at <0.7889.0> exit with reason bad argument in call to ets:lookup(couch_dbs, <<"dbs">>) in couch_server:open/2 line 78 in context child_terminated
2014-07-21_17:00:40.03916 19:00:40.009 [error] Error in process <0.7913.0> on node 'couchdb@couch3-testing.[TLD]' with exit value: {badarg,[{ets,lookup,[couch_dbs,<<3 bytes>>],[]},{couch_server,open,2,[{file,"src/couch_server.erl"},{line,78}]},{couch_db,open,2,[{file,"src/couch_db.erl"},{line,81}]},{mem3_util,ensure_exists,1,[{file,"src/mem3_util.erl"},{line,173}]},{mem3_shards... 
2014-07-21_17:00:40.03916 
2014-07-21_17:00:40.03916 
2014-07-21_17:00:40.03971 19:00:40.009 [error] CRASH REPORT Process <0.7915.0> with 0 neighbours exited with reason: no such process or port in call to gen_server:init_it/6 line 328
2014-07-21_17:00:40.04015 19:00:40.009 [error] CRASH REPORT Process <0.7914.0> with 0 neighbours exited with reason: no match of right hand value {error,noproc} in mem3_sync:init/1 line 68 in gen_server:init_it/6 line 328
2014-07-21_17:00:40.04058 19:00:40.010 [error] Supervisor mem3_sup had child mem3_sync started with mem3_sync:start_link() at {restarting,<0.214.0>} exit with reason no match of right hand value {error,noproc} in mem3_sync:init/1 line 68 in context start_error
2014-07-21_17:00:40.04084 19:00:40.010 [info] Application ddoc_cache exited with reason: shutdown
2014-07-21_17:00:40.04127 19:00:40.010 [error] Supervisor couch_sup had child couch_secondary_services started with couch_secondary_sup:start_link() at <0.128.0> exit with reason shutdown in context child_terminated