Increase the HTTP incoming connection queue

Upstream Gerrit confirms that the default value for the httpd.maxQueued
value was set too low.  The fix[1] for it was to change the default to
200 however the fix is only in version 2.12+ not in 2.11.

clarkb confirmed that we are seeing the same error (stack trace linked
in 1st ml thread in upstream fix[1]) in review.o.o logs.  This likey
indicates that VM starvation (due to GC) is making Gerrit to slow to
process incoming client requests so it pushes more requests into the
queue than it can handle.  Once the queue is exhausted the requests
get rejected with 500 errors.  Increasing the max number of client
connections may help alleviate this situation until Gerrit can catch
up.

[1] https://gerrit-review.googlesource.com/#/c/70627

depends-on: Iea202fcfe7af4a9d5b4c3a360124c24649b1133b
Change-Id: I8d6c8d9ec6348fc6b950744237e527133a625bbe
This commit is contained in:
Khai Do 2016-02-03 16:18:02 -08:00
parent 7394f928a5
commit 38dc89814f
2 changed files with 7 additions and 0 deletions

View File

@ -37,6 +37,7 @@ class openstack_project::gerrit (
$httpd_acceptorthreads = '',
$httpd_minthreads = '',
$httpd_maxthreads = '',
$httpd_maxqueued = '',
$httpd_maxwait = '',
$war = '',
$contactstore = false,
@ -135,6 +136,7 @@ class openstack_project::gerrit (
httpd_acceptorthreads => $httpd_acceptorthreads,
httpd_minthreads => $httpd_minthreads,
httpd_maxthreads => $httpd_maxthreads,
httpd_maxqueued => $httpd_maxqueued,
httpd_maxwait => $httpd_maxwait,
commentlinks => [
{

View File

@ -21,6 +21,10 @@
# http:
# //groups.google.com/group/repo-discuss/browse_thread/thread/b91491c185295a71
# httpd.maxQueued:
# Default value is too low, should increase to new default.
# https://gerrit-review.googlesource.com/#/c/70627
# httpd.maxWait:
# 12:07 <@spearce> httpd.maxwait defaults to 5 minutes and is how long gerrit
# waits for an idle sshd.thread before aboring the http request
@ -113,6 +117,7 @@ class openstack_project::review (
core_packedgitwindowsize => '16k',
sshd_threads => '100',
index_threads => 4,
httpd_maxqueued => '200',
httpd_maxwait => '5000min',
war =>
'http://tarballs.openstack.org/ci/gerrit/gerrit-v2.11.4.11.a14450f.war',