From 0f1063963ea5be59b08e45db58a1b2d82258bda0 Mon Sep 17 00:00:00 2001 From: Jeremy Stanley Date: Thu, 21 Jul 2022 20:26:23 +0000 Subject: [PATCH] Increase PyPI substitute line length limit to 20m The earlier problems identified with using mod_substitute have been narrowed down to the new PEP 691 JSON simple API responses from Warehouse, which are returned as a single line of data. The currently largest known project index response we've been diagnosing this problem with is only 1524169 characters in length, but there are undoubtedly others and they will only continue to grow with time. The main index is also already over the new 5m limit we set (nearly double it), and while we don't currently process it with mod_substitute, we shouldn't make it harder to do so if we need to later. Change-Id: Ib32acd48e5166780841695784c55793d014b3580 --- playbooks/roles/mirror/templates/mirror.vhost.j2 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/playbooks/roles/mirror/templates/mirror.vhost.j2 b/playbooks/roles/mirror/templates/mirror.vhost.j2 index fe6e286df7..9a9a58df04 100644 --- a/playbooks/roles/mirror/templates/mirror.vhost.j2 +++ b/playbooks/roles/mirror/templates/mirror.vhost.j2 @@ -100,7 +100,7 @@ ErrorLogFormat "[%{cu}t] [%-m:%l] [pid %P:tid %T] %7F: %E: [client\ %a] %M% , \ # Rewrite the locations of the actual files SetOutputFilter INFLATE;SUBSTITUTE;DEFLATE - SubstituteMaxLineLength 5m + SubstituteMaxLineLength 20m Substitute "s|https://files.pythonhosted.org/|/pypifiles/|ni" @@ -252,7 +252,7 @@ ErrorLogFormat "[%{cu}t] [%-m:%l] [pid %P:tid %T] %7F: %E: [client\ %a] %M% , \ # Rewrite the locations of the actual files SetOutputFilter INFLATE;SUBSTITUTE;DEFLATE - SubstituteMaxLineLength 5m + SubstituteMaxLineLength 20m Substitute "s|https://files.pythonhosted.org/|/pypifiles/|ni"