handle exception correctly in _make_app_iter_reader

_make_app_iter_reader uses a queue to pass read data to
_make_app_iter, and uses an empty string as a terminator.  The problem
is that it pushes the same terminator even if it fails to reads the
data.  If an exception happens before pushing all data to the queue,
_make_app_iter exits without yielding all data.  In that case,
proxy-server stops in the middle of sending data, and a client waits
for all data to come infinitely.

This patch uses a boolean value for the terminator instead of an empty
string to distinguish the results, and raises an exception in the
error cases.  Fixes bug 890888.

Change-Id: Ie4d48025843aa1bb600446da6f4fc49f82a440bf

swift/proxy/server.py

@@ -673,6 +673,7 @@ class Controller(object):
                                      logging information.
         """
         self.app.logger.thread_locals = logger_thread_locals
+        success = True
         try:
             try:
                 while True:
@@ -686,13 +687,15 @@ class Controller(object):
                     _('Client did not read from queue within %ss') %
                     self.app.client_timeout)
                 self.app.logger.increment('client_timeouts')
+                success = False
             except (Exception, Timeout):
                 self.exception_occurred(node, _('Object'),
                    _('Trying to read during GET'))
+                success = False
         finally:
-            # Ensure the queue getter gets an empty-string-terminator.
+            # Ensure the queue getter gets a terminator.
             queue.resize(2)
-            queue.put('')
+            queue.put(success)
             # Close-out the connection as best as possible.
             if getattr(source, 'swift_conn', None):
                 try:
@@ -733,7 +736,11 @@ class Controller(object):
                 source = node = None
                 while True:
                     chunk = queue.get(timeout=self.app.node_timeout)
-                    if not chunk:
+                    if isinstance(chunk, bool):  # terminator
+                        success = chunk
+                        if not success:
+                            raise Exception(_('Failed to read all data'
+                                              ' from the source'))
                         break
                     yield chunk
             except Empty: