Following a memcache restart in a SAIO, I've seen the following happen
during an object HEAD:
- etag_quoter wants to get account/container info to decide whether to
quote-wrap or not
- account info is a cache miss, so we make a no-auth'ed HEAD to the next
filter in the pipeline
- eventually this gets down to ratelimit, which *also* wants to get
account info
- still a cache miss, so we make a *separate* HEAD that eventually talks
to the backend and populates cache
- ratelimit realizes it can't ratelimit the request and lets the
original HEAD through to the backend
There's a related bug about how something similar can happen when the
backend gets overloaded, but *everything is working* -- we just ought to
be talking straight to the proxy app.
Note that there's likely something similar going on with container info,
but the hardcoded 10% sampling rate makes it harder to see if you're
monitoring raw metric streams.
I thought I fixed this in the related change, but no :-/
Change-Id: I49447c62abf9375541f396f984c91e128b8a05d5
Related-Change: If9249a42b30e2a2e7c4b0b91f947f24bf891b86f
Related-Bug: #1883214
b68cc893f7