In-depth documentation on paste.deploy configuration files.
get_logger now uses python's log routing to separate multiple log_names and log_facilities in the same process.
This commit is contained in:
gholt
Tarmac
commit
b1766acbaf
@ -134,9 +134,80 @@ can be found in the :doc:`Ring Overview <overview_ring>`.
|
||||
General Server Configuration
|
||||
----------------------------
|
||||
|
||||
Swift uses paste.deploy to manage server configurations. Default configuration
|
||||
options are set in the `[DEFAULT]` section, and any options specified there
|
||||
can be overridden in any of the other sections.
|
||||
Swift uses paste.deploy (http://pythonpaste.org/deploy/) to manage server
|
||||
configurations. Default configuration options are set in the `[DEFAULT]`
|
||||
section, and any options specified there can be overridden in any of the other
|
||||
sections BUT ONLY BY USING THE SYNTAX ``set option_name = value``. This is the
|
||||
unfortunate way paste.deploy works and I'll try to explain it in full.
|
||||
|
||||
First, here's an example paste.deploy configuration file::
|
||||
|
||||
[DEFAULT]
|
||||
name1 = globalvalue
|
||||
name2 = globalvalue
|
||||
name3 = globalvalue
|
||||
set name4 = globalvalue
|
||||
|
||||
[pipeline:main]
|
||||
pipeline = myapp
|
||||
|
||||
[app:myapp]
|
||||
use = egg:mypkg#myapp
|
||||
name2 = localvalue
|
||||
set name3 = localvalue
|
||||
set name5 = localvalue
|
||||
name6 = localvalue
|
||||
|
||||
The resulting configuration that myapp receives is::
|
||||
|
||||
global {'__file__': '/etc/mypkg/wsgi.conf', 'here': '/etc/mypkg',
|
||||
'name1': 'globalvalue',
|
||||
'name2': 'globalvalue',
|
||||
'name3': 'localvalue',
|
||||
'name4': 'globalvalue',
|
||||
'name5': 'localvalue',
|
||||
'set name4': 'globalvalue'}
|
||||
local {'name6': 'localvalue'}
|
||||
|
||||
So, `name1` got the global value which is fine since it's only in the `DEFAULT`
|
||||
section anyway.
|
||||
|
||||
`name2` got the global value from `DEFAULT` even though it's seemingly
|
||||
overridden in the `app:myapp` subsection. This is just the unfortunate way
|
||||
paste.deploy works (at least at the time of this writing.)
|
||||
|
||||
`name3` got the local value from the `app:myapp` subsection because it using
|
||||
the special paste.deploy syntax of ``set option_name = value``. So, if you want
|
||||
a default value for most app/filters but want to overridde it in one
|
||||
subsection, this is how you do it.
|
||||
|
||||
`name4` got the global value from `DEFAULT` since it's only in that section
|
||||
anyway. But, since we used the ``set`` syntax in the `DEFAULT` section even
|
||||
though we shouldn't, notice we also got a ``set name4`` variable. Weird, but
|
||||
probably not harmful.
|
||||
|
||||
`name5` got the local value from the `app:myapp` subsection since it's only
|
||||
there anyway, but notice that it is in the global configuration and not the
|
||||
local configuration. This is because we used the ``set`` syntax to set the
|
||||
value. Again, weird, but not harmful since Swift just treats the two sets of
|
||||
configuration values as one set anyway.
|
||||
|
||||
`name6` got the local value from `app:myapp` subsection since it's only there,
|
||||
and since we didn't use the ``set`` syntax, it's only in the local
|
||||
configuration and not the global one. Though, as indicated above, there is no
|
||||
special distinction with Swift.
|
||||
|
||||
That's quite an explanation for something that should be so much simpler, but
|
||||
it might be important to know how paste.deploy interprets configuration files.
|
||||
The main rule to remember when working with Swift configuration files is:
|
||||
|
||||
.. note::
|
||||
|
||||
Use the ``set option_name = value`` syntax in subsections if the option is
|
||||
also set in the ``[DEFAULT]`` section. Don't get in the habit of always
|
||||
using the ``set`` syntax or you'll probably mess up your non-paste.deploy
|
||||
configuration files.
|
||||
|
||||
|
||||
---------------------------
|
||||
Object Server Configuration
|
||||
@ -170,10 +241,10 @@ Option Default Description
|
||||
use paste.deploy entry point for the object
|
||||
server. For most cases, this should be
|
||||
`egg:swift#object`.
|
||||
log_name object-server Label used when logging
|
||||
log_facility LOG_LOCAL0 Syslog log facility
|
||||
log_level INFO Logging level
|
||||
log_requests True Whether or not to log each request
|
||||
set log_name object-server Label used when logging
|
||||
set log_facility LOG_LOCAL0 Syslog log facility
|
||||
set log_level INFO Logging level
|
||||
set log_requests True Whether or not to log each request
|
||||
user swift User to run as
|
||||
node_timeout 3 Request timeout to external services
|
||||
conn_timeout 0.5 Connection timeout to external services
|
||||
@ -271,9 +342,9 @@ Option Default Description
|
||||
use paste.deploy entry point for the
|
||||
container server. For most cases, this
|
||||
should be `egg:swift#container`.
|
||||
log_name container-server Label used when logging
|
||||
log_facility LOG_LOCAL0 Syslog log facility
|
||||
log_level INFO Logging level
|
||||
set log_name container-server Label used when logging
|
||||
set log_facility LOG_LOCAL0 Syslog log facility
|
||||
set log_level INFO Logging level
|
||||
node_timeout 3 Request timeout to external services
|
||||
conn_timeout 0.5 Connection timeout to external services
|
||||
================== ================ ========================================
|
||||
@ -358,9 +429,9 @@ Option Default Description
|
||||
use Entry point for paste.deploy for the account
|
||||
server. For most cases, this should be
|
||||
`egg:swift#account`.
|
||||
log_name account-server Label used when logging
|
||||
log_facility LOG_LOCAL0 Syslog log facility
|
||||
log_level INFO Logging level
|
||||
set log_name account-server Label used when logging
|
||||
set log_facility LOG_LOCAL0 Syslog log facility
|
||||
set log_level INFO Logging level
|
||||
================== ============== ==========================================
|
||||
|
||||
[account-replicator]
|
||||
@ -439,10 +510,10 @@ use Entry point for paste.deploy for
|
||||
the proxy server. For most
|
||||
cases, this should be
|
||||
`egg:swift#proxy`.
|
||||
log_name proxy-server Label used when logging
|
||||
log_facility LOG_LOCAL0 Syslog log facility
|
||||
log_level INFO Log level
|
||||
log_headers True If True, log headers in each
|
||||
set log_name proxy-server Label used when logging
|
||||
set log_facility LOG_LOCAL0 Syslog log facility
|
||||
set log_level INFO Log level
|
||||
set log_headers True If True, log headers in each
|
||||
request
|
||||
recheck_account_existence 60 Cache timeout in seconds to
|
||||
send memcached for account
|
||||
@ -500,10 +571,10 @@ use Entry point for
|
||||
auth. To use the swauth
|
||||
set to:
|
||||
`egg:swift#swauth`
|
||||
log_name auth-server Label used when logging
|
||||
log_facility LOG_LOCAL0 Syslog log facility
|
||||
log_level INFO Log level
|
||||
log_headers True If True, log headers in
|
||||
set log_name auth-server Label used when logging
|
||||
set log_facility LOG_LOCAL0 Syslog log facility
|
||||
set log_level INFO Log level
|
||||
set log_headers True If True, log headers in
|
||||
each request
|
||||
reseller_prefix AUTH The naming scope for the
|
||||
auth service. Swift
|
||||
|
||||
@ -7,18 +7,27 @@
|
||||
# swift_dir = /etc/swift
|
||||
# devices = /srv/node
|
||||
# mount_check = true
|
||||
# You can specify default log routing here if you want:
|
||||
# log_name = swift
|
||||
# log_facility = LOG_LOCAL0
|
||||
# log_level = INFO
|
||||
|
||||
[pipeline:main]
|
||||
pipeline = account-server
|
||||
|
||||
[app:account-server]
|
||||
use = egg:swift#account
|
||||
# log_name = account-server
|
||||
# log_facility = LOG_LOCAL0
|
||||
# log_level = INFO
|
||||
# You can override the default log routing for this app here:
|
||||
# set log_name = account-server
|
||||
# set log_facility = LOG_LOCAL0
|
||||
# set log_level = INFO
|
||||
# set log_requests = True
|
||||
|
||||
[account-replicator]
|
||||
# You can override the default log routing for this app here (don't use set!):
|
||||
# log_name = account-replicator
|
||||
# log_facility = LOG_LOCAL0
|
||||
# log_level = INFO
|
||||
# vm_test_mode = no
|
||||
# log_facility = LOG_LOCAL0
|
||||
# log_level = INFO
|
||||
@ -36,7 +45,10 @@ use = egg:swift#account
|
||||
# reclaim_age = 86400
|
||||
|
||||
[account-stats]
|
||||
# You can override the default log routing for this app here (don't use set!):
|
||||
# log_name = account-stats
|
||||
# log_facility = LOG_LOCAL0
|
||||
# log_level = INFO
|
||||
# cf_account = AUTH_7abbc116-8a07-4b63-819d-02715d3e0f31
|
||||
# container_name = account_stats
|
||||
# proxy_server_conf = /etc/swift/proxy-server.conf
|
||||
@ -44,14 +56,20 @@ use = egg:swift#account
|
||||
# log_level = INFO
|
||||
|
||||
[account-auditor]
|
||||
# You can override the default log routing for this app here (don't use set!):
|
||||
# log_name = account-auditor
|
||||
# log_facility = LOG_LOCAL0
|
||||
# log_level = INFO
|
||||
# Will audit, at most, 1 account per device per interval
|
||||
# interval = 1800
|
||||
# log_facility = LOG_LOCAL0
|
||||
# log_level = INFO
|
||||
|
||||
[account-reaper]
|
||||
# You can override the default log routing for this app here (don't use set!):
|
||||
# log_name = account-reaper
|
||||
# log_facility = LOG_LOCAL0
|
||||
# log_level = INFO
|
||||
# concurrency = 25
|
||||
# interval = 3600
|
||||
# node_timeout = 10
|
||||
|
||||
@ -7,6 +7,10 @@
|
||||
# swift_dir = /etc/swift
|
||||
# cert_file = Default is no cert; format is path like /etc/swift/auth.crt
|
||||
# key_file = Default is no key; format is path like /etc/swift/auth.key
|
||||
# You can specify default log routing here if you want:
|
||||
# log_name = swift
|
||||
# log_facility = LOG_LOCAL0
|
||||
# log_level = INFO
|
||||
|
||||
[pipeline:main]
|
||||
pipeline = auth-server
|
||||
@ -15,11 +19,12 @@ pipeline = auth-server
|
||||
use = egg:swift#auth
|
||||
# Highly recommended to change this.
|
||||
super_admin_key = devauth
|
||||
# log_name = auth-server
|
||||
# log_facility = LOG_LOCAL0
|
||||
# log_level = INFO
|
||||
# You can override the default log routing for this app here:
|
||||
# set log_name = proxy-server
|
||||
# set log_facility = LOG_LOCAL0
|
||||
# set log_level = INFO
|
||||
# set log_headers = False
|
||||
# reseller_prefix = AUTH
|
||||
# default_cluster_url = http://127.0.0.1:8080/v1
|
||||
# token_life = 86400
|
||||
# log_headers = False
|
||||
# node_timeout = 10
|
||||
|
||||
@ -7,20 +7,29 @@
|
||||
# swift_dir = /etc/swift
|
||||
# devices = /srv/node
|
||||
# mount_check = true
|
||||
# You can specify default log routing here if you want:
|
||||
# log_name = swift
|
||||
# log_facility = LOG_LOCAL0
|
||||
# log_level = INFO
|
||||
|
||||
[pipeline:main]
|
||||
pipeline = container-server
|
||||
|
||||
[app:container-server]
|
||||
use = egg:swift#container
|
||||
# log_name = container-server
|
||||
# log_facility = LOG_LOCAL0
|
||||
# log_level = INFO
|
||||
# You can override the default log routing for this app here:
|
||||
# set log_name = container-server
|
||||
# set log_facility = LOG_LOCAL0
|
||||
# set log_level = INFO
|
||||
# set log_requests = True
|
||||
# node_timeout = 3
|
||||
# conn_timeout = 0.5
|
||||
|
||||
[container-replicator]
|
||||
# You can override the default log routing for this app here (don't use set!):
|
||||
# log_name = container-replicator
|
||||
# log_facility = LOG_LOCAL0
|
||||
# log_level = INFO
|
||||
# vm_test_mode = no
|
||||
# per_diff = 1000
|
||||
# concurrency = 8
|
||||
@ -31,7 +40,10 @@ use = egg:swift#container
|
||||
# reclaim_age = 604800
|
||||
|
||||
[container-updater]
|
||||
# You can override the default log routing for this app here (don't use set!):
|
||||
# log_name = container-updater
|
||||
# log_facility = LOG_LOCAL0
|
||||
# log_level = INFO
|
||||
# interval = 300
|
||||
# concurrency = 4
|
||||
# node_timeout = 3
|
||||
@ -40,6 +52,9 @@ use = egg:swift#container
|
||||
# slowdown = 0.01
|
||||
|
||||
[container-auditor]
|
||||
# You can override the default log routing for this app here (don't use set!):
|
||||
# log_name = container-auditor
|
||||
# log_facility = LOG_LOCAL0
|
||||
# log_level = INFO
|
||||
# Will audit, at most, 1 container per device per interval
|
||||
# interval = 1800
|
||||
|
||||
@ -7,16 +7,21 @@
|
||||
# swift_dir = /etc/swift
|
||||
# devices = /srv/node
|
||||
# mount_check = true
|
||||
# You can specify default log routing here if you want:
|
||||
# log_name = swift
|
||||
# log_facility = LOG_LOCAL0
|
||||
# log_level = INFO
|
||||
|
||||
[pipeline:main]
|
||||
pipeline = object-server
|
||||
|
||||
[app:object-server]
|
||||
use = egg:swift#object
|
||||
# log_name = object-server
|
||||
# log_facility = LOG_LOCAL0
|
||||
# log_level = INFO
|
||||
# log_requests = True
|
||||
# You can override the default log routing for this app here:
|
||||
# set log_name = object-server
|
||||
# set log_facility = LOG_LOCAL0
|
||||
# set log_level = INFO
|
||||
# set log_requests = True
|
||||
# node_timeout = 3
|
||||
# conn_timeout = 0.5
|
||||
# network_chunk_size = 65536
|
||||
@ -27,7 +32,10 @@ use = egg:swift#object
|
||||
# mb_per_sync = 512
|
||||
|
||||
[object-replicator]
|
||||
# You can override the default log routing for this app here (don't use set!):
|
||||
# log_name = object-replicator
|
||||
# log_facility = LOG_LOCAL0
|
||||
# log_level = INFO
|
||||
# vm_test_mode = no
|
||||
# daemonize = on
|
||||
# run_pause = 30
|
||||
@ -45,7 +53,10 @@ use = egg:swift#object
|
||||
# reclaim_age = 604800
|
||||
|
||||
[object-updater]
|
||||
# You can override the default log routing for this app here (don't use set!):
|
||||
# log_name = object-updater
|
||||
# log_facility = LOG_LOCAL0
|
||||
# log_level = INFO
|
||||
# interval = 300
|
||||
# concurrency = 1
|
||||
# node_timeout = 10
|
||||
@ -54,7 +65,10 @@ use = egg:swift#object
|
||||
# slowdown = 0.01
|
||||
|
||||
[object-auditor]
|
||||
# You can override the default log routing for this app here (don't use set!):
|
||||
# log_name = object-auditor
|
||||
# log_facility = LOG_LOCAL0
|
||||
# log_level = INFO
|
||||
# files_per_second = 20
|
||||
# bytes_per_second = 10000000
|
||||
# log_time = 3600
|
||||
@ -7,6 +7,10 @@
|
||||
# user = swift
|
||||
# cert_file = /etc/swift/proxy.crt
|
||||
# key_file = /etc/swift/proxy.key
|
||||
# You can specify default log routing here if you want:
|
||||
# log_name = swift
|
||||
# log_facility = LOG_LOCAL0
|
||||
# log_level = INFO
|
||||
|
||||
[pipeline:main]
|
||||
# For DevAuth:
|
||||
@ -16,10 +20,11 @@ pipeline = catch_errors healthcheck cache ratelimit auth proxy-server
|
||||
|
||||
[app:proxy-server]
|
||||
use = egg:swift#proxy
|
||||
# log_name = proxy-server
|
||||
# log_facility = LOG_LOCAL0
|
||||
# log_level = INFO
|
||||
# log_headers = False
|
||||
# You can override the default log routing for this app here:
|
||||
# set log_name = proxy-server
|
||||
# set log_facility = LOG_LOCAL0
|
||||
# set log_level = INFO
|
||||
# set log_headers = False
|
||||
# recheck_account_existence = 60
|
||||
# recheck_container_existence = 60
|
||||
# object_chunk_size = 8192
|
||||
@ -39,6 +44,11 @@ use = egg:swift#proxy
|
||||
# Only needed for DevAuth
|
||||
[filter:auth]
|
||||
use = egg:swift#auth
|
||||
# You can override the default log routing for this filter here:
|
||||
# set log_name = auth-server
|
||||
# set log_facility = LOG_LOCAL0
|
||||
# set log_level = INFO
|
||||
# set log_headers = False
|
||||
# The reseller prefix will verify a token begins with this prefix before even
|
||||
# attempting to validate it with the external authentication server. Also, with
|
||||
# authorization, only Swift storage accounts with this prefix will be
|
||||
@ -54,10 +64,11 @@ use = egg:swift#auth
|
||||
# Only needed for Swauth
|
||||
[filter:swauth]
|
||||
use = egg:swift#swauth
|
||||
# log_name = auth-server
|
||||
# log_facility = LOG_LOCAL0
|
||||
# log_level = INFO
|
||||
# log_headers = False
|
||||
# You can override the default log routing for this filter here:
|
||||
# set log_name = auth-server
|
||||
# set log_facility = LOG_LOCAL0
|
||||
# set log_level = INFO
|
||||
# set log_headers = False
|
||||
# The reseller prefix will verify a token begins with this prefix before even
|
||||
# attempting to validate it. Also, with authorization, only Swift storage
|
||||
# accounts with this prefix will be authorized by this middleware. Useful if
|
||||
@ -82,15 +93,30 @@ super_admin_key = swauthkey
|
||||
|
||||
[filter:healthcheck]
|
||||
use = egg:swift#healthcheck
|
||||
# You can override the default log routing for this filter here:
|
||||
# set log_name = auth-server
|
||||
# set log_facility = LOG_LOCAL0
|
||||
# set log_level = INFO
|
||||
# set log_headers = False
|
||||
|
||||
[filter:cache]
|
||||
use = egg:swift#memcache
|
||||
# You can override the default log routing for this filter here:
|
||||
# set log_name = auth-server
|
||||
# set log_facility = LOG_LOCAL0
|
||||
# set log_level = INFO
|
||||
# set log_headers = False
|
||||
# Default for memcache_servers is below, but you can specify multiple servers
|
||||
# with the format: 10.1.2.3:11211,10.1.2.4:11211
|
||||
# memcache_servers = 127.0.0.1:11211
|
||||
|
||||
[filter:ratelimit]
|
||||
use = egg:swift#ratelimit
|
||||
# You can override the default log routing for this filter here:
|
||||
# set log_name = auth-server
|
||||
# set log_facility = LOG_LOCAL0
|
||||
# set log_level = INFO
|
||||
# set log_headers = False
|
||||
# clock_accuracy should represent how accurate the proxy servers' system clocks
|
||||
# are with each other. 1000 means that all the proxies' clock are accurate to
|
||||
# each other within 1 millisecond. No ratelimit should be higher than the
|
||||
@ -118,14 +144,29 @@ use = egg:swift#ratelimit
|
||||
|
||||
[filter:domain_remap]
|
||||
use = egg:swift#domain_remap
|
||||
# You can override the default log routing for this filter here:
|
||||
# set log_name = auth-server
|
||||
# set log_facility = LOG_LOCAL0
|
||||
# set log_level = INFO
|
||||
# set log_headers = False
|
||||
# storage_domain = example.com
|
||||
# path_root = v1
|
||||
|
||||
[filter:catch_errors]
|
||||
use = egg:swift#catch_errors
|
||||
# You can override the default log routing for this filter here:
|
||||
# set log_name = auth-server
|
||||
# set log_facility = LOG_LOCAL0
|
||||
# set log_level = INFO
|
||||
# set log_headers = False
|
||||
|
||||
[filter:cname_lookup]
|
||||
# Note: this middleware requires python-dnspython
|
||||
use = egg:swift#cname_lookup
|
||||
# You can override the default log routing for this filter here:
|
||||
# set log_name = auth-server
|
||||
# set log_facility = LOG_LOCAL0
|
||||
# set log_level = INFO
|
||||
# set log_headers = False
|
||||
# storage_domain = example.com
|
||||
# lookup_depth = 1
|
||||
|
||||
@ -382,7 +382,7 @@ class NamedFormatter(logging.Formatter):
|
||||
return msg
|
||||
|
||||
|
||||
def get_logger(conf, name=None, log_to_console=False):
|
||||
def get_logger(conf, name=None, log_to_console=False, log_route=None):
|
||||
"""
|
||||
Get the current system logger using config settings.
|
||||
|
||||
@ -396,33 +396,41 @@ def get_logger(conf, name=None, log_to_console=False):
|
||||
:param name: Name of the logger
|
||||
:param log_to_console: Add handler which writes to console on stderr
|
||||
"""
|
||||
root_logger = logging.getLogger()
|
||||
if hasattr(get_logger, 'handler') and get_logger.handler:
|
||||
root_logger.removeHandler(get_logger.handler)
|
||||
get_logger.handler.close()
|
||||
get_logger.handler = None
|
||||
if not conf:
|
||||
conf = {}
|
||||
if not hasattr(get_logger, 'root_logger_configured'):
|
||||
get_logger.root_logger_configured = True
|
||||
get_logger(conf, name, log_to_console, log_route='root')
|
||||
if name is None:
|
||||
name = conf.get('log_name', 'swift')
|
||||
if not log_route:
|
||||
log_route = name
|
||||
if log_route == 'root':
|
||||
logger = logging.getLogger()
|
||||
else:
|
||||
logger = logging.getLogger(log_route)
|
||||
if not hasattr(get_logger, 'handlers'):
|
||||
get_logger.handlers = {}
|
||||
facility = getattr(SysLogHandler, conf.get('log_facility', 'LOG_LOCAL0'),
|
||||
SysLogHandler.LOG_LOCAL0)
|
||||
if facility in get_logger.handlers:
|
||||
logger.removeHandler(get_logger.handlers[facility])
|
||||
get_logger.handlers[facility].close()
|
||||
del get_logger.handlers[facility]
|
||||
if log_to_console:
|
||||
# check if a previous call to get_logger already added a console logger
|
||||
if hasattr(get_logger, 'console') and get_logger.console:
|
||||
root_logger.removeHandler(get_logger.console)
|
||||
logger.removeHandler(get_logger.console)
|
||||
get_logger.console = logging.StreamHandler(sys.__stderr__)
|
||||
root_logger.addHandler(get_logger.console)
|
||||
if conf is None:
|
||||
root_logger.setLevel(logging.INFO)
|
||||
adapted_logger = LogAdapter(root_logger)
|
||||
return adapted_logger
|
||||
if name is None:
|
||||
name = conf.get('log_name', 'swift')
|
||||
get_logger.handler = SysLogHandler(address='/dev/log',
|
||||
facility=getattr(SysLogHandler,
|
||||
conf.get('log_facility', 'LOG_LOCAL0'),
|
||||
SysLogHandler.LOG_LOCAL0))
|
||||
root_logger.addHandler(get_logger.handler)
|
||||
root_logger.setLevel(
|
||||
logger.addHandler(get_logger.console)
|
||||
get_logger.handlers[facility] = \
|
||||
SysLogHandler(address='/dev/log', facility=facility)
|
||||
logger.addHandler(get_logger.handlers[facility])
|
||||
logger.setLevel(
|
||||
getattr(logging, conf.get('log_level', 'INFO').upper(), logging.INFO))
|
||||
adapted_logger = LogAdapter(root_logger)
|
||||
adapted_logger = LogAdapter(logger)
|
||||
formatter = NamedFormatter(name, adapted_logger)
|
||||
get_logger.handler.setFormatter(formatter)
|
||||
get_logger.handlers[facility].setFormatter(formatter)
|
||||
if hasattr(get_logger, 'console'):
|
||||
get_logger.console.setFormatter(formatter)
|
||||
return adapted_logger
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user