🐍 Python - Logging
logging standard library module is your go-to logging framework.
import logging
# you can configure aspects of your logging with e.g. `basicConfig`
# call this before initializing any loggers
logging.basicConfig(level=logging.DEBUG)
# this is a shothand for using the root logger
logging.debug('a detailed event during normal operation')
logging.info('an event during normal operation')
logging.warning('an event that is abnormal but continuing anyway')
logging.error('an event that fails the current operation')
logging.critical('an event that crashes the system')
Initialize a separate logger for each module i.e. file. As the best practice, an individual module shouldn't configure logging any futher so it's easy to utilize that code as a dependency and let the caller configure logging as they want.
import logging
log = logging.getLogger(__name__)
log.warning('something smells fishy')
You should default to raising an exception instead of logging error or critical. Then the caller can catch that and choose how to proceed e.g. fixing the issue or logging the exception themselves with logging.exception().
import logging
log = logging.getLogger(__name__)
try:
# raising exceptions is the standard error reporting during runtime
# it's better than simple logging as the caller can try to recover
raise Exception('special log handling when used inside except block')
except Exception as e:
# if you wish to record the exception and continue, use this
log.exception(e) # will also log stack trace on ERROR level
# shorthand for `logging.error(..., exc_info=True)`
Your main application should configure logging as early as possible.
import os
import logging
# this helper adds a formatter and a log handler to the root logger
logging.basicConfig(
level=os.environ.get('LOGLEVEL', logging.INFO)
)
Your main application should wrap the program in a try/catch so exceptions get properly logged.
import logging
log = logging.getLogger(__name__)
def main():
raise Exception('something went wrong')
if __name__ == '__main__':
try:
exit(main())
except Exception as e:
log.exception(e)
exit(1)
With all this knowledge, a solid Python script main looks like the following. It's a standard practice to log to stdout and stderr by default. Process management frameworks like systemd expect this. It's also a good practice to log exceptions to a single line to make log analysis easier.
import logging
import os
import sys
log = logging.getLogger(__name__)
class OneLineExceptionFormatter(logging.Formatter):
def formatException(self, exc_info):
result = super().formatException(exc_info)
return repr(result)
def format(self, record):
result = super().format(record)
if record.exc_text:
result = result.replace('\n', ' ')
return result
def setup_logging():
formatter = OneLineExceptionFormatter(logging.BASIC_FORMAT)
out_handler = logging.StreamHandler(sys.stdout)
out_handler.addFilter(lambda record: record.levelno <= logging.INFO)
out_handler.setFormatter(formatter)
err_handler = logging.StreamHandler(sys.stderr)
err_handler.addFilter(lambda record: record.levelno > logging.INFO)
err_handler.setFormatter(formatter)
logging.basicConfig(
level=os.environ.get('LOGLEVEL', logging.INFO),
handlers=[out_handler, err_handler],
)
def main():
log.info('Hello World!')
raise Exception('something went wrong!')
return 0
if __name__ == '__main__':
setup_logging()
try:
exit(main())
except Exception as e:
log.exception('exception:')
exit(1)
If you need to log somewhere else than to console like to a file or over the network; check out different handlers available in the standard library: https://docs.python.org/3/howto/logging.html#useful-handlers
You can pass an arbitrary object as a logging message, and its __str__() method will be called when the logging system needs to convert it to a string representation.
When working with expensive functions to create loggig messages, use isEnabledFor. This condition is true only if the logger in question would log messages of that level.
if log.isEnabledFor(logging.DEBUG):
log.debug(f'Message with {expensive_func()}')