jeudi 23 juin 2016

How to constrain python logging output per package when using multiple packages


Context: The current project I'm working on has been the first project serious enough to require a detailed logging process, so I'm in the process of learning the python logging module, there are some good examples in the python docs, but nothing like my use case. I'm wondering what the best practices are for keeping log files separate between two python modules, in a use case where one imports the other. My current project consists of two python packages, kazooapi, and kazooaccount. kazooaccount will eventually import kazooapi, and logging for both packages are setup in init.py. Package Structure: essentially identical in both packages kazooaccount/__init__.py: from . import helpers helpers.init_logger() kazooaccount/helpers.py: import logging def init_logger(): # set the requests library module level requests_level = log_level_from_string(config.REQUESTS_LOG_LEVEL) logging.getLogger("requests").setLevel(requests_level) logger = logging.getLogger() logger.setLevel(logging.DEBUG) fh = logging.FileHandler(config.LOG_FILE) fh_level = log_level_from_string(config.FILE_LOG_LEVEL) fh.setLevel(fh_level) ch = logging.StreamHandler() ch_level = log_level_from_string(config.CONSOLE_LOG_LEVEL) ch.setLevel(ch_level) formatter = logging.Formatter(config.LOG_FORMAT) fh.setFormatter(formatter) ch.setFormatter(formatter) logger.addHandler(fh) logger.addHandler(ch) return logger Each package has a different config module, which contains a constant LOG_FILE, that points to /var/log/kazooapi.log or /var/log/kazooaccount.log respectively. Test Case: I run the following python script to test the current behavior, while tailing both of the above log files test.py import logging logger = logging.getLogger(__name__) logger.info('Before kazooapi') import kazooapi logger.info('After kazooapi') import kazooaccount logger.info('After kazooaccount') The following shows up in my the files: /var/log/kazooapi.log [2016-06-23 01:25:59,589 - INFO - __main__ - <module>] After kazooapi [2016-06-23 01:25:59,590 - INFO - __main__ - <module>] After kazooaccount /var/log/kazooaccount.log [2016-06-23 01:25:59,589 - INFO - __main__ - <module>] After kazooapi [2016-06-23 01:25:59,590 - INFO - __main__ - <module>] After kazooaccount Question: I'm sure this is expected behavior, but the behavior I'm aiming for is for log messages issued inside the kazooapi package to goto /var/log/kazooapi.log only and for log messages made inside kazooaccount to goto /var/log/kazooaccount.log only. Since the logging module more or less has singleton behavior, I'm unsure how to go about this in the best manner. What am I not understanding and how can I keep these log messages separate? Thanks in advance

Aucun commentaire:

Enregistrer un commentaire