我很难在HTTP请求的日志文件中获取DEBUG级别的日志,例如从控制台获取日志:
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): URI:443
DEBUG:urllib3.connectionpool:URL:443 "POST /endpoint HTTP/1.1" 200 None
对于以下代码:
import logging
from logging.handlers import TimedRotatingFileHandler
_logger = logging.getLogger(__name__)
def setup_logging(loglevel):
logFormatter = logging.Formatter("%(asctime)s [%(threadName)-12.12s] [%(levelname)-5.5s] %(message)s")
if loglevel is not None:
if loglevel == 10:
http.client.HTTPConnection.debuglevel = 1
logformat = "%(asctime)s [%(threadName)-12.12s] [%(levelname)-5.5s] %(message)s"
logging.basicConfig(level=loglevel, stream=sys.stdout, format=logformat, datefmt="%Y-%m-%d %H:%M:%S")
fileHandler = logging.handlers.TimedRotatingFileHandler("{0}/{1}.log".format(logPath, logFileName), when="midnight")
fileHandler.setFormatter(logFormatter)
_logger.setLevel(logging.DEBUG)
_logger.addHandler(fileHandler)
当我将其与logging.DEBUG
日志文件一起调用时,它将仅包含我将在代码中指定为_logger.info
或_logger.debug
与控制台日志输出类似的内容。
PS。示例代码我怎么称呼它:
def main(args):
args = parse_args(args)
cfg = config(args.env)
setup_logging(logging.DEBUG, cfg)
requests.get("https://stackoverflow.com/a/58769712/100297")
您要在错误的位置添加处理程序和级别更改。
Python日志记录模块根据记录器对象的名称以及.
这些名称中是否存在分隔符将其视为存在于层次结构中。"foo.bar.baz"
逻辑上,记录器的名称应作为foo.bar
和的子级放置(foo
如果存在)。根记录器是层次结构的基础,它没有名称。您访问它logging.getLogger()
(没有参数,但''
并None
也将工作)。
现在,在记录消息时,首先,消息必须通过记录器的有效级别。如果它们通过了,则将消息传递到每个记录器的处理程序,从当前记录器到根,前提是它们清除了找到的每个处理程序的级别。
为了找到有效的级别,遍历层次结构以找到具有级别集的最近的记录器对象。如果没有,则消息总是通过。在遍历层次结构以查找处理程序时,日志对象可以阻止传播(propagate
设置为False
),此时遍历停止。
当您尝试处理的消息时urllib3.connectionpool()
,您需要在以下三个位置之一放置处理程序:的logger urllib3.connectionpool
,forurllib3
或root logger。您的代码无法做到这一点。
相反,您可以使用其他名称在自己的记录器上设置处理程序:
_logger = logging.getLogger(__name__)
这样可以保证不匹配根记录器(__name__
需要为空,永远都不是),也不匹配urllib3
orurllib3.connectionpool
记录器(这意味着您的模块也称为urllib3
or)urllib3.connectionpool)
。
因为urllib3.connectionpool
日志消息不在路径中,所以永远不会为您的处理程序提供这些消息。
相反,您要配置root记录器:
fileHandler = logging.handlers.TimedRotatingFileHandler("{0}/{1}.log".format(logPath, logFileName), when="midnight")
fileHandler.setFormatter(logFormatter)
root = logging.getLogger()
root.setLevel(logging.DEBUG)
root.addHandler(fileHandler)
您可以将每个处理程序的日志级别设置为要在该处理程序上看到的日志级别,而不是在根记录器上(或除根记录器之外)。请记住,在根记录器上设置的级别用于确定层次结构中其他未直接设置级别的记录器的有效级别,并且该有效级别充当所有邮件的“高水位标记”。如果将根记录器设置为INFO
,并且未配置其他处理程序,则您的处理程序将永远不会看到DEBUG
消息。根记录器的默认级别为WARNING
。
通常,您通常只想在模块中使用命名记录器来生成日志消息,并将所有处理程序放在根目录上。可以使用其他任何特殊的记录器模块(例如,专门用于urllib3
记录消息的专用处理程序,或通过将其最低级别的记录器对象设置为来使整个程序包静音propagate = False
)。
最后,logging.basicConfig()
配置根记录过,但只有如果有根logger进行处理了。如果你使用force=True
那么basicConfig()
将删除所有现存的处理器,然后在根配置处理程序。它将始终创建一个Formatter()
实例并将其设置在所有处理程序上。
你可以使用basicConfig()
的所有的根记录的需求:
import http.client
import logging
import os.path
import sys
from logging.handlers import TimedRotatingFileHandler
def setup_logging(loglevel):
# the file handler receives all messages from level DEBUG on up, regardless
fileHandler = TimedRotatingFileHandler(
os.path.join(logPath, logFileName + ".log"),
when="midnight"
)
fileHandler.setLevel(logging.DEBUG)
handlers = [fileHandler]
if loglevel is not None:
# if a log level is configured, use that for logging to the console
stream_handler = logging.StreamHandler(sys.stdout)
stream_handler.setLevel(loglevel)
handlers.append(stream_handler)
if loglevel == logging.DEBUG:
# when logging at debug level, make http.client extra chatty too
# http.client *uses `print()` calls*, not logging.
http.client.HTTPConnection.debuglevel = 1
# finally, configure the root logger with our choice of handlers
# the logging level of the root set to DEBUG (defaults to WARNING otherwise).
logformat = "%(asctime)s [%(threadName)-12.12s] [%(levelname)-5.5s] %(message)s"
logging.basicConfig(
format=logformat, datefmt="%Y-%m-%d %H:%M:%S",
handlers=handlers, level=logging.DEBUG
)
旁注:该http.client
库不logging
用于输出调试消息;它将始终用于print()
输出那些。如果您希望看到这些消息出现在日志中,则需要对库进行猴子补丁,并添加一个替代print()
全局变量:
import http.client
import logging
http_client_logger = logging.getLogger("http.client")
def print_to_log(*args):
http_client_logger.debug(" ".join(args))
# monkey-patch a `print` global into the http.client module; all calls to
# print() in that module will then use our print_to_log implementation
http.client.print = print_to_log
通过http.client
应用上述技巧和setup_logging(logging.DEBUG)
,stdout
当我使用时,我看到以下日志同时出现在文件中和文件中requests.get("https://stackoverflow.com/a/58769712/100297")
:
2019-11-08 16:17:26 [MainThread ] [DEBUG] Starting new HTTPS connection (1): stackoverflow.com:443
2019-11-08 16:17:27 [MainThread ] [DEBUG] send: b'GET /a/58769712/100297 HTTP/1.1\r\nHost: stackoverflow.com\r\nUser-Agent: python-requests/2.22.0\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep-alive\r\n\r\n'
2019-11-08 16:17:27 [MainThread ] [DEBUG] reply: 'HTTP/1.1 302 Found\r\n'
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: Cache-Control: private
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: Content-Type: text/html; charset=utf-8
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: Location: /questions/58738195/python-http-request-and-debug-level-logging-to-the-log-file/58769712#58769712
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: X-Frame-Options: SAMEORIGIN
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: X-Request-Guid: 761bd2f8-3e5c-453a-ab46-d01284940541
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: Strict-Transport-Security: max-age=15552000
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: Feature-Policy: microphone 'none'; speaker 'none'
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: Content-Security-Policy: upgrade-insecure-requests; frame-ancestors 'self' https://stackexchange.com
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: Accept-Ranges: bytes
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: Age: 0
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: Content-Length: 214
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: Accept-Ranges: bytes
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: Date: Fri, 08 Nov 2019 16:17:27 GMT
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: Via: 1.1 varnish
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: Age: 0
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: Connection: keep-alive
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: X-Served-By: cache-lhr7324-LHR
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: X-Cache: MISS
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: X-Cache-Hits: 0
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: X-Timer: S1573229847.069848,VS0,VE80
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: Vary: Fastly-SSL
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: X-DNS-Prefetch-Control: off
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: Set-Cookie: prov=0e92634f-abce-9f8e-1865-0d35ebecc595; domain=.stackoverflow.com; expires=Fri, 01-Jan-2055 00:00:00 GMT; path=/; HttpOnly
2019-11-08 16:17:27 [MainThread ] [DEBUG] https://stackoverflow.com:443 "GET /a/58769712/100297 HTTP/1.1" 302 214
2019-11-08 16:17:27 [MainThread ] [DEBUG] send: b'GET /questions/58738195/python-http-request-and-debug-level-logging-to-the-log-file/58769712 HTTP/1.1\r\nHost: stackoverflow.com\r\nUser-Agent: python-requests/2.22.0\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep-alive\r\nCookie: prov=0e92634f-abce-9f8e-1865-0d35ebecc595\r\n\r\n'
2019-11-08 16:17:27 [MainThread ] [DEBUG] reply: 'HTTP/1.1 200 OK\r\n'
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: Cache-Control: private
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: Content-Type: text/html; charset=utf-8
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: Content-Encoding: gzip
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: Last-Modified: Fri, 08 Nov 2019 16:16:07 GMT
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: X-Frame-Options: SAMEORIGIN
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: X-Request-Guid: 5e48399e-a91c-44aa-aad6-00a96014131f
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: Strict-Transport-Security: max-age=15552000
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: Feature-Policy: microphone 'none'; speaker 'none'
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: Content-Security-Policy: upgrade-insecure-requests; frame-ancestors 'self' https://stackexchange.com
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: Accept-Ranges: bytes
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: Age: 0
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: Content-Length: 42625
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: Accept-Ranges: bytes
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: Date: Fri, 08 Nov 2019 16:17:27 GMT
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: Via: 1.1 varnish
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: Age: 0
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: Connection: keep-alive
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: X-Served-By: cache-lhr7324-LHR
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: X-Cache: MISS
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: X-Cache-Hits: 0
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: X-Timer: S1573229847.189349,VS0,VE95
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: Vary: Accept-Encoding,Fastly-SSL
2019-11-08 16:17:27 [MainThread ] [DEBUG] header: X-DNS-Prefetch-Control: off
2019-11-08 16:17:27 [MainThread ] [DEBUG] https://stackoverflow.com:443 "GET /questions/58738195/python-http-request-and-debug-level-logging-to-the-log-file/58769712 HTTP/1.1" 200 42625
您可以使用以下文件复制该文件:
import requests
import sys
logPath, logFileName = "/tmp", "demo"
level = logging.DEBUG if "-v" in sys.argv else None
setup_logging(level)
requests.get("https://stackoverflow.com/a/58769712/100297")
除了上面的代码外,还添加了。然后,您可以python <script.py> -v
用于将级别设置为详细,并将其python <script.py>
与根本没有设置的级别进行比较。
本文收集自互联网,转载请注明来源。
如有侵权,请联系[email protected] 删除。
我来说两句