最近,我开始着手构建一个简单的Web搜寻器。我刚迭代两次的初始代码可以完美地工作,但是当我尝试将其转换为具有错误异常处理的类时,便不再进行编译。
import re, urllib
class WebCrawler:
"""A Simple Web Crawler That Is Readily Extensible"""
def __init__():
size = 1
def containsAny(seq, aset):
for c in seq:
if c in aset: return True
return False
def crawlUrls(url, depth):
textfile = file('UrlMap.txt', 'wt')
urlList = [url]
size = 1
for i in range(depth):
for ee in range(size):
if containsAny(urlList[ee], "http://"):
try:
webpage = urllib.urlopen(urlList[ee]).read()
break
except:
print "Following URL failed!"
print urlList[ee]
for ee in re.findall('''href=["'](.[^"']+)["']''',webpage, re.I):
print ee
urlList.append(ee)
size+=1
textfile.write(ee+'\n')
myCrawler = WebCrawler
myCrawler.crawlUrls("http://www.wordsmakeworlds.com/", 2)
这是生成的错误代码。
Traceback (most recent call last):
File "C:/Users/Noah Huber-Feely/Desktop/Python/WebCrawlerClass", line 33, in <module>
myCrawler.crawlUrls("http://www.wordsmakeworlds.com/", 2)
TypeError: unbound method crawlUrls() must be called with WebCrawler instance as first argument (got str instance instead)
你有两个问题。这一行是一:
myCrawler = WebCrawler
您不是在创建的实例WebCrawler
,而只是将名称绑定myCrawler
到WebCrawler
(基本上是为该类创建别名)。您应该改为这样做:
myCrawler = WebCrawler()
然后,在此行上:
def crawlUrls(url, depth):
Python实例方法将接收方作为该方法的第一个参数。它通常称为self
,但从技术上讲,您可以随心所欲地调用它。因此,您应该将方法签名更改为:
def crawlUrls(self, url, depth):
(您还需要对定义的其他方法执行此操作。)
本文收集自互联网,转载请注明来源。
如有侵权,请联系[email protected] 删除。
我来说两句