from urllib import request
from bs4 import BeautifulSoup
def trade_spider(max_pages):
page = 1
while page <= max_pages:
x = 16958 + int(page)
url = 'http://mery.jp/' + str(x)
source_code = request.get(url)
plain_text = source_code.text
soup = BeautifulSoup(plain_text)
print(soup)
break
trade_spider(2)
when I try to run the code above in my Python3.4, I get an error that says the following:
File "web_crawler.py", line 15, in <module>
trade_spider(2)
File "web_crawler.py", line 9, in trade_spider
source_code = request.get(url)
AttributeError: 'module' object has no attribute 'get'
any ideas on how to fix this? Thank you in advance.
You imported request from urllib, but urllib.request doesn't have a get method. I think you want to import the requests module and use requests.get in lieu of request.get. Either that or you want to replace get with urlopen. Considering that you reference the .text attribute in the next line, it probably is the former and not the latter.
Collected from the Internet
Please contact [email protected] to delete if infringement.
Comments