Python Web抓取“ IndexError：列表索引超出范围”

debugcn 发表于 Dev

布罗格默

from selenium import webdriver
import csv
import requests
from bs4 import BeautifulSoup

driver = webdriver.Chrome(executable_path="C:\\Users\\dylan\\Documents\\chromedriver.exe")
data_list=[]


site = requests.get('https://www.visitnh.gov/things-to-do/food-and-drink/restaurants')

if site.status_code is 200:
   content = BeautifulSoup(site.content, 'html.parser')
   Resultswrapper = content.find_all('div', attrs={'results-wrapper'})

for Results in Resultswrapper:
    print("Random phrase: ")
    #print(Results.select(class_='results-wrapper').prettify())
    BusinessName = Results.select('.item-title.ng-binding')[0].get_text()
    Address =   Results.select('.ng-binding')[0].get_text()
    PhoneNumber = Results.select('.ng-binding')[0].get_text()
    new_data = {"BusinessName": BusinessName, "Address": Address, "PhoneNumber": PhoneNumber}
    data_list.append(new_data)
    print("data: " + data_list)
    print("new data: " + new_data)

with open ('find.csv','w') as file:
        writer = csv.DictWriter(file, fieldnames = ["BusinessName", "Address", "PhoneNumber"], delimiter = ';')
        writer.writeheader()
        for row in data_list:
            writer.writerow(row)

尝试制作简单的Web抓取工具时出现索引超出范围错误，并且尝试使用选择器循环。我使用以下URL：https : //www.visitnh.gov/things-to-do/food-and-drink/restaurants

Traceback (most recent call last):
File "c:/Users/dylan/Documents/Webscrape/web-s.py", line 19, in <module>
BusinessName = Results.select('p.item-title.ng-binding')[0].get_text()
IndexError: list index out of range

我尝试将结果包装器更改为HTM1中的其他内容，但并不相同。我还尝试弄乱选择中的文本，但没有用。有任何想法吗？任何帮助将不胜感激。

安德烈·凯斯利（Andrej Kesely）

数据是通过javascript从外部URL加载的。您可以使用requests/json模块来模拟此请求：

import json
import requests


url = 'https://www.visitnh.gov/BusinessListingService.asmx/GetUsers'
search_params = {"searchParams":{"ResultsPerPage":15,"BusinessType":"attraction","SubcategoryID":25}}
data = json.loads(requests.post(url, json=search_params).json()['d'])

# uncomment this to print all data:
# print(json.dumps(data, indent=4))

for result in data['Results']:
    print(result['BusinessName'])
    print(result['Address'])
    print(result['Phone'])
    print('-' * 80)

印刷品：

NazBar & Grill
1086 Weirs Blvd, Laconia, NH 03246
(603) 366-4341
--------------------------------------------------------------------------------
Giant of Siam
5 E Hollis Street, Nashua, NH 03060
(603) 595-2222
--------------------------------------------------------------------------------
Hobbs Tavern & Brewing Company
2415 NH Route 16, Ossipee, NH 03890
(603) 539-2000
--------------------------------------------------------------------------------

...and so on. (Total 164 items.)

本文收集自互联网，转载请注明来源。

如有侵权，请联系[email protected] 删除。