我正在尝试从重复以下HTML的网站上抓取一些信息：
<div class="product-details">
<h2 class="product-name" title=" Weekly Roundup"> Weekly Roundup</h2>
<span class="reference-number">REF NO. A1400.5</span>
我正在尝试刮擦产品名称和文本“ REF NO。A1400.5”。我需要在同一页面上抓取多个产品名称和参考编号，并将它们存储在列表中。我试过了：
product_new = []
product_ref = []
for caption in soup.find_all(class_='product-details'):
product_name_new = caption.find(class_='product-name').text
product_new.append(product_name_new)
product_name_ref = (soup.select_one("span[class*=reference]").text)
product_ref.append(product_name_ref)
product_size_new = len(product_new)
print("Setup Complete", product_size_new)
print(*product_new,sep='\n')
print(*product_ref,sep='\n')
product_new可以正常工作并返回所有产品名称的列表，但是product_ref只包含REF NO. A1400.5重复的次数，该次数表示该类reference出现在网页上的次数。我如何更改此设置以正确存储reference页面中每次出现的信息？
谢谢！
在您的代码中，product_name_ref始终是相同的值，因为您是从中进行选择soup，而不是从中进行选择caption。
要获取所需的信息，可以使用以下示例：
from bs4 import BeautifulSoup
txt = '''
<div class="product-details">
<h2 class="product-name" title=" Weekly Roundup"> Weekly Roundup</h2>
<span class="reference-number">REF NO. A1400.5</span>
</div>
<div class="product-details">
<h2 class="product-name" title=" Weekly Roundup"> Weekly Roundup 2</h2>
<span class="reference-number">REF NO. A1400.5 2</span>
</div>
'''
soup = BeautifulSoup(txt, 'html.parser')
product_new = []
product_ref = []
for product in soup.select('div.product-details'):
product_new.append(product.h2.get_text(strip=True))
product_ref.append(product.select_one('span.reference-number').get_text(strip=True))
print(product_new)
print(product_ref)
印刷品：
['Weekly Roundup', 'Weekly Roundup 2']
['REF NO. A1400.5', 'REF NO. A1400.5 2']
编辑：
product_new = []
product_ref = []
for product in soup.select('div.product-details'):
n = product.h2
r = product.select_one('span.reference-number')
if n and r:
product_new.append(n.get_text(strip=True))
product_ref.append(r.get_text(strip=True))
print(product_new)
print(product_ref)
编辑2：
from bs4 import BeautifulSoup
txt = '''
<div class="product-details">
<h2 class="product-name" title=" Weekly Roundup"> Weekly Roundup</h2>
<span class="reference-number">REF NO. A1400.5</span>
</div>
<div class="product-details">
<h2 class="product-name" title=" Weekly Roundup"> Weekly Roundup 2</h2>
<span class="reference-number">REF NO. A1400.6</span>
</div>
'''
soup = BeautifulSoup(txt, 'html.parser')
product_new = []
product_ref = []
for product in soup.select('div.product-details'):
n = product.h2
r = product.select_one('span.reference-number')
if n and r:
product_new.append(n.get_text(strip=True))
product_ref.append(r.get_text(strip=True).rsplit(maxsplit=1)[-1])
print(product_new)
print(product_ref)
印刷品：
['Weekly Roundup', 'Weekly Roundup 2']
['A1400.5', 'A1400.6']
编辑3：
for a, b in zip(product_new, product_ref):
print('{:<30} {}'.format(a, b))
印刷品：
Weekly Roundup A1400.5
Weekly Roundup 2 A1400.6
本文收集自互联网，转载请注明来源。
如有侵权，请联系[email protected] 删除。
我来说两句