如何抓取由按钮锁定的数据？

使用request.Session对象非常简单，您只需__RequestVerificationToken要从初始页面和几张表单数据中提取令牌即可。我使用了完整列表页面来获取电话号码和指向医生页面的链接，无论您决定从何处获取电话号码，都适用相同的逻辑：

from bs4 import BeautifulSoup
import requests
from urlparse import urljoin

head = {
    "User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36",
    "X-Requested-With": "XMLHttpRequest"}

base = "http://www.doctoralia.com.br/"

with requests.Session() as s:
    r = s.get('http://www.doctoralia.com.br/medicos/especialidade/dermatologistas-1314')
    page = BeautifulSoup(r.content, "html.parser")
    token = page.select_one("input[name=__RequestVerificationToken]"["value"])
    hidden_tags = page.select("article.media.doctor")
    for tag in hidden_tags:
        h3 = tag.select_one("h3")
        key = h3.a["data-track-click"]
        place = tag.select_one("span[data-location]")["data-location"].split("|", 1)[0]

       data = {"__RequestVerificationToken": token,
            "entityKey": key,
            "placeKey": place}
        resp = s.post("http://www.doctoralia.com.br/noindex/doctor-phone", data=data, headers=head)
        soup = BeautifulSoup(resp.content,"html.parser")
        print(urljoin(base,h3.a["href"]))
        print(soup.select_one("li.phone").text.strip())

这样就可以获取每个医生的所有链接和电话号码，单击该按钮时在弹出窗口中看到的所有内容都可以解析。必不可少的表单数据是__RequestVerificationToken和entityKey，placeKey似乎并不会影响帖子，但包括在内不会造成任何伤害。在这种情况下，标头也不是必不可少的，但是添加用户代理始终是一个好主意。您可能希望在请求之间添加睡眠，以便在发出大量请求时不要锤打服务器。还要查看robots.txt：

User-agent: *
Disallow: /noindex/
Disallow: /usuarios/
Disallow: /users/
Disallow: /utilisateurs/
Disallow: /utenti/
Disallow: /gebruikers/
Disallow: /nutzer/
Disallow: /medical-center/m/
Disallow: /consultant/m/
Disallow: /centro-medico/m/
Disallow: /medico/m/
Disallow: /centre-medical/m/
Disallow: /medicin/m/
Disallow: /centro-medico/m/
Disallow: /medico/m/
Disallow: /centri-medici/m/
Disallow: /medecin/m/
Disallow: /healthpro/m/
Disallow: /facharzt/m/
Disallow: /sanit�tszentrum/m/
Disallow: /clickfav/
Disallow: /clicktlf/
Disallow: /reservas/
Disallow: /citas/
Disallow: /medisch-centrum/m/
Disallow: /deskundige/m/
Disallow: /arzt/m/
Disallow: /klinik/m/
Disallow: /citas/
Disallow: /turnos/
Disallow: /appuntamenti/
Disallow: /appointments/
Disallow: /consultas/
Disallow: /ws/Schedules.asmx/
Disallow: /RESOURCE NOT FOUND/
Disallow: /RESOURCE+NOT+FOUND/
Disallow: /RESOURCE%20NOT%20FOUND/
Disallow: /entities/

没有用户代理限制，并且禁止您抓取任何内容