我当时正在研究ratemyprofessor，它具有一个“加载更多”按钮以加载更多教授，并且我使用调试器来分析网络，它显示了一个js请求。评价我的教授网站
我当时在想，对于请求URL，有一个开始和一行，所以我每次都将开始增加20个，这样行得通吗？
有人告诉我可以尝试使用formdata，但是在这种情况下，没有formdata，而且它不是POST方法，对吗？
我对scrapy和python真的很陌生，希望你们能给我一些见识。真的很感激
他们不允许我上传图片...但是无论如何
Request URL:https://search-a.akamaihd.net/typeahead/suggest/?solrformat=true&rows=10&callback=noCB&q=*%3A*+AND+schoolid_s%3A1273&defType=edismax&qf=teacherfullname_t%5E1000+autosuggest&bf=pow(total_number_of_ratings_i%2C2.1)&sort=total_number_of_ratings_i+desc&siteName=rmp&rows=20&start=20&fl=pk_id+teacherfirstname_t+teacherlastname_t+total_number_of_ratings_i+averageratingscore_rf+schoolid_s
Request Method:GET
Status Code:200 OK
Remote Address:23.212.53.206:443
查询设置参数
solrformat:true
rows:10
callback:noCB
q:*:* AND schoolid_s:1273
defType:edismax
qf:teacherfullname_t^1000 autosuggest
bf:pow(total_number_of_ratings_i,2.1)
sort:total_number_of_ratings_i desc
siteName:rmp
rows:20
start:20
fl:pk_id teacherfirstname_t teacherlastname_t total_number_of_ratings_iaverageratingscore_rf schoolid_s
是的，您是正确的-不需要formdata，使用GET方法调用就很简单。
使用以下参数，使用以下命令获取json响应：
data = json.loads(response.body)
records = data['response']['docs']
要使用的参数：开始，行
要避免的参数：回调
参数说明：
solrformat:true
#callback:noCB # Drop this parameter, this will give you response data in json format
q:*:* AND schoolid_s:1273
defType:edismax
qf:teacherfullname_t^1000 autosuggest
bf:pow(total_number_of_ratings_i,2.1)
sort:total_number_of_ratings_i desc
siteName:rmp
rows:20
# rows: the amount of records you want in a single response, this will give you 20
# you can try with maximum that can be returned in single response i.e 500, 1000..etc, this will minimize no of request made to website
start:20
#start :you can start this from 0,10,20,30,40 manually as well if you know the total number of records.
fl:pk_id teacherfirstname_t teacherlastname_t total_number_of_ratings_iaverageratingscore_rf schoolid_s
本文收集自互联网，转载请注明来源。
如有侵权，请联系[email protected] 删除。
我来说两句