I have used net/http package to get url html response, it works fine until now.
but when I get one page that use Ajax to fill some html element, I could not get all contents of html page.
How can use http.Get that will wait page load completely then get whole web page contents. Thanks!
response, err := http.Get(url)
if err != nil {
fmt.Printf("%s", err)
os.Exit(1)
} else {
defer response.Body.Close()
contents, err := ioutil.ReadAll(response.Body)
if err != nil {
fmt.Printf("%s", err)
os.Exit(1)
}
fmt.Printf("%s\n", string(contents))
}
Content generated via javascript isn't part of the HTML that the server is sending out. It needs to be evaluated client-side, and since your client in the case is a Go program you will need to do that evaluation yourself.
A library like Otto may help. However, depending on your needs, it may even be better to re-evaluate your tools and make use of a headless "browser" like PhantomJS or similar.
Collected from the Internet
Please contact [email protected] to delete if infringement.
Comments