Im trying to scrape the response to this website using a pre-filled zip: zip who (i.e. the zip code is already filled in.) I tried to do this using the scrapy shell as follows
scrapy shell http://zipwho.com/?zip=77098&mode=zip
but the response does not contain the form-filled page, but only the content from the main zipwho.com page and without the details specific to that zip code. I tried filling in the form information using requests and lxml, but clearly I am doing something wrong.
import requests
import lxml.html as lh
url = 'http://zipwho.com'
form_data = {
'zip': '77098'
}
response = requests.post(url, data=form_data)
tree = lh.document_fromstring(response.content)
tree.xpath('//td[@class="keysplit"]')
and the table element for the data (td where class = 'keysplit') still does not exist. If you have ideas to get this working (hopefully with something simple in like requests & lxml) that would be best.
Aucun commentaire:
Enregistrer un commentaire