I need to scrape car makes and model. Now I can scrape car makes select option list but can't scrape car models select option list because it is loaded when you select car make.
Maybe you have any ideas how I could get car models select options list which is loaded when you select car make.
Here is my code.
# -*- coding: utf-8 -*-
import requests
from bs4 import BeautifulSoup
url="http://autoplius.lt/redaguoti/naudoti-automobiliai/"
page = requests.get(url)
soup = BeautifulSoup(page.content, "html.parser")
car_makes_select = soup.find("select", {"id": "make_id"})
car_makes = car_makes_select.select("option")
for item in car_makes:
itemMain = item
itemMain = itemMain.get('value')
payload = {
'make_id': itemMain
}
form = requests.post(url, params=payload)
soup11 = BeautifulSoup(form.text, "html.parser")
model_select = soup11.find("select", {"id": "model_id"})
print model_select
Aucun commentaire:
Enregistrer un commentaire