Background:
I have a list of nested dictionaries named result Each nested dictionary has a string key (e.g 'diet') and each dictionary value is a unique URL.
Some examples demonstrated below -
[{'diet': 'https://www.simplyrecipes.com/recipes/diet/dairy-free/'},
{'diet': 'https://www.simplyrecipes.com/recipes/diet/gluten-free/'},
{'diet': 'https://www.simplyrecipes.com/recipes/diet/healthy/'},
{'diet': 'https://www.simplyrecipes.com/recipes/diet/low_carb/'},
{'diet': 'https://www.simplyrecipes.com/recipes/diet/paleo/'},
{'diet': 'https://www.simplyrecipes.com/recipes/diet/vegan/'},
{'diet': 'https://www.simplyrecipes.com/recipes/diet/vegetarian/'},
{'main-ingredient': 'https://www.simplyrecipes.com/recipes/main-ingredient/beef/'},
{'main-ingredient': 'https://www.simplyrecipes.com/recipes/main-ingredient/cheese/'},
{'main-ingredient': 'https://www.simplyrecipes.com/recipes/main-ingredient/chicken/'},
{'main-ingredient': 'https://www.simplyrecipes.com/recipes/main-ingredient/egg/'},
{'main-ingredient': 'https://www.simplyrecipes.com/recipes/main-ingredient/fish/'},
{'main-ingredient': 'https://www.simplyrecipes.com/recipes/main-ingredient/fish_and_seafood/'}]
I'm calling this out, just to give some context as to how I'll be using the keys and values: I am going to be writing a for loop, that will iterate through all the values / URLs and execute this code, that basically extracts the ingredients from each recipe's (URL) web page:
from splinter import Browser
from webdriver_manager.chrome import ChromeDriverManager
resp = requests.get("https://www.simplyrecipes.com/recipes/egg_salad_sandwich/")
soup = BeautifulSoup(resp.text, "html.parser")
div_ = soup.find("div", attrs={"class": "recipe-callout"})
recipes = {"_".join(div_.find("h2").text.split()):
[x.text for x in div_.findAll("li", attrs={"class": "ingredient"})]}
executable_path = {'executable_path': ChromeDriverManager().install()} browser = Browser('chrome', **executable_path)
Objective:
I would like to merge the dictionaries by key value. The below examples shows the structure I'm hoping for -
[{'diet': 'https://www.simplyrecipes.com/recipes/diet/allergy-friendly/',
'https://www.simplyrecipes.com/recipes/diet/dairy-free/',
'https://www.simplyrecipes.com/recipes/diet/vegetarian/',
{'main-ingredient': 'https://www.simplyrecipes.com/recipes/main-ingredient/beef/'
'https://www.simplyrecipes.com/recipes/main-ingredient/lamb/',
'https://www.simplyrecipes.com/recipes/main-ingredient/chicken/'}]
My code:
So far I have code as follows, however I've completely screwed it up; it does nothing useful and I don't know what part of my head it came from!
master_dict = NestedDict(result)
for i in d:
path = [i['diet'], i['ingredient']]
master_dict[path] = i['https:']
When you analyse the page, the link - https://www.simplyrecipes.com/recipes gives all the links to different types of recipes. So, scraping that page and formatting correctly will give you the desired result.
import requests
from bs4 import BeautifulSoup
import pprint
res = requests.get("https://www.simplyrecipes.com/recipes")
soup = BeautifulSoup(res.text,"html.parser")
links = {}
for div in soup.find("div", class_="rnav-menus").find_all("div", class_="rnav-menu"):
recipe_type = div.find("span").get_text(strip=True)
links[recipe_type] = [i.find("a")["href"] for i in div.find_all("li")]
pprint.pprint(links)
Output:
{'Course': ['https://www.simplyrecipes.com/recipes/course/appetizer/',
'https://www.simplyrecipes.com/recipes/course/breakfast/',
'https://www.simplyrecipes.com/recipes/course/brunch/',
'https://www.simplyrecipes.com/recipes/course/dessert/',
'https://www.simplyrecipes.com/recipes/course/dinner/',
'https://www.simplyrecipes.com/recipes/course/drink/',
'https://www.simplyrecipes.com/recipes/course/lunch/',
'https://www.simplyrecipes.com/recipes/course/salad/',
'https://www.simplyrecipes.com/recipes/course/sandwich/',
'https://www.simplyrecipes.com/recipes/course/side_dish/',
'https://www.simplyrecipes.com/recipes/course/snack/',
'https://www.simplyrecipes.com/recipes/course/soup/',
'https://www.simplyrecipes.com/recipes/course/soup_and_stew/',
'https://www.simplyrecipes.com/recipes/course/stew/'],
'Cuisine': ['https://www.simplyrecipes.com/recipes/cuisine/african/',
'https://www.simplyrecipes.com/recipes/cuisine/basque/',
'https://www.simplyrecipes.com/recipes/cuisine/belgian/',
'https://www.simplyrecipes.com/recipes/cuisine/brazilian/',
'https://www.simplyrecipes.com/recipes/cuisine/british/',
'https://www.simplyrecipes.com/recipes/cuisine/cajun/',
'https://www.simplyrecipes.com/recipes/cuisine/cambodian/',
'https://www.simplyrecipes.com/recipes/cuisine/chinese/',
'https://www.simplyrecipes.com/recipes/cuisine/cowboy/',
'https://www.simplyrecipes.com/recipes/cuisine/creole/',
'https://www.simplyrecipes.com/recipes/cuisine/danish/',
'https://www.simplyrecipes.com/recipes/cuisine/ethiopian/',
'https://www.simplyrecipes.com/recipes/cuisine/french/',
'https://www.simplyrecipes.com/recipes/cuisine/german/',
'https://www.simplyrecipes.com/recipes/cuisine/greek/',
'https://www.simplyrecipes.com/recipes/cuisine/hawaiian/',
'https://www.simplyrecipes.com/recipes/cuisine/hungarian/',
'https://www.simplyrecipes.com/recipes/cuisine/indian/',
'https://www.simplyrecipes.com/recipes/cuisine/irish/',
'https://www.simplyrecipes.com/recipes/cuisine/italian/',
'https://www.simplyrecipes.com/recipes/cuisine/jamaican/',
'https://www.simplyrecipes.com/recipes/cuisine/japanese/',
'https://www.simplyrecipes.com/recipes/cuisine/jewish/',
'https://www.simplyrecipes.com/recipes/cuisine/korean/',
'https://www.simplyrecipes.com/recipes/cuisine/latin-american/',
'https://www.simplyrecipes.com/recipes/cuisine/mediterranean/',
'https://www.simplyrecipes.com/recipes/cuisine/mexican/',
'https://www.simplyrecipes.com/recipes/cuisine/mexican_and_tex_mex/',
'https://www.simplyrecipes.com/recipes/cuisine/middle-eastern/',
'https://www.simplyrecipes.com/recipes/cuisine/moroccan/',
'https://www.simplyrecipes.com/recipes/cuisine/new_england/',
'https://www.simplyrecipes.com/recipes/cuisine/new_orleans/',
'https://www.simplyrecipes.com/recipes/cuisine/persian/',
'https://www.simplyrecipes.com/recipes/cuisine/polish/',
'https://www.simplyrecipes.com/recipes/cuisine/portuguese/',
'https://www.simplyrecipes.com/recipes/cuisine/provencal/',
'https://www.simplyrecipes.com/recipes/cuisine/puerto-rican/',
'https://www.simplyrecipes.com/recipes/cuisine/southern/',
'https://www.simplyrecipes.com/recipes/cuisine/southwestern/',
'https://www.simplyrecipes.com/recipes/cuisine/spanish/',
'https://www.simplyrecipes.com/recipes/cuisine/swedish/',
'https://www.simplyrecipes.com/recipes/cuisine/texmex/',
'https://www.simplyrecipes.com/recipes/cuisine/thai/',
'https://www.simplyrecipes.com/recipes/cuisine/vietnamese/'],
'Featured': ['https://www.simplyrecipes.com/hub/grill_recipes/',
'https://www.simplyrecipes.com/hub/best_copycat_recipes_restaurant_favorites/',
'https://www.simplyrecipes.com/hub/cookbook_club/',
'https://www.simplyrecipes.com/category/meal-plans/',
'https://www.simplyrecipes.com/category/eat-your-food/',
'https://www.simplyrecipes.com/category/cooking-for-two/',
'https://www.simplyrecipes.com/category/use-it-up/',
'https://www.simplyrecipes.com/category/editors-picks/',
'https://www.simplyrecipes.com/category/pantry-power/',
'https://www.simplyrecipes.com/category/produce-guides/',
'https://www.simplyrecipes.com/category/equipment-guides/'],
'Ingredient': ['https://www.simplyrecipes.com/recipes/main-ingredient/beef/',
'https://www.simplyrecipes.com/recipes/main-ingredient/cheese/',
'https://www.simplyrecipes.com/recipes/main-ingredient/chicken/',
'https://www.simplyrecipes.com/recipes/main-ingredient/egg/',
'https://www.simplyrecipes.com/recipes/main-ingredient/fish/',
'https://www.simplyrecipes.com/recipes/main-ingredient/fish_and_seafood/',
'https://www.simplyrecipes.com/recipes/main-ingredient/fruit/',
'https://www.simplyrecipes.com/recipes/main-ingredient/lamb/',
'https://www.simplyrecipes.com/recipes/main-ingredient/pasta/',
'https://www.simplyrecipes.com/recipes/main-ingredient/pork/',
'https://www.simplyrecipes.com/recipes/main-ingredient/rice/',
'https://www.simplyrecipes.com/recipes/main-ingredient/seafood/',
'https://www.simplyrecipes.com/recipes/main-ingredient/turkey/',
'https://www.simplyrecipes.com/recipes/main-ingredient/vegetables/'],
'Season': ['https://www.simplyrecipes.com/recipes/season/birthday/',
'https://www.simplyrecipes.com/recipes/season/christmas/',
'https://www.simplyrecipes.com/recipes/season/easter/',
'https://www.simplyrecipes.com/recipes/season/fathers-day/',
'https://www.simplyrecipes.com/recipes/season/seasonal_favorites_fall/',
'https://www.simplyrecipes.com/recipes/season/seasonal_favorites_spring/',
'https://www.simplyrecipes.com/recipes/season/seasonal_favorites_summer/',
'https://www.simplyrecipes.com/recipes/season/seasonal_favorites_winter/',
'https://www.simplyrecipes.com/recipes/season/fourth-of-july/',
'https://www.simplyrecipes.com/recipes/season/game-day/',
'https://www.simplyrecipes.com/recipes/season/halloween/',
'https://www.simplyrecipes.com/recipes/season/hanukkah/',
'https://www.simplyrecipes.com/recipes/season/holiday/',
'https://www.simplyrecipes.com/recipes/season/lent/',
'https://www.simplyrecipes.com/recipes/season/mardi-gras/',
'https://www.simplyrecipes.com/recipes/season/mothers_day/',
'https://www.simplyrecipes.com/recipes/season/new-years-day/',
'https://www.simplyrecipes.com/recipes/season/passover/',
'https://www.simplyrecipes.com/recipes/season/st_patricks_day/',
'https://www.simplyrecipes.com/recipes/season/super_bowl/',
'https://www.simplyrecipes.com/recipes/season/thanksgiving/',
'https://www.simplyrecipes.com/recipes/season/valentines_day/'],
'Special Diets': ['https://www.simplyrecipes.com/recipes/diet/allergy-friendly/',
'https://www.simplyrecipes.com/recipes/diet/dairy-free/',
'https://www.simplyrecipes.com/recipes/diet/gluten-free/',
'https://www.simplyrecipes.com/recipes/diet/healthy/',
'https://www.simplyrecipes.com/recipes/diet/low_carb/',
'https://www.simplyrecipes.com/recipes/diet/paleo/',
'https://www.simplyrecipes.com/recipes/diet/vegan/',
'https://www.simplyrecipes.com/recipes/diet/vegetarian/'],
'Type': ['https://www.simplyrecipes.com/recipes/type/1-pot/',
'https://www.simplyrecipes.com/recipes/type/air-fryer/',
'https://www.simplyrecipes.com/recipes/type/bbq/',
'https://www.simplyrecipes.com/recipes/type/baking/',
'https://www.simplyrecipes.com/recipes/type/budget/',
'https://www.simplyrecipes.com/recipes/type/candy/',
'https://www.simplyrecipes.com/recipes/type/canning/',
'https://www.simplyrecipes.com/recipes/type/casserole/',
'https://www.simplyrecipes.com/recipes/type/comfort_food/',
'https://www.simplyrecipes.com/recipes/type/condiment/',
'https://www.simplyrecipes.com/recipes/type/cookie/',
'https://www.simplyrecipes.com/recipes/type/deep_fried/',
'https://www.simplyrecipes.com/recipes/type/dip/',
'https://www.simplyrecipes.com/recipes/type/freezer-friendly/',
'https://www.simplyrecipes.com/recipes/type/grill/',
'https://www.simplyrecipes.com/recipes/type/how_to/',
'https://www.simplyrecipes.com/recipes/type/instant-pot/',
'https://www.simplyrecipes.com/recipes/type/jams_and_jellies/',
'https://www.simplyrecipes.com/recipes/type/kidfriendly/',
'https://www.simplyrecipes.com/recipes/type/make-ahead/',
'https://www.simplyrecipes.com/recipes/type/microwave/',
'https://www.simplyrecipes.com/recipes/type/pantry-meal/',
'https://www.simplyrecipes.com/recipes/type/pressure-cooker/',
'https://www.simplyrecipes.com/recipes/type/quick/',
'https://www.simplyrecipes.com/recipes/type/restaurant_favorite/',
'https://www.simplyrecipes.com/recipes/type/salsa/',
'https://www.simplyrecipes.com/recipes/type/sauce/',
'https://www.simplyrecipes.com/recipes/type/sheet-pan-dinner/',
'https://www.simplyrecipes.com/recipes/type/skillet-recipe/',
'https://www.simplyrecipes.com/recipes/type/slow_cooker/',
'https://www.simplyrecipes.com/recipes/type/sous-vide/',
'https://www.simplyrecipes.com/recipes/type/stirfry/']}
この記事はインターネットから収集されたものであり、転載の際にはソースを示してください。
侵害の場合は、連絡してください[email protected]
コメントを追加