Skip to content

lucyColeclough/recipe-scrapers

 
 

Repository files navigation

Version Travis Coveralls License Github Black formatted

A simple web scraping tool for recipe sites.

pip install recipe-scrapers

then:

from recipe_scrapers import scrape_me

# give the url as a string, it can be url from any site listed below
scraper = scrape_me('https://www.allrecipes.com/recipe/158968/spinach-and-feta-turkey-burgers/')

scraper.title()
scraper.total_time()
scraper.yields()
scraper.ingredients()
scraper.instructions()
scraper.image()
scraper.links()

Note: scraper.links() returns a list of dictionaries containing all of the <a> tag attributes. The attribute names are the dictionary keys.

Scrapers available for:

Contribute

Part of the reason I want this open sourced is because if a site makes a design change, the scraper for it should be modified.

If you spot a design change (or something else) that makes the scraper unable to work for a given site - please fire an issue asap.

If you are programmer PRs with fixes are warmly welcomed and acknowledged with a virtual beer.

If you want a scraper for a new site added

For Devs / Contribute

Assuming you have python3 installed, navigate to the directory where you want this project to live in and drop these lines

git clone [email protected]:hhursev/recipe-scrapers.git &&
cd recipe-scrapers &&
python3 -m venv .venv &&
source .venv/bin/activate &&
pip install -r requirements.txt &&
pre-commit install &&
python -m coverage run -m unittest &&
python -m coverage report

FAQ

  • How do I know if a website has a Recipe Schema?
    • Go to a recipe on the website you want to be supported.
    • Hit Ctrl - u on your keyboard
    • Search (Ctrl -f) for application/ld+json. It should be inside a script tag.
    • If you found it then it's highly likely your website supports recipe schemas. Otherwise, you'll need to parse the HTML.

Spacial thanks to:

All the contributors that helped improving the package. You are awesome!

About

Python package for scraping recipes data

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Python 100.0%