Skip to content

scrapinghub/web-poet

Repository files navigation

web-poet

PyPI Version Supported Python Versions Tox Ubuntu Tox Windows Coverage report Documentation Status

web-poet is a Python 3.9+ implementation of the page object pattern for web scraping. It enables writing portable, reusable web parsing code.

See the documentation.

Developing

Setup your local Python environment via:

  1. pip install -r requirements-dev.txt
  2. pre-commit install

Now everytime you perform a git commit, these tools will run against the staged files:

  • black
  • isort
  • flake8

You can also directly invoke pre-commit run --all-files or tox -e linters to run them without performing a commit.