Skip to content
This repository has been archived by the owner on Oct 6, 2021. It is now read-only.

Headless mode to Selenium scraper. #29

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open

Conversation

raghuramdr
Copy link
Collaborator

Resolves issue #28.

@raghuramdr raghuramdr added the enhancement New feature or request label Apr 28, 2020
@raghuramdr raghuramdr self-assigned this Apr 28, 2020
@raghuramdr raghuramdr linked an issue Apr 28, 2020 that may be closed by this pull request
raghuram-kukun and others added 6 commits April 28, 2020 13:05
1) Fetching `website_baseurl` variable in `ncs_scraper.py` file from `properties.ini` file
2) Updated README to add steps on how to run `ncs_scraper.py` file.
:
Removed typo
Removed commented code which was not required
Changed the name of the argparse flag from -he to -f. -he can be confused for the --help argument.
Used argparse to specify command line arguments that fetches the name(s) of state(s) that we fetch datafor. Updated the README file to reflect these changes.
Added functionality to fetch specific states.
@@ -1,6 +1,16 @@
beautifulsoup4==4.7.1
lxml>=4.3,<5
requests==2.21.0
certifi==2020.4.5.1
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are all these direct dependencies or are they dependencies of packages installed

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dependencies of packages.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Run scraper in headless mode for ncs data
3 participants