The aim of this project is to offer a scalable & flexible data ingest / scraping platform with the lowest barrier to entry, without sacrificing functionality.
NOTE: Assuming you already have Git & Docker installed as prerequisites.
To get started running your webingest stack, simply run the following command:
git clone https://github.com/does-not-compute-au/web-ingest-stack.git && cd web-ingest-stack && docker-compose up -d
Browse to http://localhost:5000
Default login credentials:
[email protected]
Password123