CabinetJS is a CLI tool that helps you crawl and archive content across different types of web services. with the configuration file, you can define the crawling rules about which web services to archive, crawling interval, and filtering for selecting the content to archive.
See CabinetJS Documentation for more information.