Skip to content

Simple(single thread, with no framework) crawler script for bilibili animes' information and their reviews using web API.

Notifications You must be signed in to change notification settings

AngelMsger/Bangumi-Provider

Repository files navigation

Bangumi-Provider

Build Status

Bangumi-Crawler

Overview

Bangumi-Provider is a content provider for Bangumi-Visualizer. It crawl, analyze the data from Bilibili, and persist the result to database, using Collaborative Filtering algorithm when analyzing data. It run with single thread and no framework was used.

Features

  • Incremental Crawl & Analyze
  • Multi Storage Backend Support
  • Containerized

Usage

With Docker-Compose

docker-compose up

With Docker

docker run -itd --name=provider --net=host -e DB_PASSWORD=$DB_PASSWORD angelmsger/bangumi-provider

Manually

python -m venv venv && source venv/bin/activate && pip install -r requirements && python exec.py

Todo

  1. MySQL support
  2. Concurrent feature

Quote

  1. Use Kaaass's API when auth after v1.3.x.

About

Simple(single thread, with no framework) crawler script for bilibili animes' information and their reviews using web API.

Resources

Stars

Watchers

Forks

Packages

No packages published