Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Astra scraper #31

Merged
merged 11 commits into from
Oct 14, 2024
Merged

Astra scraper #31

merged 11 commits into from
Oct 14, 2024

Commits on Sep 25, 2024

  1. Log in to Astra

    TyHil committed Sep 25, 2024
    Configuration menu
    Copy the full SHA
    d6750ab View commit details
    Browse the repository at this point in the history

Commits on Sep 26, 2024

  1. Configuration menu
    Copy the full SHA
    012f0de View commit details
    Browse the repository at this point in the history
  2. Uncomment cookie code

    TyHil committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    bf2503a View commit details
    Browse the repository at this point in the history
  3. Successful scrape!

    TODOs:
    sorting
    scrape each day
    look into login inputting user/pass in wrong sometimes
    TyHil committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    f1cccc9 View commit details
    Browse the repository at this point in the history
  4. Request in loop

    TyHil committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    b9b4bda View commit details
    Browse the repository at this point in the history

Commits on Sep 27, 2024

  1. Scrape until 90 days of less than 10 events

    After this and next semester seems there's only ever 2 events, one in FO 3.616 with no time (?) and one with no location that always shows up after the current semester and says either the holiday and "Events for Future Terms" as well as "No Events Allowed". This just scrapes 90 days into that, stops at about a year and 2 months out.
    TyHil committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    90a97e5 View commit details
    Browse the repository at this point in the history
  2. Sort by start time

    TyHil committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    5ed6907 View commit details
    Browse the repository at this point in the history

Commits on Sep 28, 2024

  1. Check if max events exceeded

    TyHil committed Sep 28, 2024
    Configuration menu
    Copy the full SHA
    12cd58d View commit details
    Browse the repository at this point in the history
  2. Run close commands when not needed

    Closes chromedp when not necessary
    TyHil committed Sep 28, 2024
    Configuration menu
    Copy the full SHA
    6b98eb5 View commit details
    Browse the repository at this point in the history
  3. Start on previous day

    TyHil committed Sep 28, 2024
    Configuration menu
    Copy the full SHA
    e4d2501 View commit details
    Browse the repository at this point in the history

Commits on Oct 14, 2024

  1. Configuration menu
    Copy the full SHA
    3579c4b View commit details
    Browse the repository at this point in the history