Skip to content

Commit

Permalink
Update readme
Browse files Browse the repository at this point in the history
  • Loading branch information
bporter816 committed Aug 12, 2023
1 parent ab4f6cd commit 7ceb947
Showing 1 changed file with 35 additions and 1 deletion.
36 changes: 35 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1 +1,35 @@
# issilksongoutyet.com
# issilksongoutyet.com

A website that tracks the release date of [Hollow Knight: Silksong](https://store.steampowered.com/app/1030300/Hollow_Knight_Silksong/).

To get the site to update dynamically, I looked into options for how to fetch data from Steam.
Steam's website hits an underlying API that can be queried directly:
```sh
$ curl 'https://store.steampowered.com/api/appdetails?appids=1030300' | jq '.["1030300"].data.release_date'
{
"coming_soon": true,
"date": "To be announced"
}
```

My first thought was to fetch this data client-side in the browser, but this didn't work due to CORS (the API doesn't return an `Access-Control-Allow-Origin` header).

Next, I tried to fetch the data in a Cloudflare Worker, still processing it client-side. When I started making the request in the Worker, I began getting 403 responses back.
I suspected the Steam API was blocking Cloudflare Workers, and confirmed this by making a test request with one of the headers that Cloudflare injects into outbound requests from Workers:
```sh
$ curl -I 'https://store.steampowered.com/api/appdetails?appids=1030300'
HTTP/1.1 200 OK
...

$ curl -I -H 'cf-worker: blah' 'https://store.steampowered.com/api/appdetails?appids=1030300'
HTTP/1.1 403 Forbidden
...
```

OK, looks like Steam doesn't want Cloudflare Workers to scrape their API. Fair enough. Not wanting to give up so easily, I rewrote the Worker script in an AWS Lambda function.
This worked, but still had a couple of undesirable characteristics:
* Data fetching on page load takes time, and there is a noticable lag when the page loads before the content appears.
* The Lambda function is public, so I can rack up costs from bots hitting it, etc. It's just not very elegant.

To overcome these issues, I changed the site to build hourly, producing a static site with the data from the Steam API injected at build time.
The finished site is built with Jekyll on GitHub Pages and updates via a scheduled GitHub Actions workflow.

0 comments on commit 7ceb947

Please sign in to comment.