Skip to content

Commit

Permalink
Expose HTTP endpoint to trigger a poll (#48)
Browse files Browse the repository at this point in the history
* expose HTTP endpoint to trigger a poll

* add Procfile

* update CI & CD configs

* add basic auth

* add warning logs

* update readme & improve logging
  • Loading branch information
utkuufuk authored Mar 26, 2022
1 parent b7a623c commit 3454797
Show file tree
Hide file tree
Showing 12 changed files with 180 additions and 90 deletions.
7 changes: 5 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,11 @@ jobs:
- name: Make sure that go.mod has already been tidied
run: go mod tidy && git diff --no-patch --exit-code

- name: Build
run: go build ./cmd/entrello
- name: Build Runner
run: go build ./cmd/runner

- name: Build Server
run: go build ./cmd/server

- name: Run tests
run: go test -covermode=count -coverprofile=profile.cov ./...
Expand Down
24 changes: 20 additions & 4 deletions .goreleaser.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,24 @@ before:
- go mod download

builds:
- id: entrello
main: ./cmd/entrello
binary: entrello
- id: entrello-runner
main: ./cmd/runner
binary: entrello-runner
env:
- CGO_ENABLED=0
goos:
- linux
- darwin
- windows
goarch:
- amd64
- arm
- arm64
goarm: [6, 7]

- id: entrello-server
main: ./cmd/server
binary: entrello-server
env:
- CGO_ENABLED=0
goos:
Expand All @@ -28,7 +43,8 @@ archives:
{{- end -}}
{{- end -}}
builds:
- entrello
- entrello-runner
- entrello-server
replacements:
386: i386
amd64: x86_64
Expand Down
1 change: 1 addition & 0 deletions Procfile
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
web: bin/server
34 changes: 20 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,23 +4,35 @@
[![Go Report Card](https://goreportcard.com/badge/github.com/utkuufuk/entrello)](https://goreportcard.com/report/github.com/utkuufuk/entrello)
[![Coverage Status](https://coveralls.io/repos/github/utkuufuk/entrello/badge.svg)](https://coveralls.io/github/utkuufuk/entrello)

Polls compatible data sources and keeps your Trello cards synchronized with fresh data. Meant to be run as a scheduled job.
Polls compatible data sources and keeps your Trello cards synchronized with fresh data.

Let's say you have an HTTP endpoint that returns GitHub issues assigned to you upon a `GET` request.
You can point `entrello` to that endpoint to keep your GitHub issues synchronized in your Trello board.

Each data source must return a JSON array of Trello card objects upon a `GET` request. You can import and use the `NewCard` function from `pkg/trello/trello.go` in order to construct Trello card objects.

- Can be run as a scheduled job:
```sh
go run ./cmd/runner
```
- Can be run as an HTTP server:
```sh
PORT=<port> USERNAME=<user> PASSWORD=<password> go run ./cmd/server
```

In this case, the runner can be triggered by a `POST` request to the server like this:
```sh
curl -d @config.json <SERVER_URL> -H "Authorization: Basic <base64(<user>:<password>)>"
```

## Configuration
Copy and rename `config.example.json` as `config.json` (default), then set your own values in `config.json`.

You can also use a custom config file path using the `-c` flag:
```sh
go run ./cmd/entrello -c /path/to/config/file
go run ./cmd/runner -c /path/to/config/file
```

Alternatively, you can store the configuration inside the `ENTRELLO_CONFIG` environment variable as a JSON string.

### Trello
You need to set your [Trello API key & token](https://trello.com/app-key) in the configuraiton file, as well as the Trello board ID.

Expand Down Expand Up @@ -65,15 +77,9 @@ For each data source, the following parameters have to be specified. (See `confi
```

## Example Cron Job
Make sure that the cron job runs frequently enough to keep up with the most frequent custom interval in your configuration.

For instance, it wouldn't make sense to define a custom period of 15 minutes while the cron job only runs every hour.

Both of the following jobs run every hour and both assume that `config.json` is located in the current working directory, or its contents are stored within the `ENTRELLO_CONFIG` environment variable.
Assuming `config.json` is located in the current working directory:
``` sh
# using "go run"
0 * * * * cd /home/you/git/entrello && /usr/local/go/bin/go run ./cmd/entrello

# use binary executable (see releases: https://github.com/you/entrello/releases)
0 * * * * cd /path/to/binary && ./entrello
0 * * * * cd /home/you/git/entrello && /usr/local/go/bin/go run ./cmd/runner
```

Make sure that the cron job runs frequently enough to keep up with the most frequent custom interval in your configuration. For instance, it wouldn't make sense to define a custom period of 15 minutes while the cron job only runs every hour.
53 changes: 0 additions & 53 deletions cmd/entrello/main.go

This file was deleted.

25 changes: 25 additions & 0 deletions cmd/runner/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
package main

import (
"flag"
"log"

"github.com/utkuufuk/entrello/internal/config"
"github.com/utkuufuk/entrello/internal/logger"
"github.com/utkuufuk/entrello/internal/service"
)

func main() {
var configFile string
flag.StringVar(&configFile, "c", "config.json", "config file path")
flag.Parse()

cfg, err := config.ReadConfig(configFile)
if err != nil {
log.Fatalf("Could not read configuration: %v", err)
}

if err = service.Poll(cfg); err != nil {
logger.Error(err.Error())
}
}
65 changes: 65 additions & 0 deletions cmd/server/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
package main

import (
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"os"

"github.com/utkuufuk/entrello/internal/config"
"github.com/utkuufuk/entrello/internal/logger"
"github.com/utkuufuk/entrello/internal/service"
)

func main() {
http.HandleFunc("/", handler)
http.ListenAndServe(fmt.Sprintf(":%s", os.Getenv("PORT")), nil)
}

func handler(w http.ResponseWriter, req *http.Request) {
if req.Method != http.MethodPost {
logger.Warn("Method %s not allowed", req.Method)
w.WriteHeader(http.StatusMethodNotAllowed)
return
}

user, pwd, ok := req.BasicAuth()
if !ok {
logger.Warn("Could not parse basic auth.")
w.WriteHeader(http.StatusUnauthorized)
return
}
if user != os.Getenv("USERNAME") {
logger.Warn("Invalid user name: %s", user)
w.WriteHeader(http.StatusUnauthorized)
return
}
if pwd != os.Getenv("PASSWORD") {
logger.Warn("Invalid password: %s", pwd)
w.WriteHeader(http.StatusUnauthorized)
return
}

body, err := ioutil.ReadAll(req.Body)
if err != nil {
logger.Error("Could not read request body: %v", err)
w.WriteHeader(http.StatusInternalServerError)
return
}

var cfg config.Config
if err = json.Unmarshal(body, &cfg); err != nil {
logger.Warn("Invalid request body: %v", err)
w.WriteHeader(http.StatusBadRequest)
return
}

if err = service.Poll(cfg); err != nil {
logger.Error(err.Error())
w.WriteHeader(http.StatusInternalServerError)
return
}

w.WriteHeader(http.StatusOK)
}
3 changes: 2 additions & 1 deletion go.mod
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
module github.com/utkuufuk/entrello

go 1.13
// +heroku goVersion go1.16
go 1.16

require (
github.com/adlio/trello v1.8.0
Expand Down
14 changes: 0 additions & 14 deletions internal/config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,6 @@ import (
"encoding/json"
"fmt"
"os"

"github.com/utkuufuk/entrello/internal/logger"
)

const (
Expand Down Expand Up @@ -42,18 +40,6 @@ type Config struct {
}

func ReadConfig(fileName string) (cfg Config, err error) {
jsonStr := os.Getenv("ENTRELLO_CONFIG")
if jsonStr != "" {
logger.Info("Attempting to read configuration from environment variable 'ENTRELLO_CONFIG'")
err = json.Unmarshal([]byte(jsonStr), &cfg)
if err != nil {
return cfg, fmt.Errorf("could not parse config stored in the environment variable: %s", err)
}

return cfg, nil
}

logger.Info("Attempting to read configuration from file '%s'", fileName)
f, err := os.Open(fileName)
if err != nil {
return cfg, fmt.Errorf("could not open config file: %v", err)
Expand Down
40 changes: 40 additions & 0 deletions internal/service/poller.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
package service

import (
"fmt"
"sync"
"time"

"github.com/utkuufuk/entrello/internal/config"
"github.com/utkuufuk/entrello/pkg/trello"
)

func Poll(cfg config.Config) error {
loc, err := time.LoadLocation(cfg.TimezoneLocation)
if err != nil {
return fmt.Errorf("invalid timezone location: %v", loc)
}

sources, labels := getSources(cfg.Sources, time.Now().In(loc))
if len(sources) == 0 {
return nil
}

client, err := trello.NewClient(cfg.Trello)
if err != nil {
return fmt.Errorf("could not create trello client: %v", err)
}

if err := client.LoadBoard(labels); err != nil {
return fmt.Errorf("Could not load existing cards from the board: %v", err)
}

var wg sync.WaitGroup
wg.Add(len(sources))
for _, src := range sources {
go process(src, client, &wg)
}
wg.Wait()

return nil
}
2 changes: 1 addition & 1 deletion cmd/entrello/source.go → internal/service/source.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package main
package service

import (
"encoding/json"
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package main
package service

import (
"fmt"
Expand Down

0 comments on commit 3454797

Please sign in to comment.