BrowserBro 😎

BrowserBro is a simple HTTP server and a collection of plugins that are useful for web scraping, testing and many other things. With browserBro you can turn any website into an API.

At its core, BrowserBro is using Rod, a high-level Go browser automation library.

Getting started

First, clone the repository and navigate to the project directory.

To run the project you will need to install Docker.

Docker

docker-compose up --build -d

The command above will start two containers: the API server and the browser server. The API server is this project, and the browser server is Rod and is pre-built.

Manually

Alternatively, you can build the project manually and run the browser server separately in any way you want.

go build main.go

Environment variables

You can configure the server by setting the following environment variables:

BROWSERBRO_SERVER_ADDRESS - the address the API server will listen on (default: :10001)

BROWSERBRO_FILE_STORE_BASE_PATH - the directory where the files will be stored on the API server (default: /tmp/browserBro_files)

BROWSERBRO_BROWSER_SERVICE_URL - the address of the browser server (default: ws://localhost:7317)

BROWSERBRO_BROWSER_SERVER_ID - the ID of the browser server. Only necessary if you are running multiple browser instances (default: 1)

BROWSERBRO_BROWSER_MONITOR_ENABLED - enable/disable the browser monitor. Useful for debugging (default: true)

BROWSERBRO_BROWSER_USER_DATA_DIR - the directory where the browser data will be stored on the browser server (default: /tmp/rod/user-data/browserBro_userData)

Plugins ⚙️

Plugins in context of the BrowserBro are automation scripts used to control the browser and perform various tasks. BrowserBro comes with a basic collection of plugins that are maintained by the contributors. If you want to create a plugin yourself to do something specific and/or contribute to the plugins collection - read more here

Plugins are available as HTTP endpoints and can be accessed by sending a POST request to the server. You can supply the plugin with the necessary parameters in the request body.

POST /api/v1/plugins/{plugin-name}

Example

Look how simple it is to scrape google search results with BrowserBro 🔍

curl -X POST -d '{"query":"latest Golang news"}' http://localhost:10001/api/v1/plugins/googlesearch

Response:

{
  "googlesearch": {
    "all": [
      {
        "description": "A weekly newsletter about the Go programming language ...",
        "link": "https://golangweekly.com/",
        "title": "Golang Weekly"
      },
      ...
    ]
  }
}

Available plugins

To get a list of all available plugins:

GET /api/v1/plugins

1. Google search
2. Screenshot

Files 📁

BrowserBro can also serve static files generated or downloaded by the plugins. The files are available at the following URL:

GET /api/v1/files/{fileID}

Files can also be deleted by sending a DELETE request to the same URL.

DELETE /api/v1/files/{fileID}

Example

When you use the screenshot plugin, the plugin will save the screenshots as files and return the file IDs in the response.

{
  "screenshot": {
    "files": [
      "Shu2vLZm.screenshot.png"
    ]
  }
}

You can then access the files by sending a GET request to the file URL.

curl http://localhost:10001/api/v1/files/Shu2vLZm.screenshot.png

Health Check

To check if the server is running, you can send a GET request to the health check endpoint.

GET /api/v1/health

API Clients

• Golang client