Overview API Overview

You can drive Overview's document handling engine from your own code, create new visualizations that replace Overview's default Topic Tree, or write document handling or data extraction apps.

You can put whatever content you want into the Overview website.

Architecture

Here's a diagram of a simple script:

Here's what it means:

• Overview is a web service that stores documents.
• Your script is a program that you write (possibly from scratch). Use whatever programming language you like.
• Your script can read and write information on Overview using API methods.
• Overview does not control or restrict your script. You may store as much data as you wish in Overview's databases; you may use as many or as few API calls as you wish.

Your script can read and search the text of all documents in the document set, as well as document tags.

In general, Overview handles all the boring parts: importing documents, displaying them, search, persistent data, etc. All you have to write is the text processing and interactive visualization.

What an app looks like

Here's an example simple Node app, written in JavaScript. It searches for documents containing the word "word" and then tags them.

var documentSetId = 1; // REPLACE ME
var apiToken = 'api-token'; // REPLACE ME
var search = 'word'; // REPLACE ME
var request = require('request');

function path(end) {
  return 'https://www.overviewproject.org/api/v1/document-sets/' + documentSetId + end;
}

var req = request.defaults({
  auth: { user: apiToken, pass: 'x-auth-token' }
});

function GET(path, done) {
  req.get(path(path), done);
}

function POST(path, json, done) {
  req.post({ url: path(path), method: 'post', json: json }, done);
}

console.log('Fetching list of document IDs containing search string', search);
GET('/documents?fields=id&q=' + encodeURIComponent(search), function(error, response, ids) {
  if (error) throw error;

  if (ids.length == 0) {
    console.log('No matches. Exiting.');
  } else {
    console.log('Number of matches:', ids.length);

    console.log('Creating tag');
    POST('/tags', { title: "App search results: " + search }, function(error, response, tag) {
      if (error) throw error;

      console.log('Tagging documents');
      var idPairs = ids.map(function(id) { return [ id, tag.id ] });
      POST('/document-tags', idPairs, function(error, response, body) {
        console.log('Tagged ' + body.length + ' documents');
      }));
    }));
  }
}));

Custom Views

A standalone app is missing a user interface. You can make your standalone app into a full-fledged interactive application, but you'll probably end up re-implementing search, tags, document lists, and so on. You'll probably be happier coexisting with Overview, as in the picture above. Here's a diagram of how the pieces fit together:

You can create custom Views by building a Plugin.

Custom Views can set a selection (a set of documents) and respond to selection changes elsewhere in Overview. For example, a user could select all documents within a certain geographic region using a map view, then see where these documents fall in time by viewing that same selection in a timeline view.

Here's the difference between a script and a Plugin:

• Your Plugin is a web server
• Overview includes your Plugin in an <iframe> element. The user activates your plugin by creating a new View that points to your Plugin.
• The <iframe> calls your Plugin with server, documentSetId and apiToken query parameters, by opening a URL such as https://your.app/show?server=https%3a%2f%2fwww.overviewproject.org&apiToken=api-token&documentSetId=123
• Your Plugin still reads and writes using the Overview API
• The View iframe (which you rendered) and the main page (which Overview rendered) communicate using Window.postMessage
• The View can alter the "selection", which is the set of parameters that decides what's in the document list. The Overview API server may query your App to translate the "selection" to a list of document IDs.

View calls to Overview:

• get document text and other attributes
• search document set, returning list of matching documents
• write/read persistent objects (use this to store state or preprocessed data)
• set selection (a selection is a list of documents)

Overview calls to View

• render HTML/JS in an iframe
• respond to selection change

Whoa, isn't a whole web server a lot of work?

We're talking a minimal web server:

• You don't need to store anything. Overview can handle storage for you.
• You only need to handle two routes: /metadata and /show.
• You don't need server-side logic. /metadata and /show can be flat files.
• You don't need any background processes.

Start by hosting flat files somewhere. You can code everything in JavaScript. Then if you want to add advanced features such as selections, add server-side logic.

On the other hand, you may prefer to code background processes and lots of routes. If so, go for it!

Calling the API

We encourage you to get started. Browse our API documentation to get excited about what you can do.

Next steps

• Developing your first Overview plugin
• Writing a plugin