Skip to content
This repository has been archived by the owner on Nov 30, 2017. It is now read-only.

Proposal #5: HTTP REST interface for accessing consumer info

Jump to bottom
Emanuele Tajariol edited this page Aug 30, 2013 · 5 revisions

Use case

We need an entry point in GeoBatch to programatically find out if a given consumer instance has completed, has failed, and, if still executing, which Action is performing.

Operations

The new interface could be implemented this way:

Run a new consumer

This operation would be the same as putting a new file in a watch directory of flow flowid. The file content will be sent as payload of the HTTP request.

  • POST geobatch/rest/flow/{flowid}/run
  • Payload: the input content for the consumer

The response will contain the UUID of the consumer that is dealing with the request.

The call will be blocked until a thread in the consumer threadpool will be available.

We may want to add a query param fastfail=true in order to return immediately with an error condition if no thread is available at the moment of the request.

The HTTP request will need basic authentication, with credentials of a user allowed to access the request flow.

Return codes

  • 202 accepted
  • 403 forbidden

Run a new consumer with local file

This operation is similar to the previous one, but instead of transmitting the data, the payload provides a list of filepaths that will be then passed to the consumer.

Since the action processing may alter or delete the passed files, the files will be copied into the flowinstance temp dir; the processing will be then performed on the copied files.

NOTE: since this operation may request access to any system file (think about /etc/passwd) we may want to implement some sort of black/whitelisting for the input files.

  • POST geobatch/rest/flow/{flowid}/runlocal
  • Payload: the list of the local files to process in the format
    <runInfo>
       <file>file01</file>
       <file>file02</file>
       ...
    </runInfo>

The response will contain the UUID of the consumer that is dealing with the request.

The call will be blocked until a thread in the consumer threadpool will be available.

We may want to add a query param fastfail=true in order to return immediately with an error condition if no thread is available at the moment of the request.

The HTTP request will need basic authentication, with credentials of a user allowed to access the request flow.

Return codes

  • 202 accepted
  • 403 forbidden

Get status info

Get the status for a given consumer id.

  • GET geobatch/rest/consumer/{consumerid}/status

Authenticated request.

Responses can be:

  • 200 OK: see below for the format of the responses
  • 404 not found : the consumer id does not exist OR the consumerId belongs to a flow not accessible to the authenticated user

Status responses

Success
   <consumer status="success">
      <uuid>UUID</uuid>
      <startdate>yyyy-mm-dd hh:mm:ss.zzz</startdate>
   </consumer>
Fail
   <consumer status="fail">
      <uuid>UUID</uuid>
      <startdate>yyyy-mm-dd hh:mm:ss.zzz</startdate>

      <message>error message</message>
      <!-- action in which the error occurred -->
      <action> 
         <id>action id<id>
         <name>action name<name>
      </action>
   </consumer>
Running
   <consumer status="running">
      <uuid>UUID</uuid>
      <startdate>yyyy-mm-dd hh:mm:ss.zzz</startdate>

      <action>
         <id>action id<id>
         <name>action name<name>
      </action>
      <task>CURRENTTASK</task>
      <completed>PERCENT</completed>
   </consumer>

NOTE: logs about a completed (or failed) consumer may be expunged from memory. Please refer also to this other proposal about retaining information for longer time and between GeoBatch runs.

Get consumer log

Get the log for a given consumer id.

  • GET geobatch/rest/consumer/{consumerid}/log

Authenticated request.

Responses can be:

  • 200 OK: response is the full log of the consumer.
  • 404 not found : the consumer id does not exist OR the consumerId belongs to a flow not accessible to the authenticated user

NOTE: logs about a completed (or failed) consumer may be expunged from memory. Please refer also to this other proposal about retaining information for longer time and between GeoBatch runs.

Get configured flows

Get a list of configured flows. Authenticated request. Only the flows accessible to the authenticated user will be returned

  • GET geobatch/rest/flow
<flows>
   <flow>
      <id>ID</id>
      <name>NAME</NAME>
      <description>DESC</description>
   </flow>
   ...
</flows>

Get flow info

Get detailed info about a flow.

Authenticated request. Only the flows accessible to the authenticated user will be returned

NOTE: The flow definition may contain sensitive information. The user may be allowed to use the flow, but not to view the configuration. Only a subset of the flow info shall be made visible. See also this proposal.

  • GET geobatch/rest/flow/{flowid}
<flow>
   <id>ID</id>
   <name>NAME</NAME>
   <description>DESC</description>

   <action> 
      <id>action id<id>
      <name>action name<name>
      <description>DESC</description>
   </action>

   ... other actions...
</flow>

Get flow consumers

Get the list of consumers for a given flow.

Authenticated request.
Only the consumers associated to flows accessible to the authenticated user will be returned

  • GET geobatch/rest/flow/{flowid}/consumers
<flow>
   <id>ID</id>
   <name>NAME</NAME>
   <description>DESC</description>

   <consumer status="STATUS"> 
      <uuid>UUID<uuid>
      <startdate>yyyy-mm-dd hh:mm:ss.zzz</startdate>
   </consumer>

   ... other consumers...
</flow>

Managing flows

The operations described above are a subset of what is possible to do using the GeoBatch user interface.

The standard UI also allows some control over the flows, such as pausing, restarting and stopping a flow.
We may want to provide access to such operations using this HTTP REST interface. These operations allow to build a completely decoupled UI to the GeoBatch backend.

Pause a consumer

Pause a running consumer.

The pause may happen within the Action, if the current Action checks for progress control commands. If it does not, the current action will be completed and the consumer will pause after the action has completed, since a pause command check is performed at consumer level before executing each action.

  • PUT geobatch/rest/consumer/{consumerid}/pause

Authenticated request.

Responses can be:

  • 202 accepted: the consumer will be paused as soon as possible.
  • 304 not modified: the consumer is already paused
  • 401 unauthorized: no auth info sent
  • 404 not found : the consumer id does not exist OR the consumerId belongs to a flow not accessible to the authenticated user

Resume a consumer

Restart a paused consumer.

  • PUT geobatch/rest/consumer/{consumerid}/resume

Authenticated request.

Responses can be:

  • 202 accepted: the consumer will be resumed.
  • 304 not modified: the consumer is already running
  • 401 unauthorized: no auth info sent
  • 404 not found : the consumer id does not exist OR the consumerId belongs to a flow not accessible to the authenticated user

Cleanup a completed consumer

Dispose the resources used by a completed consumer.

  • PUT geobatch/rest/consumer/{consumerid}/clean

Authenticated request.

Responses can be:

  • 202 accepted: the consumer resources will be disposed as soon as possible.
  • 401 unauthorized: no auth info sent
  • 409 conflict: the consumer is still executing
  • 404 not found : the consumer id does not exist OR the consumerId belongs to a flow not accessible to the authenticated user
Clone this wiki locally