Skip to content

adaptor

Jump to bottom
Tom Russell edited this page Feb 11, 2019 · 1 revision

Adaptors: A SectorModel can access data at the spatial & temporal resolution required by the model

E.g. population, stored in scenarios at output-area level aggregation, required by the water supply model at catchment or water authority area aggregation.

A data translator function allows conversion between known geographies.

The geography types and boundaries must be specified up front, e.g. output areas, local authority districts, and water authority regions, each with a set of (id, boundary) values.

The proxy function (if any) must be specified up front. The flexibility to encode alternate translation functions should be built in to the framework but will not be entirely dynamic.

SosModel defines a gazateer of inputs and outputs used

Basic operation:

  • Sum
  • Average
  • Disaggregate

From:

  • Geography
  • Time
  • Value
  • Unit

To:

  • Geography
  • Time
  • Value
  • Unit

Corner cases:

  • Overlaps (50% of A goes to 0, 50% of A goes to 1)
  • Identical regions
  • Target regions only partially covered by source regions

Users should be able to define functions which generate spatial or temporal conversion coefficients.

Many of the simulation modelling teams have mentioned the need to convert data across differing geographies. For example, the energy supply model requires aggregate final energy demand across area centering on substation nodes in the electricity network. This data may be produced across a different geography, such as an output area based on socio-economic and population data.

The functions for performing this aggregation and disaggregation should be specifiable as a function of other data (e.g. population) or via an abstraction e.g. assumption of uniformity

Stories

A PresetAdaptor converts between specified region/interval definitions

Labels:

  • adaptor
  • data_handle
  • smif

On a similar pattern as DefaultAdaptor, except here we load the mapping from a precomputed file, for a given pair of specs.

This fits well with use cases where a geographical extent might not be totally apparent (between abstract model zones), or where boundary changes are minimal (LAD 2009 -> LAD 2016) and it makes sense to specify precisely the mapping between region sets.

At init, a PresetAdaptor can parse and load its conversion matrix, and check that it is connected with inputs and outputs in resolutions that it knows how to handle.

In contrast with DefaultConvertor, there is no programmatic way to generate conversion coefficients, so this would fail if the presets are not found.

At simulate, we convert inputs to the output resolutions.

Performance of adapters that perform conversions on inputs-outputs could be improved

Labels:

  • adaptor

  • performance

  • smif

    Adapters that perform spatial-temporal-unit conversion take a list of inputs and convert them in series. A significant performance boost could be achieved through:

  • caching (using functools.lru_cache in smif.convert.register) the coefficients avoiding load from disk (as input/output pairs normally share conversion coefficients)

  • performing the conversions in parallel as each input-output pair is independent of all the others (in the adapter)

  • queuing the reads and writes to disk (datafileinterface)

Clone this wiki locally