Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More sophisticated object-oriented AWS data processing #134

Open
PennyHow opened this issue Apr 14, 2023 · 0 comments
Open

More sophisticated object-oriented AWS data processing #134

PennyHow opened this issue Apr 14, 2023 · 0 comments
Labels
enhancement New feature or request

Comments

@PennyHow
Copy link
Member

Currently the pypromice.process.AWS object holds all of the information and functions for processing data from a level 0 to level 3 standard. The object largely holds processing parameters, which are inputted and passed as object attributes on initialisation, namely:

  1. The station config parameters
  2. The standard variables formatting
  3. The metadata formatting and content

class AWS(object):
'''AWS object to load and process PROMICE AWS data'''
def __init__(self, config_file, inpath, var_file=None, meta_file=None):
'''Object initialisation
Parameters
----------
config_file : str
Configuration file path
inpath : str
Input file path
var_file: str, optional
Variables look-up table file path. If not given then pypromice's
variables file is used. The default is None.
meta_file: str, optional
Metadata info file path. If not given then pypromice's
metadata file is used. The default is None.
'''
assert(os.path.isfile(config_file))
assert(os.path.isdir(inpath))
print('\nAWS object initialising...')
# Load config, variables CSF standards, and L0 files
self.config = self.loadConfig(config_file, inpath)
self.vars = getVars(var_file)
self.meta = getMeta(meta_file)
# Hard-wire the msg_lat and msg_lon here
# Prevents having to list these vars in the individual station toml files
config_keys = list(self.config.keys())
for i in config_keys:
self.config[i]['columns'].extend(['msg_lat', 'msg_lon'])
# Load config file
L0 = self.loadL0()
self.L0=[]
for l in L0:
n = getColNames(self.vars, l.attrs['number_of_booms'], l.attrs['format'])
self.L0.append(popCols(l, n))

This object could be more sophisticated, with each sensor represented as an object and therefore a weather station and its data processing represented as a set of sensor objects. In doing so, we could better adapt to different station types and perhaps the pypromice package could be more applicable to other projects and groups.

This is a long-term goal rather than a quick fix, but thought it best to just put it here in order to exchange ideas, document progress etc.

@PennyHow PennyHow added the enhancement New feature or request label Apr 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant