v0.3.8

This is a minor release, focusing on under-the-hood improvements and introducing better handling of files with varying vertical layers.

New methods

A method, vertical_integration for calculating vertically integrated totals for netCDF data of the likes of oceanic data, where the vertical levels vary spatially, is introduced. vertical_mean has been improved and can now calculate vertical mean in cases where the cell thickness varies in space.

Deprecations

merge_time is deprecated, and its functionality will be incorporated into merge. So, following this release ensemble merging operations should use merge.

Improvements

open_url is now able to handle multiple urls. Previously it could only handle one.

Some under-the-hood improvements have been made to assign to ensure that truth statements do not occassionally throw an error.

Full Changelog: v0.3.7...v0.3.8

v0.3.7

This is a minor release with new features and one bug fix.

New methods

New mathematical methods for simple operations on variables will be added: abs, power, square, sqrt, exp, log and log10. These methods match numpy names.

Bug fix

assign previously did not accept log10 in the lambda function. Now fixed.

v0.3.6

This is a minor release.

New methods

New methods ensemble_mean and ensemble_stdev are introduced for calculating variance and standard deviation across ensembles.

Method renaming

The method tvariance is deprecated and is now renamed tvar for naming consistency.

v0.3.5

This is a minor release focusing on some under-the-hood improvements in performance and a couple of new methods.

It drops support for CDO version 1.9.3, as this is becoming too time-consuming to continue given the increasingly low reward.

A couple of new methods have been added. distribute enables files to be split up spatially into equally sized m by n rectangles. collect is the reverse of distribute. It will collect distributed data into one file.

In prior releases assign calls could not be split over multiple lines. This is now fixed.

There was a bug in previous releases where regrid did not work with multi-file datasets. This was due to the enabling of parallel processing with nctoolkit. The issue is now fixed.

The deprecated methods mutate and assign have now been removed. Variable creation should use assign.

v0.3.4

This is a minor release. All non-deprecated methods are unchanged from the prior release.

A new method fill_na has been introduced that allows missing values to be filled with the distanced weighted average neighbour.

The previously deprecated methods remove_variables and cell_areas have been removed and are replaced permanently by drop and cell_area.

v0.3.2

This is a quick and minor release to fix a bug in to_nc that resulted in files not being saved to the base directory if specified.

Other than code tidying, there are not other changes to the package.

v0.3.1

This is a minor release that includes new methods, under-the-hood improvements, minor bug fixes and the removal of deprecated methods. Code written for non-deprecated methods in v0.3.0 will be compatible with v0.3.1.

New methods

New methods are introduced for identifying the first time step when specific numerical thresholds are first exceeded or fallen below etc: first_above, first_below, last_above and last_below. The thresholds are either single numbers or can come from a gridded dataset for grid-cell specific thresholds.

Methods to compare a dataset with another dataset or netCDF file have been added: gt and lt, which stand for ‘greater than’ and ‘less than’.

Method improvements

Users will now be able to recycle the weights calculated when interpolating data. This can enable much faster interpolation of multiple files with the same grid.

There have been some under-the-hood improvements to improve automatic deletion of temporary files when running in parallel.

Bug fixes

assign threw an error when an internal CDO function was used more than once. This is now fixed.

Method removals post-deprecation

The temporal methods replaced by tmean etc. have now been removed from the package. So all previous temporal averaging methods of the form daily_, monthly_ and annual_ can no longer be used.

v0.3.0

This is a major release that includes important significant enhancements to nctoolkit's functionality and breaking changes to prior releases.

Variable creation

A new method has been added for creating new variables: assign. This operates in a similar way to the assign method in pandas, and means users can create new variables using lambda functions. This replaces the mutate and transmute methods, which were always intended to be place-holders for a more effective method.

Parallel processing

nctoolkit processing chains can now be run in parallel after under-the-hood changes. This means you can wrap a processing chain into a function, and use the multiprocessing package to process files in parallel.

Evaluation is now lazy by default

In previous releases, evaluation was eager, not lazy, by default. From now on, evaluation will be lazy by default, but can be changed using nc.options(lazy = False).

Configuration files

The package now accepts the use of a configuration file, which should be called .nctoolkitrc or nctoolkitrc and be placed in your working directory or home directory. This means arguments sent to options can be set permanently or on a project basis. This makes life easier for people who have a small or shared temporary folder that they do not want to write to.

Datasets now have clearer list-like behaviour

Improvements have been made to datasets to make them behave in a more list-like manner. data.current will now return a list, whereas in the past it would return a str representing the file in a single-file dataset. This has enabled improvements to how files in datasets can be handled. You can now create an empty dataset, for cases when you need to progressively build a dataset. Likewise, there is now a remove method for removing files from datasets, which goes along with the existing append method for appending files.

A potential consequence of this is that code that assumes data.current etc. is a str will likely break, as it will now be a list.

Removed methods

select_ methods, such as select_years, have been removed after a few months of deprecation. Use select instead.

write_nc has been removed after a few months of deprecation. Use to_nc instead.

clip has been removed after a few months of deprecation. Use crop instead.

release has been removed after a few months of deprecations. Use run instead.

Renamings and deprecations

cell_areas renamed to cell_area.
mutate and transmute have been replaced by assign. These methods have therefore been deprecated and will be deleted in a couple of months.

remove_variables is now replaced by drop. remove_variables is deprecated.

compare_all is now replaced by compare. compare_all is deprecated.

Bug fixes

An error was being thrown when deleting temp files hanging round from NCO failing to create files. This is now fixed.

v0.2.4

This minor release adds new features. Code written for prior releases are compatible with this release.

Improved temporal statistics

Calculating temporal statistics has been streamlined. Temporal methods are now prefixed with the letter t, and accept grouping options. So, for example, data.tmean(["year", "month"]) will calculate the mean value for each month in each year.

Mild enhancements to methods

The select and shift methods have been tweaked to make life slightly easier for users, and allow partial matches for args. For example data.shift(hours = 1) and data.shift(hour = 1) are both valid.

open_url is now able to keep attempting to download files for a set period until complete.

open_thredds now has a wait option to stop

Bug fixes

Issues related to user specified temp folders not being set are now fixed.

Renamings and deprecations

Existing temporal statistics methods, such as mean, max etc., have now been deprecated and the new methods tmean etc. should be used.

vertical_cum_sum has been renamed to vertical_cumsum. vertical_cum_sum is now deprecated.

v0.2.3

Plotting improvements

Automatic plotting now relies on the new ncplot package. Plotting is now greatly enhanced and includes automatic coastlines.

Mild enhancements to methods

• Users can now specify the temporary folder to be used by nctoolkit with nctoolkit.options(temp_dir = x)

Deprecations and renaming

• var is now called variance to be more consistent with python. var is deprecated and will be removed early 2021.

bug fixes

• append was not working properly for ensembles. Now fixed