diff --git a/NEWS.md b/NEWS.md index 7eb3e78..2f68e4f 100644 --- a/NEWS.md +++ b/NEWS.md @@ -1,4 +1,4 @@ -# hydroscoper 0.2.3 (Release date: 2018-02-21) +# hydroscoper 0.2.4 (Release date: 2018-02-21) * General diff --git a/README.md b/README.md index 40fc320..c7aa007 100644 --- a/README.md +++ b/README.md @@ -202,7 +202,7 @@ station_ts #> # A tibble: 4 x 8 #> time_id station_id variable timestep units start_date end_date subdomain #> -#> 1 761 200200 wind_di~ B0 1948-01-0~ 1997-07~ kyy +#> 1 761 200200 wind_di~ ° 1948-01-0~ 1997-07~ kyy #> 2 56 200200 precipi~ 30_minu~ mm 1985-05-0~ 1997-01~ kyy #> 3 760 200200 snow daily mm 1948-01-0~ 1997-07~ kyy #> 4 759 200200 precipi~ daily mm 1953-03-0~ 2011-06~ kyy diff --git a/docs/LICENSE.html b/docs/LICENSE.html index 1ae0a26..baf5400 100644 --- a/docs/LICENSE.html +++ b/docs/LICENSE.html @@ -68,7 +68,10 @@ diff --git a/docs/articles/index.html b/docs/articles/index.html index f310a7f..e7f6672 100644 --- a/docs/articles/index.html +++ b/docs/articles/index.html @@ -68,7 +68,10 @@ @@ -93,7 +96,7 @@
-

Articles version 0.2.3

+

Articles version 0.2.4

@@ -103,7 +106,8 @@

All vignettes

diff --git a/docs/articles/intro_hydroscoper.html b/docs/articles/intro_hydroscoper.html new file mode 100644 index 0000000..e9d628f --- /dev/null +++ b/docs/articles/intro_hydroscoper.html @@ -0,0 +1,268 @@ + + + + + + + +An introduction to <code>hydroscoper</code> • hydroscoper + + + + + + +
+
+ + + +
+
+
+

An introduction to hydroscoper +

+

Konstantinos Vantas

+ +

2018-02-22

+
+ + + +
+

What is Hydroscope?

+

Hydroscope is the Greek National Data Bank for Hydrological and Meteorological Information, a result of long-standing efforts by numerous Greek scientists in collaboration with various companies and associations. It was implemented in three phases, funded by the Ministry of Development, the Ministry of Environment and Energy and the European Union.

+

This National Data Bank provides several data sources from various organisations via a web interface. Each participating organisation keeps its data on its own server using a database system for the storage and management of hydrological and meteorological information. These organisations are:

+
    +
  • Ministry of Environment and Energy.
    • +
  • Ministry of Rural Development and Food.
    • +
  • National Meteorological Service.
    • +
  • National Observatory of Athens.
    • +
  • Greek Prefectures.
    • +
  • Public Power Corporation.
    • +
+

The above data are structured as tables and space separated text files, but are in Greek, thus limiting their usefulness. Another issue with Hydroscope is the lack of comprehensive look-up tables about the available data, which are spread across many different databases.

+

What does hydroscoper?

+

hydroscoper provides functionality for automatic retrieval and translation of Hydroscope’s data to English. The main functions that can be utilized is the family of functions, get_stations, get_timeseries, get_data, etc., to easily download Hydroscope’s data as tibbles.

+

Internal datasets

+

The internal datasets of the package can be used to run queries on the available Hydroscope’s stations and time series data, reducing the time needed for downloading and data wrangling, as these data are rarely modified. These datasets are:

+

stations

+

It is a comprehensive look-up table with geographical and ownership information of the available stations in all Hydroscope’s databases. The variables are:

+
    +
  1. +station_id The station’s ID.
    2. +
  2. +name The station’s name.
    3. +
  3. +water_basin The station’s Water Basin.
    4. +
  4. +water_division The station’s Water Division.
    5. +
  5. +owner The station’s owner.
    6. +
  6. +longitude The station’s longitude in decimal degrees, ETRS89.
    7. +
  7. +latitude The station’s latitude in decimal degrees, ETRS89.
    8. +
  8. +altitude The station’s altitude, meters above sea level.
    9. +
  9. +subdomain The corresponding Hydroscope’s database.
    10. +

timeseries

+

It is also a look-up table with all the available measurements for a given station in a given Hydroscope’s database, with units of measurement and times of those measurements. The variables are:

+
    +
  1. +time_id The time series ID.
    2. +
  2. +station_id The corresponding station’s ID.
    3. +
  3. +variable The time series variable type.
    4. +
  4. +timestep The timestep of time series.
    5. +
  5. +units The units of the time series.
    6. +
  6. +start_date The starting date of time series values.
    7. +
  7. +end_date The ending date of time series values.
    8. +
  8. +subdomain The corresponding Hydroscope’s database.
    9. +

Data sources

+ +

Note that:

+
    +
  1. Only the two Ministries allow to download time series values freely.
    2. +
  2. +ypaat, emy and kyy sub-domains are maintained by the National Technical University Of Athens and these servers work seamlessly.
    3. +
  3. +deh sub-domain is maintained by the Greek Public Power Corporation and occasionally the server is down.
    4. +

Example

+

This is a basic example which shows how to get the stations’ and time series’ data from the Hydroscope’s Ministry of Environment and Energy database, http://kyy.hydroscope.gr/.

+

Load libraries:

+ +

We will use the package’s data stations and timeseries, to reduce the time needed with data munging. We can subset the station’s data for the kyy sub-domain with:

+ +

Let’s plot these stations using the package’s dataset greece_borders.

+ +

+

To get the time series’ data for the station 200200 (http://kyy.hydroscope.gr/stations/d/200200/) we can use:

+ +

We can download the station’s precipitation time series 56 (http://kyy.hydroscope.gr/timeseries/d/56/):

+ +

Let’s create a plot:

+ +

+
+
+
+ + + +
+ + +
+

Developed by Konstantinos Vantas.

+
+ +
+

Site built with pkgdown.

+
+ +
+
+ + + diff --git a/docs/articles/intro_hydroscoper_files/figure-html/kyy_stations_map-1.png b/docs/articles/intro_hydroscoper_files/figure-html/kyy_stations_map-1.png new file mode 100644 index 0000000..e6bcf67 Binary files /dev/null and b/docs/articles/intro_hydroscoper_files/figure-html/kyy_stations_map-1.png differ diff --git a/docs/articles/intro_hydroscoper_files/figure-html/plot_time_series-1.png b/docs/articles/intro_hydroscoper_files/figure-html/plot_time_series-1.png new file mode 100644 index 0000000..839e084 Binary files /dev/null and b/docs/articles/intro_hydroscoper_files/figure-html/plot_time_series-1.png differ diff --git a/docs/articles/stations_with_data.html b/docs/articles/stations_with_data.html index 3bbd017..2638886 100644 --- a/docs/articles/stations_with_data.html +++ b/docs/articles/stations_with_data.html @@ -5,7 +5,7 @@ -Hydroscoper’s data-sets • hydroscoper +Hydroscoper’s data sets • hydroscoper @@ -45,7 +45,10 @@ @@ -72,22 +75,22 @@
-

Hydroscoper’s data-sets

+

Hydroscoper’s data sets

Konstantinos Vantas

-

2018-02-11

+

2018-02-22

-

This vignette shows hot to use the package’s internal data-sets.

+

This vignette shows how to use the package’s internal data sets.

Load libraries

-

Data-sets

-

There are two data-sets stored in the package. stations is comprised of the stations’ id, name, longitude, latitude, etc.

+

Data sets

+

There are three data sets stored in the package. stations is comprised of the stations’ id, name, longitude, latitude, etc.

stations
 #> # A tibble: 2,322 x 9
 #>    station_id name    water_basin  water_division owner longitude latitude
@@ -107,18 +110,18 @@ 
2018-02-11

 
timeseries of the time series’ id, the corresponding station, variable type, time step etc.

 
timeseries
 #> # A tibble: 10,804 x 8
-#>    timeser_id station_id variable   timestep units start_date   end_date  
-#>         <int>      <int> <chr>      <chr>    <chr> <chr>        <chr>     
-#>  1       2248     501049 temperatu~ <NA>     °     2009-02-01T~ 2010-08-3~
-#>  2        430     200103 wind_dire~ <NA>     °     1950-10-26T~ 1997-07-1~
-#>  3        905     200247 wind_dire~ <NA>     °     1967-01-01T~ 1997-12-3~
-#>  4       2243     501058 temperatu~ <NA>     °     1999-01-01T~ 2010-08-3~
-#>  5        438     200105 wind_dire~ <NA>     °     1950-06-05T~ 1997-07-3~
-#>  6        553     200135 wind_dire~ <NA>     °     1964-11-21T~ 1997-08-3~
-#>  7        966     200265 wind_dire~ <NA>     °     1967-01-01T~ 1997-03-3~
-#>  8        775     200203 wind_dire~ <NA>     °     1964-05-20T~ 1997-06-3~
-#>  9       2245     501046 temperatu~ <NA>     °     2007-07-01T~ 2010-07-0~
-#> 10        247     200034 wind_dire~ <NA>     °     1969-11-25T~ 1997-09-1~
+#>    time_id station_id variable   timestep units start_date     end_date   
+#>      <int>      <int> <chr>      <chr>    <chr> <chr>          <chr>      
+#>  1    2248     501049 temperatu~ <NA>     °     2009-02-01T00~ 2010-08-31~
+#>  2     430     200103 wind_dire~ <NA>     °     1950-10-26T08~ 1997-07-19~
+#>  3     905     200247 wind_dire~ <NA>     °     1967-01-01T08~ 1997-12-31~
+#>  4    2243     501058 temperatu~ <NA>     °     1999-01-01T00~ 2010-08-31~
+#>  5     438     200105 wind_dire~ <NA>     °     1950-06-05T08~ 1997-07-31~
+#>  6     553     200135 wind_dire~ <NA>     °     1964-11-21T08~ 1997-08-31~
+#>  7     966     200265 wind_dire~ <NA>     °     1967-01-01T08~ 1997-03-31~
+#>  8     775     200203 wind_dire~ <NA>     °     1964-05-20T08~ 1997-06-30~
+#>  9    2245     501046 temperatu~ <NA>     °     2007-07-01T01~ 2010-07-07~
+#> 10     247     200034 wind_dire~ <NA>     °     1969-11-25T08~ 1997-09-18~
 #> # ... with 10,794 more rows, and 1 more variable: subdomain <chr>

 
greece_borders is a data-frame for use with the function geom_polygon from the ggplot2 package.

Stations location

@@ -165,7 +168,7 @@

Contents

diff --git a/docs/authors.html b/docs/authors.html index 7fceaeb..613ec52 100644 --- a/docs/authors.html +++ b/docs/authors.html @@ -68,7 +68,10 @@ @@ -103,6 +106,10 @@

Authors

Konstantinos Vantas. Author, maintainer.
0000-0001-6387-8791

+
  • +

    Sharla Gelfand. Contributor. +

    +
    • diff --git a/docs/index.html b/docs/index.html index 104fdb1..3a2f8db 100644 --- a/docs/index.html +++ b/docs/index.html @@ -45,7 +45,10 @@ @@ -80,7 +83,7 @@

    hydroscoper is an R interface to the Greek National Data Bank for Hydrological and Meteorological Information, Hydroscope. Hydroscope is the result of long-standing efforts by numerous Greek scientists in collaboration with various companies and associations. It was implemented in three phases, funded by the Ministry of Development, the Ministry of Environment and Energy and the European Union.

    -

    Hydroscope, provides several national data sources from various organisations via a web interface. Each participating organisation keeps its data on its own server using the Enhydris database system for the storage and management of hydrological and meteorological data. These organisations are:

    +

    Hydroscope provides several national data sources from various organisations via a web interface. Each participating organisation keeps its data on its own server using the Enhydris database system for the storage and management of hydrological and meteorological data. These organisations are:

    • Ministry of Environment and Energy.
    • Ministry of Rural Development and Food.
      • @@ -89,7 +92,7 @@
    • Greek Prefectures.
    • Public Power Corporation.
    -

    The data are structured as tables and space separated text files, but are in Greek, thus limiting their usefulness.

    +

    The data are structured as tables and space separated text files, but are in Greek, thus limiting their usefulness. Another issue with Hydroscope is the lack of comprehensive look-up tables about the available data, which are spread across many different databases.

    hydroscoper covers Hydroscope’s data sources using the Enhydris API. The Enhydris database is implemented in PostgreSQL and details about the database can be found here, and about the Web-service API here.

    hydroscoper provides functions to:

      @@ -97,7 +100,48 @@
    1. Transliterate the Greek Unicode text to Latin.
    2. Translate various Greek terms to English.
    -

    Data sources

    +

    The internal datasets of the package can be used to run queries on the available Hydroscope’s stations and time series data, reducing the time needed for downloading and data wrangling, as these data are rarely modified. These datasets are:

    +

    stations

    +

    It is a comprehensive look-up table with geographical and ownership information of the available stations in all Hydroscope’s databases. The variables are:

    +
      +
    1. +station_id The station’s ID.
      2. +
    2. +name The station’s name.
      3. +
    3. +water_basin The station’s Water Basin.
      4. +
    4. +water_division The station’s Water Division.
      5. +
    5. +owner The station’s owner.
      6. +
    6. +longitude The station’s longitude in decimal degrees, ETRS89.
      7. +
    7. +latitude The station’s latitude in decimal degrees, ETRS89.
      8. +
    8. +altitude The station’s altitude, meters above sea level.
      9. +
    9. +subdomain The corresponding Hydroscope’s database.
      10. +

    timeseries

    +

    It is also a look-up table with all the available measurements for a given station in a given Hydroscope’s database, with units of measurement and times of those measurements. The variables are:

    +
      +
    1. +time_id The time series ID.
      2. +
    2. +station_id The corresponding station’s ID.
      3. +
    3. +variable The time series variable type.
      4. +
    4. +timestep The timestep of time series.
      5. +
    5. +units The units of the time series.
      6. +
    6. +start_date The starting date of time series values.
      7. +
    7. +end_date The ending date of time series values.
      8. +
    8. +subdomain The corresponding Hydroscope’s database.
      9. +

    Data sources

    • Ministry of Environment and Energy, National Observatory of Athens and Greek Prefectures, http://kyy.hydroscope.gr/.
    • Ministry of Rural Development and Food, http://ypaat.hydroscope.gr.
      • @@ -137,7 +181,7 @@
    • timeseries a tibble with time series’ data from Hydroscope.
    • -greece_borders a data frame with the borders of Greece.
      • +greece_borders a tibble with the borders of Greece.

    Example

    This is a basic example which shows how to get the stations’ and time series’ data from the Hydroscope’s Ministry of Environment and Energy database, http://kyy.hydroscope.gr/.

    Load libraries:

    @@ -172,33 +216,32 @@ +#> time_id station_id variable timestep units start_date end_date subdomain +#> <int> <int> <chr> <chr> <chr> <chr> <chr> <chr> +#> 1 761 200200 wind_di~ <NA> ° 1948-01-0~ 1997-07~ kyy +#> 2 56 200200 precipi~ 30_minu~ mm 1985-05-0~ 1997-01~ kyy +#> 3 760 200200 snow daily mm 1948-01-0~ 1997-07~ kyy +#> 4 759 200200 precipi~ daily mm 1953-03-0~ 2011-06~ kyy

    We can download the station’s precipitation time series 56 (http://kyy.hydroscope.gr/timeseries/d/56/):

    -
    ts_raw <- get_data(subdomain = "kyy",time_id = 56)
+
 
    Let’s create a plot:
    
 
    library(ggplot2)
-ggplot(data = ts_raw, aes(x = dates, y = values))+
+ggplot(data = ts_raw, aes(x = date, y = value))+
   geom_line()+
   labs(title= "30 min precipitation for station 200200",
        x="Date", y = "Rain height (mm)")+
@@ -248,6 +291,7 @@ 
    Developers
    
 
      
 
    • Konstantinos Vantas 
       Author, maintainer  
      (0000-0001-6387-8791)
 
      • 
+
    • All authors...
      • 
 
    
 
    Dev status
    
 
      
@@ -256,7 +300,7 @@ 
      Dev status
      
 
    • codecov
      • 
 
    • minimal R version
      • 
 
    • CRAN_Status_Badge
      • 
-
    • packageversion
      • 
+
    • packageversion
      • 
 
      • 
 
    • ropensci
      • 
 
    • DOI
      • 
diff --git a/docs/news/index.html b/docs/news/index.html
index a6e8717..1c4363a 100644
--- a/docs/news/index.html
+++ b/docs/news/index.html
@@ -68,7 +68,10 @@
   
   
 
@@ -101,17 +104,17 @@ 
      Change log All releases
      
 
     
      
     
      
-
      hydroscoper 0.2.3 (Release date: 2018-02-11)
      
+
      hydroscoper 0.2.4 (Release date: 2018-02-21)
      
 
        
 
      • 
 
        General
        
 
          
 
        • This is a major update. All the functions were rewritten utilizing the Enhydris API.
          • 
 
        • The included data in the package cover all Hydroscope’s databases.
          • 
-
        • Added vignettes “Hydroscoper’s data-sets”.
          • 
+
        • Added vignettes “An introduction to hydroscoper” and “Hydroscoper’s data-sets”.
          • 
 
        • Added package documentation page.
          • 
 
        • Use pingr package to check if a sub-domain is alive.
          • 
-
        • All the functions return a tibble.
          • 
+
        • All the functions return tibbles.
          • 
 
        • Added greece_borders dataset.
          • 
 
        
 
        • 
@@ -152,7 +155,7 @@ 
        hydroscoper 0.2.3 (Release date: 2018-02-11)
        
 
      • 
 get_stations and get_timeseries use the Enhydris API and are considerably faster.
        • 
 
      • 
-get_data uses lower case variable naming: dates, values, comments
+get_data uses lower case variable naming: date, value, comment
 
        • 
 
      
 
@@ -179,7 +182,7 @@ 
      hydroscoper 0.1.0 (Release date: 2017-12-22)
      
     
      
       
      Contents
      
       
     
      
diff --git a/docs/reference/get_data.html b/docs/reference/get_data.html
index ed1275e..c275714 100644
--- a/docs/reference/get_data.html
+++ b/docs/reference/get_data.html
@@ -68,7 +68,10 @@
   
   
 
@@ -120,15 +123,16 @@ 
       Ar
     
     
      Value
      
 
-    
      If subdomain is one of "kyy" (Ministry of Environment
-and Energy) or "ypaat" (Ministry of Rural Development and Food),
-and timeID is not NULL, returns a tidy dataframe with the time series values
-from the corresponding database of hydroscope.gr. Otherwise returns an error
-message.
      
+    
      If subdomain is one of:
        
+
      • kyy, Ministry of Environment and Energy
        • 
+
      • ypaat, Ministry of Rural Development and Food
        • 
+
      • deh, Greek Public Power Corporation
        • 
+
      • emy, National Meteorological Service
        • 
+
      and time_id exists in that subdomain, returns a tibble with the time series values. Otherwise returns an error message.
      
 
      The dataframe columns are:
      
-    
      dates
      The time series Dates (POSIXct)
      
-    
      values
      The time series values (numeric)
      
-    
      comments
      Comments about the values (character)
      
+    
      date
      The time series Dates (POSIXct)
      
+    
      value
      The time series values (numeric)
      
+    
      comment
      Comments about the values (character)
      
 
      
 
     
diff --git a/docs/reference/get_tables.html b/docs/reference/get_tables.html
index b0871fe..2959826 100644
--- a/docs/reference/get_tables.html
+++ b/docs/reference/get_tables.html
@@ -68,7 +68,10 @@
   
   
 
diff --git a/docs/reference/greece_borders.html b/docs/reference/greece_borders.html
index 43fa4b0..dd46e7c 100644
--- a/docs/reference/greece_borders.html
+++ b/docs/reference/greece_borders.html
@@ -68,7 +68,10 @@
   
   
 
@@ -108,7 +111,7 @@ 
      Greek borders
      
         
     
      Format
      
 
-    
      A data frame with 18474 rows and 8 variables:
      
+    
      A tibble with 18,474 rows and 8 variables:
      
   
      long
      Longitude in decimal degrees, ETRS89
      
   
      lat
      Latitude in decimal degrees, ETRS89
      
   
      order
      order, integer
      
diff --git a/docs/reference/hydro_coords.html b/docs/reference/hydro_coords.html
index 21be8a7..3d33131 100644
--- a/docs/reference/hydro_coords.html
+++ b/docs/reference/hydro_coords.html
@@ -68,7 +68,10 @@
   
   
 
diff --git a/docs/reference/hydro_translate.html b/docs/reference/hydro_translate.html
index 49f44c5..be2dedc 100644
--- a/docs/reference/hydro_translate.html
+++ b/docs/reference/hydro_translate.html
@@ -68,7 +68,10 @@
   
   
 
diff --git a/docs/reference/hydroscoper.html b/docs/reference/hydroscoper.html
index 200f525..6e804fb 100644
--- a/docs/reference/hydroscoper.html
+++ b/docs/reference/hydroscoper.html
@@ -68,7 +68,10 @@
   
   
 
@@ -156,6 +159,11 @@ 
      Author
      
     
 Maintainer: Konstantinos Vantas kon.vantas@gmail.com (0000-0001-6387-8791)
 
+Other contributors:
+
+
 
   
      
 
      
diff --git a/docs/reference/hydroscoper_defunct.html b/docs/reference/hydroscoper_defunct.html
index b6c7635..14bf179 100644
--- a/docs/reference/hydroscoper_defunct.html
+++ b/docs/reference/hydroscoper_defunct.html
@@ -68,7 +68,10 @@
   
   
 
diff --git a/docs/reference/index.html b/docs/reference/index.html
index 28e675c..03887ac 100644
--- a/docs/reference/index.html
+++ b/docs/reference/index.html
@@ -68,7 +68,10 @@
   
   
 
@@ -97,7 +100,7 @@
     
      
       
      
         Reference
-        version 0.2.3
+        version 0.2.4
       
      
     
      
 
diff --git a/docs/reference/stations.html b/docs/reference/stations.html
index 8b585ec..2f1b4f9 100644
--- a/docs/reference/stations.html
+++ b/docs/reference/stations.html
@@ -68,7 +68,10 @@
   
   
 
@@ -100,14 +103,16 @@ 
      stations
      
 
     
     
      Stations' data from the Greek National Data Bank for
-Hydrological and Meteorological Information.
      
+Hydrological and Meteorological Information. This dataset is a comprehensive
+look-up table with geographical and ownership information of the available
+stations in all Hydroscope's databases.
      
     
 
     
      stations
      
         
     
      Format
      
 
-    
      A data frame:
      
+    
      A tibble with 2,322 rows and 9 variables:
      
     
      station_id
      The station's ID from the domain's database
      
     
      name
      The station's name
      
     
      water_basin
      The station's Water Basin
      
diff --git a/docs/reference/timeseries.html b/docs/reference/timeseries.html
index 29bac35..e9a8631 100644
--- a/docs/reference/timeseries.html
+++ b/docs/reference/timeseries.html
@@ -68,7 +68,10 @@
   
   
 
@@ -100,15 +103,18 @@ 
      timeseries
      
 
     
     
      Time series' data from the Greek National Data Bank for
-Hydrological and Meteorological Information.
      
+Hydrological and Meteorological Information.  This dataset is a comprehensive
+look-up table of all of the available measurements for a given station in a
+given Hydroscope's  database, with units of measurement and times  of
+those measurements.
      
     
 
     
      timeseries
      
         
     
      Format
      
 
-    
      A data frame:
      
-    
      timeser_id
      The time series ID from the database
      
+    
      A tibble with 10,804 rows and 9 variables:
      
+    
      time_id
      The time series ID
      
     
      station_id
      The corresponding station's ID
      
     
      variable
      The time series variable type
      
     
      timestep
      The timestep of time series