Change to NWFSC Data Warehouse URL

[Curt-Whitmire-NOAA]

Hi Jim,
NWFSC IT recently changed the server URL for the FRAM Data Warehouse. The new URL adds "webapps" to the front end of the URL, like the example below:

https://www.webapps.nwfsc.noaa.gov/data/api/v1/source/trawl.catch_fact

[afredston]

I assume this breaks download_catch_rates()? I can submit a PR to update the URLs if that's helpful...

[James-Thorson-NOAA]

Yeah, I haven't checked. Are you interested in taking a moment to confirm and then do the PR @afredston ...? Or tell me if you're too busy and ... err... I might ignore it until I'm back from leave :0

[afredston]

happy to do it this week! stay tuned and thanks for the heads-up @Curt-Whitmire-NOAA

[Curt-Whitmire-NOAA]

Thanks @afredston! Yes, the new URL causes download_catch_rates() to not access data for survey == "WCGBTS", "WGHL" and "WGT".

[Curt-Whitmire-NOAA]

I should have wrote the "current URL" causes download_catch_rates() to not access data...

[kellijohnson-NOAA]

Perhaps another option would be to rely on nwfsc-assess/nwfscSurvey::PullCatch.fn that @chantelwetzel-noaa maintains.

[James-Thorson-NOAA]

@chantelwetzel-noaa @kellijohnson-NOAA Could either of you comment on what output nwfsc-assess/nwfscSurvey::PullCatch.fn(.) generates, or even how it compares with the function here?

I've always hoped to transition to each region maintaining their own function to pull their API, so that FishData is really more about harmonizing and publicizing those regional sources (as needed realistically for any comparative analysis). So I'm excited to learn about that other option!

[kellijohnson-NOAA]

@James-Thorson-NOAA nwfscSurvey::PullCatch.fn(SurveyName = "NWFSC.Combo") results in a data frame with the following columns:
Trawl_id
Common_name
Project
Year
Pass
Vessel
Tow
Date
Depth_m
Longitude_dd
Latitude_dd
Area_Swept_ha
Subsample_count
Subsample_wt_kg
total_catch_numbers
total_catch_wt_kg
cpue_kg_km2

so I think we would have to add a few columns to match what you have, though I don't know exactly what columns are necessary.

[afredston]

I added a PR to fix the URLs for now.

The output columns of download_catch_rates() are "Sci" "Year" "TowID" "Lat" "Long" "Wt" (or Num in some cases) and "AreaSwept_ha" -- looks like those are all in nwfscSurvey except scientific names?

[James-Thorson]

Thanks Alexa for the PR! It looks like nwfscSurvey::PullCatch.fn will work as better option to pull data from NWFSC warehouse, although I haven't actually explored much. If anyone is interested in comparing the existing code with that alternative option, please respond on this thread.

…

On Sat, Aug 22, 2020 at 9:45 AM Alexa Fredston ***@***.***> wrote: I added a PR <#25> to fix the URLs for now. The output columns of download_catch_rates() are "Sci" "Year" "TowID" "Lat" "Long" "Wt" "AreaSwept_ha" -- looks like those are all in nwfscSurvey except species names? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#24 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB46UTOTUDUNQ6UOJUUOHWDSB7Y4FANCNFSM4P3SE3RA> .