Reversed Timeseries Data in LISFLOOD Output

[Nooshdokht-Bayatafshary]

Dear Developers,

I am currently running LISFLOOD and its calibration module for my region and have encountered an issue. As I understand it, the timeseries output of LISFLOOD does not include timestamp indices; instead, it starts from 1. Initially, I assumed that an index of 1 corresponded to the start time in LISFLOOD and incremented up to the final timestep, which would be the end time. However, when I compared the time series of input precipitation and average temperature with rainUps.tss and tAvgUps.tss, I noticed a discrepancy. The data do not match.

Upon closer examination, I realized that the data in the LISFLOOD output timeseries is reversed. Additionally, when the model is run for more than one year (as in my case) and the inputs are separated into yearly netCDFs, each year's data is reversed separately, but the order in the output file is ascending. This means that data from 01-01-2015 to 31-12-2015 is reversed and printed in the first part of the tss file, then data from 01-01-2016 to 31-12-2016 is reversed and appended to the 2015 data, and so on for subsequent years.

I am unsure why this is happening. I also know this occurs for the discharge output as well. Could you please advise on what might be causing this issue? Have I set something incorrectly in the XML file? Additionally, if the outputs are reversed, does LISCAL use the reversed discharge data for calculating the KGE each time the model is run? Because, I am unsure about the output of the LISCAL calibration code. The values of the streamflow_simulated_best.csv file is not reversed, so the timeseries data still does not appear to be correct. This discrepancy adds to my confusion and makes it difficult to determine the root cause of the issue.

I am attaching my XML file, the air temperature and precipitation input files, and the model output files, including rainUps.tss, tAvgUps.tss, and dis.tss. Additionally, I am sending an Excel file that shows the data before and after reversing the timeseries (Data.zip).

Thank you for your assistance.

Best regards,
Nooshdokht

[StefaniaGrimaldi]

Dear @Nooshdokht-Bayatafshary,

thank you for reaching out. We advise to further investigate the reported issue before performing model calibration.
We were not able to reproduce your error on our premises because we do not have your complete and corrected model set-up (the maps that you kindly shared with us in previous messages were still affected by errors).

Concerning the general implementation of OS LISFLOOD, we would like to stress that a 7-years pre-run is very likely too short to allow adequate initialization of model internal states (and this lack of adequate initialization will negatively affect the results of the model run and of the calibration). Furthermore, we recommend referring to this example https://github.com/ec-jrc/lisflood-calibration/blob/master/integration/settings_GloFAS3arcmin_referencetemplate.xml for the initialization of the lower groundwater zone. Finally, in case you run the calibration for separate inter-catchments, and your settings include water use and water regions, it could be better to set the option "groundwatersmooth" as OFF. We can surely discuss these topics further in sub-sequent messages. We thought it could be useful to share these comments here. However, as stated above, we should first solve the reported issue.

Since this is a very busy period for us, we were wondering whether you can help us debug your issue. For this purpose, we would like to suggest a preliminary test. The error shows up in the very first year of the model run. Have you tried to run the model for the month of January 2015, and then for the full 2015 ? Are the values in reversed order also when you run the model for one month / one year? (please also note that Step Start 01/01/2015 00:00 means that you are modelling the last day of December 2014: https://ec-jrc.github.io/lisflood-code/2_ESSENTIAL_time-management/)

Kind regards,
on behalf of the developers team,
Stefania

[Nooshdokht-Bayatafshary]

Dear @StefaniaGrimaldi,

Thank you for your detailed response and suggestions. I appreciate your guidance on addressing the issue before proceeding with model calibration.

Regarding the problem, I understand that the previously shared maps contained errors. I am sending you the complete maps and other required inputs (Data.zip).

I acknowledge the importance of an adequate initialization period for the model's internal states and will extend the pre-run period beyond the current 7 years to ensure better initialization. I also appreciate the example provided for the initialization of the lower groundwater zone and will refer to it in my setup. Your recommendation about setting the "groundwatersmooth" option to OFF when running calibration for separate inter-catchments with water use and water regions is noted, and I will incorporate this into my settings. Currently, I am testing different calibration parameters of the model in LISFLOOD and investigating their effect on baseflow and discharge. I am happy to test your recommendations as well. Thank you for offering to discuss these topics further in subsequent messages. I am very grateful for your expertise.

To help debug the issue, I performed the suggested preliminary tests. I ran the model for January 2015 and then for the entire year of 2015. The output time series had reversed order in both tests. I have sent the rainfall time series (original and reversed compared to input precipitation) in an Excel file in the shared data. I also checked one output (RainUps.tss), and it seems that the writing output of LISFLOOD in time series uses an equal module, so if one is reversed, all of them have the same results.

Best regards,
Nooshdokht

[StefaniaGrimaldi]

Dear @Nooshdokht-Bayatafshary,

thank you for sharing your set-up and thank you for running a first test. We were able to reproduce your error on our premises, and we also identified the cause of the issue.

You surely noticed the warning:

LISFLOOD code correctly handles single maps with inverted y (or x) axis. However, the inversion of maps with time dimension had an issue. Your .tss output time series were flipped because the values in the forcings input maps were used in reverse order within the code.

Carlo (@doc78) has already identified and implemented the required improvement to the LISFLOOD code in the DEVELOPMENT branch:
https://github.com/ec-jrc/lisflood-code/tree/development

We tested the improvements of the development branch with specific focus on the usage of the meteo forcings: the .tss files now show the correct values order.

The correction will be pushed to the LISFLOOD master branch in the next few months. Thank you for raising the issue: it helped us identify a weakness in the code.

To proceed with your work, you might either (1) use the branch https://github.com/ec-jrc/lisflood-code/tree/development, or (2) you can make sure that all your maps are designed according to the standard reference system (the latter solution will also avoid the various warnings that you see when you launch LISFLOOD).

The correction of the DEVELOPMENT branch is expected to work for all variables with time dimension, however, so far our tests focused on the meteo forcings, and we have not done yet tests with specific focus on other time-dependent maps (e.g. water demand, land cover, LAI). If you decide to proceed with solution 1, we will be grateful for your feedback.

Kind regards,
Carlo and Stefania

[Nooshdokht-Bayatafshary]

Dear @StefaniaGrimaldi and @doc78,

Thank you for your prompt and thorough response. I appreciate the efforts made to identify and address the issue.

I am very pleased to hear that an update will be released in the next few months, and I am looking forward to using it. In the meantime, I will proceed with your first solution and use the development branch, as recreating all my maps, especially the meteorological forcing, would be quite time-consuming.

Additionally, I have checked the actEvapo output, and I can confirm that it shows the correct trend with a good correlation to the LAI maps, suggesting that the LAI section is functioning well. However, although I work with transient land cover, I'm unsure how to test it effectively. If you have any suggestions, I’d be happy to check it and provide further feedback.

Your support has been invaluable in moving forward with my work.

Kind regards,
Nooshdokht