HydPy 6.2.0

We are pleased to release HydPy 6.2, which accelerates simulation runs through multithreading, introduces the new main model families HydPy-G and HydPy-WHMod, supports Python 3.13 and 3.14, and includes several smaller enhancements.

All these new features should be implemented without relevant breaking changes. For example, "normal" single-threaded simulations are still possible and remain the default behaviour. The only exception is that we have dropped support for Python 3.9 and 3.10.

Please note that the new submodel family HydPy-Snow is functional, but it is still not as convenient as other submodel families. We plan to completely submodularise snow processes in HydPy 7.0.

New models:

• The new model family HydPy-G brings the ideas of the GR model series (modèle due Génie Rural) into HydPy. Among the available application models is HydPy-G-GR4, which implements the well-known, parameter-sparse lumped model GR4J.
• The new submodel family HydPy-Snow so far provides the relatively simple application models HydPy-Snow-CemaNeige_T-Mean and HydPy-Snow-CemaNeige-T-MinMax
• The new model family HydPy-WHMod stands out from the so-far available model collection by providing SVAT models that focus more on water balance aspects of individual sites than on simulating the discharge of entire river basins.
• The new main model HydPy-KinW-Implicit-Euler is a very robust hydrological routing method. Its numerical stability is not restricted by the Courant condition, which makes it very efficient when applied to short routing distances or long time simulation timesteps.
• The new submodel HydPy-WQ-Widths-Strickler serves to specify complex cross-section profiles based on width measurements. It enables distinguishing subareas actively involved in water routing from those that are not, allowing one-dimensional routing methods to approximate some two-dimensional phenomena.
• The new main model HydPy-Dam-DB serves to simulate detention basins that actively control their release. It is the first member of the HydPy-Dam model family capable of utilising discharge information from distant observation points to enhance flood protection in river sections further downstream.
• The new submodel HydPy-Exch-Interp transfers arbitrary information from multiple observation points to its main model and transforms it via configurable interpolation functions.

New model features:

• The new HydPy-L model parameter BSf0 gives more flexibility in configuring direct runoff generation. This can help prevent the simulation of excessively high runoff peaks from short rainfall events during otherwise dry summer periods.
• All HydPy-L models now allow specifying the elevation of individual response units (parameter GH) and the atmospheric temperature gradient (parameter ATG) to ease adjusting subbasin-wide average air temperatures to local terrain differences.
• HydPy-L-DD now has a temperature-based mechanism that helps to prevent the formation of persistent "snow towers" in alpine catchments. One can enable it via the new parameters AGGH and AGSH.
• HydPy-L-DD now has a simple glacier routine. The new parameters FEis and GSF serve to configure the transformation of snow into glacier ice and the melting of glacier ice.
• HydPy-Dam-L-Lake, HydPy-Dam-L-RB, HydPy-Dam-L-Res, and also the new application model HydPy-Dam-L-DB now allow defining a commissioning date, before which they behave inactively.
• Calibrating the ARMA coefficients of the routing model HydPy-ARMA-RIMO/RIDO by optimising, for example, the parameters of the translation diffusion equation is now more robust and much faster (but maybe sometimes a little less accurate).

New general features:

• After offering additional CPU resources via the new option threads, HydPy tries to accelerate simulation runs by performing calculations in parallel. In different projects, the speedup ranged from less than 1% per additional thread (due to widely "unparallelisable" networks) to more than 100% (due to an improved processing order of "parallelisable" network parts).
• Node instances now provide methods to plot (flow) duration curves of simulated and measured time series.
• The new efficiency criteria functions fdc_nse and fdc_nse_log allow putting more emphasis on the statistical distribution than on the timing of individual events when calibrating or evaluating hydrological models.
• All efficiency criteria functions now have the skip_notpositive option, which is especially useful for functions like nse_log that cannot deal with negative and zero values.
• Function write_ncfile makes writing NetCDF time series files readable by HydPy more comfortable.
• The different file managers (for example, the ConditionManager) now print a note if their current working directory is not explicitly defined but successfully "guessed". You can turn this off by setting printprogress to False.
• Changing the project root directory (for example, to store a modified project version in a separate directory) now requires only a single command: pub.projectname = "new_project".
• The "complete" selection of a project's network is now automatically created upon request by the new property complete of class Selections. This change decreases the risk of working with outdated data.
• We added top-level update_parameter methods that should help not to forget to update the derived parameters of all relevant model instances after changing their control parameters (for example, during calibration).
• The new calibration rule LogReplace supports optimising selected model parameters by using logarithmic transformation. CalibrationInterface provides interfaces that allow users to define custom transformation rules.
• Reading time series from files that cover only a part of the whole considered period previously generally resulted in a loss of data for the uncovered parts. This overwriting of already available data with nan values can now be turned off with the reset option.

HydPy 6.1.5

As discussed in issue #168, HydPy-Musk-MCT's behaviour for negative inflow was previously left to chance to some extent. Starting with HydPy 6.1.5, it is always well-defined and configurable by users who wish to have complete control over situations where the occurrence of negative inflow values cannot be excluded.

Additionally, commit 27591db fixed a possible zero division in method Calc_MaxTimeStep_V5 of application model sw1d_gate_out that could occur when testing or experimenting in the pure Python mode.

HydPy 6.1.4

Again, we encountered cases where HydPy-Musk-MCT calculated nan values for low-flow situations. This time, these were caused by negative estimates of the reference discharge. We fixed this problem by simply resetting negative values to zero (see Calc_ReferenceDischarge_V1 for an example).

HydPy 6.1.3

This bugfix release improves how main models share information with their sub-submodels. Previously, some data was not passed as deep copies (see 54a5b7d), which could result in wrong assumptions of sub-submodels on, for example, which hydrological response units of the main model consider interception processes.

The previous bug could impact simulation results. However, only those projects could be affected that use a main model that defines a specific "prepare method" (for example, prepare_interception), a submodel that does not specify an equivalent method, and a sub-submodel that uses the information provided by this method. Such combinations can currently only occur when HydPy-Evap is involved and should be rare. Hence, it is unclear if any real projects have been affected.

HydPy 6.1.2

We encountered cases where HydPy-Musk-MCT calculated nan values for extremely small reference discharge values. We could fix this problem by increasing the precision of the iterative search for the corresponding reference water level (based on the Pegasus method). Hence, this bugfix release should not only avoid the nan value problem but might also result in slightly different (typically slightly more precise) results during low flow periods compared to previous releases. Please have a look at #162 for more information.

HydPy 6.1.1

The function create_projectstructure introduced in HydPy 6.1.0 was only of limited use because one still had to manually create the project's subdirectories (for example, projectroot/network/default). With HydPy 6.1.1, the different FileManager subclasses do this automatically if their base directory (for example, projectroot/network) is still empty. See (the end) of the documentation on the currentdir property for an example.

HydPy 6.1.0

With HydPy 6.1, we switch to a tighter release cycle. Compared to HydPy 6.0, there are a few breaking changes, but those require, at most, little client code adjustments (ask you type checker) and do not change any functionality. Hence, the transition from 6.0 to 6.1 should be smooth.

Here is the list of the most notable new features, documentation improvements, and fixes.

New features:

• The different file managers (for example, the ConditionManager) now print a note when they create or remove a directory. This feature can be turned off by setting printprogress to False.
• The new "script function" xml_validate allows to check if an XML workflow file fits to XSD schema definition file of the relevant HydPy version from the command line.
• Reading time series from or writing time series to NetCDF files via the related methods of the classes HydPy, Elements, and Nodes now works "automatically", meaning one has not to bother with manually opening and closing NetCDF reader and writer objects anymore.
• The option reprdigits now controls the number format for writing ASCII time series files. By default, all numbers are written in scientific notation (consistent with HydPy 6.0). If you set reprdigits to two, for example, all numbers are written in the decimal form with at most two decimal places
• Class ConditionManager now has the option prefix for changing the prefix of the automatically determined, time-dependent condition directory names. This feature eases, for example, to read the initial conditions of different ensemble members from separate directories
• All subelements of <conditions_io> in XML workflow files are now optional. Making the XML elements <inputpath> and <outputpath> optional was necessary because otherwise, the new element <prefix> would be generally impactless. Making the XML element <zip> optional was primarily for consistency but also for convenience, as its default value false should apply to most use cases.
• When using HydPy from the command line, you can now select two alternative error styles (single_line and splittable) to squeeze printed error messages in single lines, which is more easily processable for tools like Delft-FEWS. The old behaviour is now named multiline and the default of the new errorstyle option. For more information, please look at the documentation for the central hyd.py script.
• Function summarise_ncfile now automatically determines and prints the time grid of the analysed NetCDF file if it has the time reference attribute defined.
• The new functions create_projectstructure and check_projectstructure create and check the basic directory structure of a HydPy project. By default, checking also happens automatically when initialising class HydPy. This feature can be turned off by setting checkprojectstructure to False.

Documentation improvements:

• We added the Simulation section to the User Guide. It explains semi-comprehensively performing a simulation run via Python scripts or XML workflow files. It should build a bridge between the Quick Start's introductory remarks and the Reference Manual's extremely detailed explanations.
• The new Creating an Environment section of the Installation Instructions introduces to using virtual environments for the robust installation of, for example, different HydPy versions on the same computer.

Fixes:

• Reading initial conditions files with initial condition data already in RAM (for example, resulting from a previous simulation run) could result in unnecessary trimmings of the read values. The user was warned but had to solve the problem himself (for example, by deleting the data first and rereading the files afterwards). We fixed this issue by applying the "postponed trimming" strategy already implemented for resetting data via dictionaries.

HydPy 6.0.1

In HydPy 6.0.0, when iterating through the individual Timegrid instances handled by an instance of class Timegrids, eval_ was skipped. We fixed this in commit a32142d.

HydPy 6.0.0

We are happy to announce the release of HydPy 6.0, which comes with many documentation improvements, lots of new models, and, most importantly, a boost in flexibility in combining different model components.

Documentation improvements

We gave the documentation a new structure. Beginners will find it convenient that it now differentiates between a User Guide, which offers a basic introduction to HydPy, and a Reference Manual, which explains everything in depth. We included introductory tutorials (and plan to add tutorials that cover more complex use cases soon) based on the reworked HydPy-H-Lahn example project.

Submodel concept

Previously, one could easily combine models of different "families" in one project (for example, of HydPy-H(BV) and HydPy-L(ARSIM). However, no out-of-the-box mechanism existed for creating new combinations of different model components. Therefore, HydPy 6.0 introduces the Submodel concept. It allows, for example, to set up an HBV-like model that calculates evaporation similar to LARSIM.

So far, the new Submodel concept has been widely rolled out for meteorological factors like global radiation and evapotranspiration, and it at least partly covers topics like runoff concentration and flow formulas. We will modularise more model components in future HydPy versions. See the Submodel Graph in the new Model overview section for a complete list of the currently possible (and advisable) model compositions.

New models

At first sight, HydPy 6.0 comes with a multitude of new models. However, many are just new submodels that we extracted from already existing main models (the new Definitions section explains these terms in detail). For example, HydPy-Evap-PET-HBV96 stems from HydPy-H-HBV96.

Nevertheless, there are still some truly new models worth mentioning:

• HydPy-Evap-PET-AMBAV-1.0 calculates potential evapotranspiration in agreement with the AMBAV 1.0 model of the Deutscher Wetterdienst.
• HydPy-Dam contains three new models for modelling pumps and sluices.
• The new model family HydPy-GA is supposed to provide Green-Ampt infiltration models. The first available models implement the GARTO method, which can be used both as a main model to simulate infiltration excess overland flow for complex rainfall events independently and as a submodel to refine the volume-based runoff generation approach of HydPy-L(ARSIM).
• The new model family HydPy-SW1D introduces "hydrodynamic flood routing" to HydPy. It aims to robustly solve the 1-dimensional shallow water equations in river and channel networks. So, HydPy can now deal much better with catchments in lowland regions where hydraulic structures like weirs and sluices or tidal effects play a major role.

Model name changes

HydPy's model names were too cryptic, so we switched to more descriptive ones. For example, hland_v1 or HydPy-H-Land Version 1 is from now on hland_96 or HydPy-H-HBV96 (adoption of SMHI-IHMS-HBV96). You can find a complete list of all renamings here.

HydPy 6.0a0

This is the first pre-release of HydPy 6.0. Almost all of its new features have been implemented. Please look at issue #122 for the things still to come, which are mainly documentation-related.

We now support NumPy 1 and 2 (see issue #147). We included both in our test suites, but the now consistent usage of 64-bit integers (and floats) on all systems and all NumPy versions might cause unexpected inconveniences. Please open an issue if you encounter any problems.