mysterious NULL optimization result

[ladderup]

Hi,

In the attachment you’ll find an ESDLfile that I showed during the DesignTool Kit support meeting [20251106].

There are 3 files:

File 1 – not duplicated ESDLfile.
This ESDLfile was put together using a Python script. The Python script only uses the PyESDL library (https://pypi.org/project/pyESDL/). The ESDL objects are created with only 1 port (InPort if consumer, OutPort if producer). The 'opposite' port is created during duplication (as I explained during the meeting).
All capacities and capacity units, supply carriers (the return carrier is created during duplication), profiles, and multipliers are created in the Python script using PyESDL library and added to the objects and the ESDL file.

File 2 – duplicated ESDLfile
note: during duplication it sometimes happens that the port is duplicated incorrectly, about 50% of the time [spooky deterministic problem].
Duplication was done in the normal DesignTool Kit interface (https://nwn-design-toolkit.nl/).

File 3 – result file.
This file is calculated in about 9 minutes using Gurobi.
This is way faster the the non commercial solver.
But the result is --- NULL ---

Where does it go wrong?
The obvious suspect is of course that I generated the file using PyESDL. But in the NWN interface nothing looks wrong with file 1. Carriers, units, multipliers, and profiles are correct and visible.

ESDLexport.txt
ESDLexport-return.txt
ESDLexport-return-result.txt

Kind regards,
Andries

[samvanderzwan]

First I would like to note that we do not support esdl with are created via pyesdl.
You also note that you create assets with only one port, this is also not supported functionality.

I run the validator and i get 7 pages with errors and warnings. I used the optimization validation scheme.

this is one example for one assets:
Pijnacker Zuid max piek

ERROR - Asset is not connected as producer. Check if ports or carriers have been added properly (InPort with return carrier assigned; OutPort with supply carrier assigned; Temperature of OutPort should be greater than Temperature of InPort)

[port] should contain a [InPort] ([name] = 'In'), but not found.

ERROR - Investment Costs [EUR/MW] or [EUR/kW] is incomplete or incorrect

[costInformation.investmentCosts.value] should be defined, but is unset.

ERROR - Installation Costs [EUR] is incomplete or incorrect

[costInformation.installationCosts.value] should be defined, but is unset.

WARNING - Fixed Operational Costs [EUR/MW/yr] or [EUR/kW/yr] is incomplete or incorrect

[costInformation.fixedOperationalCosts.value] should be defined, but is unset.
[perTimeUnit] should be [YEAR], but is unset.

WARNING - Fixed Maintenance Costs [EUR/MW/yr] or [[EUR/kW/yr] is incomplete or incorrect

[costInformation.fixedMaintenanceCosts.value] should be defined, but is unset.
[perTimeUnit] should be [YEAR], but is unset.

So your esdl is corrupt.
@KobusVanRooyen if you want you can have a look, but I guess the esdl first needs to be fixed.

[KobusVanRooyen]

Thanks @samvanderzwan I will have a quick check later this morning

[ladderup]

Thanks @samvanderzwan,

I'm not familiar with the 'optimization validation scheme'
It is not with the DesignTool Kit interface (https://nwn-design-toolkit.nl/).
In the test environment I can find it.
Thanks for enlighten me.

This gives me great clues how to fix the problem.
My first judgment is the OutPort naming.
The duplicator (not intended for duplicating ports) creates a OutPort called 'ret_port', But is shoud be called 'Out'.

[KobusVanRooyen]

@ladderup I used the supply network in https://nwn-design-toolkit.nl/editor# and then I used the duplicator. After that I used the validator and I get a very long list of esdl errors. So it does seem there are esdl issues (connections etc) to be resolved.

@samvanderzwan and @ladderup please be careful in using the validator in https://test.nwn-design-toolkit.nl/editor#. This is the developer's test environment and the validators (Optimizer validation for example) is still a work in progress and does not necessary match what is needed/catered for in https://nwn-design-toolkit.nl/editor#.

[KobusVanRooyen]

@ladderup I am not sure why you did not get errors from the validator in https://nwn-design-toolkit.nl/editor#. Can you maybe double check this? If you follow the same steps I followed in my previous message.

[ladderup]

@KobusVanRooyen,
I repeated you steps, but 'my' validator (PoC Validation Schema) says, 'Well Done'. So no errors at all.
So I can not use it for debugging de pyESDL files.
But with the 'optimization validation scheme' only available in de test environment, I can....
Perhaps I can show you the situation in a short meeting.

I managed to fix all the errors with the 'optimization validation scheme'. The big 'mistake' of my script was, no appending the OutPort to de Heatdemand assets (and InPort to the Heatproducer assets). The duplicator then creates an OutPort but it has the wrong name ("ret_port" instate of "Out").

Now the errors are gone, I can use the Gurobi optimization.
Calculation time 10min.
Calculation time MESIDO without Gurobi was >1 hour. progress_message: Optimization task 3.0 out of 4.0 has completed

Result file attached.

MAX_Outport test13 - return.txt

[KobusVanRooyen]

@ladderup I see if I use the network with supply and return pipes in, then I also do not get any error via the validator in the DTK prod environment. But I do not think this is a big issue at the moment, since the validator in the DTK test env (will soon replace the validator in the prod env) does give errors.

[KobusVanRooyen]

@samvanderzwan please have a look at my last comment so that you can decide if you agree (then close this issue) or else pass this on to Cheng-Kai if you want to.

[cwang39403]

@ladderup I tested your ESDL on Prod and Test environments and checked the logs. The ESDL is not compatible with the version of Validator on Prod, and as indicated by @KobusVanRooyen, this will soon be replaced the version on Test. We will give you an update in the support meeting.

The improvement can be made in this case is to show some messages such as "Invalid ESDL" in the frontend, I will take care of this in the new release too.

@KobusVanRooyen, I repeated you steps, but 'my' validator (PoC Validation Schema) says, 'Well Done'. So no errors at all. So I can not use it for debugging de pyESDL files. But with the 'optimization validation scheme' only available in de test environment, I can.... Perhaps I can show you the situation in a short meeting.

I managed to fix all the errors with the 'optimization validation scheme'. The big 'mistake' of my script was, no appending the OutPort to de Heatdemand assets (and InPort to the Heatproducer assets). The duplicator then creates an OutPort but it has the wrong name ("ret_port" instate of "Out").

Now the errors are gone, I can use the Gurobi optimization. Calculation time 10min. Calculation time MESIDO without Gurobi was >1 hour. progress_message: Optimization task 3.0 out of 4.0 has completed

Result file attached.

MAX_Outport test13 - return.txt