Skip to content

[Bug] OsemosysClass.RYE() inserts a spurious empty dict for the 'EmisId' key, corrupting the emission parameter index #423

@tejassinghbhati

Description

@tejassinghbhati

Summary

OsemosysClass.RYE() in API/Classes/Case/OsemosysClass.py contains a structural logic error that causes a spurious 'EmisId': {} entry to be silently injected into the returned data structure on every call. This is a silent data-corruption bug — it produces no exception, but the malformed intermediate dict is immediately consumed by downstream methods (update_RYE in UpdateCaseClass.py and the OSeMOSYS solver input-file generators), potentially leading to missing emission parameter rows in the solver input or incorrect constraint generation.

What did you expect? (optional)

RYE() should return a dict keyed only by year strings (e.g. '2020', '2021'), matching the behaviour of every other pivot method in the class (RYT, RYC, RYS, RYTs, etc.). The 'EmisId' string is an index column — it should never appear as a year-level key in the output.

How can we reproduce it?

Run this directly in a PowerShell terminal from the repo root — no case or database needed:

@"
rye_raw = {'AnnualExogenousEmission': {'SC_0': [{'EmisId': 'CO2', '2020': 0.0, '2021': 0.5}]}}
RYE = {}
for param, obj1 in rye_raw.items():
    RYE[param] = {}
    for sc, array in obj1.items():
        RYE[param][sc] = {}
        for o in array:
            for year, val in o.items():
                if year not in RYE[param][sc]:
                    RYE[param][sc][year] = {}
                if year != 'EmisId':
                    RYE[param][sc][year][o['EmisId']] = val
keys = list(RYE['AnnualExogenousEmission']['SC_0'].keys())
print('Keys found:', keys)
print('Expected:  [2020, 2021]')
print('Bug confirmed:', 'EmisId' in RYE['AnnualExogenousEmission']['SC_0'])
"@ | python

Actual output:

Keys found: ['EmisId', '2020', '2021']
Expected:  [2020, 2021]
Bug confirmed: True

Root cause — inverted guard order in RYE() (lines 615–617):

# BUGGY (RYE) — dict init runs unconditionally, even for 'EmisId'
if year not in RYE[param][sc]:
    RYE[param][sc][year] = {}      # ← inserts 'EmisId': {} phantom entry
if (year != 'EmisId'):
    RYE[param][sc][year][o['EmisId']] = val

# CORRECT (all other methods e.g. RYT line 541, RYC line 601)
if (year != 'TechId'):             # ← guard FIRST
    if year not in RYT[param][sc]:
        RYT[param][sc][year] = {}
    RYT[param][sc][year][o['TechId']] = val

Running Select-String across the file confirms RYE is the only method with this inverted order — every other pivot method places the ID-key guard before the dict initialisation.

Environment (optional)

  • Python 3.x
  • Flask API (API/app.py)
  • All platforms (Windows / Linux)
  • Present on latest upstream mainOsemosysClass.py has no prior fix commits

Logs or screenshots (optional)

Image Image Image Image

Related issue, PR, or discussion (optional)

No response

Proposed track

{"Track" => "Stability"}

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    Status

    In progress

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions