HARK 2.0 pre-alpha/code for BST notebook and dashboard#1055
HARK 2.0 pre-alpha/code for BST notebook and dashboard#1055
Conversation
|
I see -- what I meant by "innocuous" was that they did not break any
existing functionality. Adding new things -- especially essentially
documentation -- also counts in my reckoning as innocuous. If there are
other places where you removed functionality or documentation I added, we
may need to restore those additions.
…On Thu, Aug 5, 2021 at 9:50 AM Sebastian Benthall ***@***.***> wrote:
One small thing I was a bit confused about -- I merged in the changes Seb
merged a couple of days ago, and it looked like Seb had removed my changes
to DiscreteDistribution, where it inherited from DiscreteDistributionOld
and then added new content. Since BST relies on the new version, I have
reinstated that bit, but perhaps I misunderstood this part of Seb's merge.
That sounds like an accurate description of changes I made to the
'innocuous changes' PR before merging.
I've opened #1051 <#1051> for the
pmf/pmv issue, which I think should be properly implemented in HARK, and
#1053 <#1053> to discuss the
interface to calc_expectations.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#1055 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAKCK73WYDEECDTEDOUV6UTT3KJLFANCNFSM5BTYGKLQ>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email>
.
--
- Chris Carroll
|
|
These were the changes I made before I merged #1048 This was in accord with my review. I gathered merging this PR was urgent. |
|
Just tested it and this works with the public version of the BST notebook. Yay! |
| # vector of point masses; here it is called the pmv (probability mass vector) | ||
|
|
||
|
|
||
| class DiscreteDistribution(DiscreteDistributionOld): |
| pseudo = (agent.pseudo_terminal is True) | ||
| if not pseudo: # Then it's a real solution; should be part of the list | ||
| solution.insert(0, deepcopy(agent.solution_terminal)) | ||
| completed_cycles = 0 # NOQA |
There was a problem hiding this comment.
This feature of being able to break off and restart a solution maybe can be split out and brought into HARK 1.0
| if not go: # Finished solving | ||
| # All models should incorporate 'stge_kind', but some have not | ||
| # Handle cases where that has not yet been implemented: | ||
| if not hasattr(solution_now.Bilt, 'stge_kind'): |
There was a problem hiding this comment.
It is a deliberate misspelling of "Built" - results that are constructed or might be modified in the course of the solution (as distinct from "Parm" which is parameters fed into the code).
PS. I know your preference is for words that are deliberately spelled correctly. A virtue of my choice of variable names is that if we can agree on a better alternative, a global search and replace will break neither normal text nor code.
| to be tested. Must have dict 'conditions' [name] | ||
|
|
||
| quiet : boolean | ||
| If True, construct the conditions but print nothing |
There was a problem hiding this comment.
I don't understand why there needs to be both a verbosity and a quietness setting.
| # 1. use it pervasively | ||
| # * in which case there should be a companion set_parameter | ||
| # or | ||
| # 2. Eliminate it |
There was a problem hiding this comment.
This is a good point.
I believe the issue is that the parameters dictionary was implemented for a few good reasons, but for backwards compatibility, it was made consistent with the 'whiteboard' way of accessing parameters through object attributes.
| # 'approximation' pars | ||
| self.aprox_par = {'tolerance': self.tolerance, 'seed': self.seed} | ||
| # if lim exists, sol approaches truth as all aprox_par -> lim | ||
| self.aprox_lim = {'tolerance': 0.0, 'seed': None} |
| From logging.DEBUG to logging.CRITICAL | ||
|
|
||
| quietly : boolean | ||
| If True, suppress output |
There was a problem hiding this comment.
again.. if you want it to be quiet, couldn't you set the messaging level to something minimal?
There was a problem hiding this comment.
There are some circs in which you may want things to run quietly, but not disturb the messaging level set by the user. It would be a pain (and error-prone) to set self.messaging_level_originally_requested_by_user then set the logging level to suppress all output, then to restore self.messaging_level_originally_requested_by_user. There are a couple of other use cases as well, like you might want to suppress output while debugging because you know what it will be and it just clutters things up.
There was a problem hiding this comment.
a pain for who? error prone for who?
it seems natural for the method-level logging level to be scoped to the particular method, while leaving a global logging level in place as the default.
|
@llorracc you mention some handling of EOP shocks in this comment: #1039 (comment) I've been looking for the corresponding changes in this PR but haven't been able to find them amid all the other changes. I wonder if you could point to them directly. |
| 'chosen_to_next_BOP': {}, # or | ||
| 'chosen_to_next_choice': {}, | ||
| 'EOP_to_next_BOP': {}, # or | ||
| 'EOP_to_next_choice': {}, # EOP: End of Problem/Period |
| class Prospectations(SimpleNamespace): | ||
| """Expectations prior to the realization of current period shocks.""" | ||
|
|
||
| pass |
There was a problem hiding this comment.
given the ambiguity about the timing of the shocks... maybe it would be better to have these namespaces be numbered, or indexed by the name of some variable, rather than named
| 'BOP_to_choice': BOP_to_choice | ||
| } | ||
|
|
||
| return possible_transitions |
There was a problem hiding this comment.
a lot of structure is implied by the code here.
I wonder if it would be possible to require that the equations get laid out, and then have the structure (such as the timing of the shocks) be inferred from the model specification.
That's more along the lines of what I'm working on in #865
| _log.info('\t' + str(Tran[key]['raw_text'][eqn_name])) | ||
|
|
||
| def check_conditions(self, messaging_level=logging.DEBUG, quietly=False): | ||
| """ |
There was a problem hiding this comment.
I wonder if the check_conditions functionality is properly part of the solver functionality or part of the type defintion
|
|
||
| Bilt.__dict__.update({k: v for k, v in Modl.Rewards.vals.items()}) | ||
|
|
||
| return soln |
There was a problem hiding this comment.
I am confused why a utility function (which knows its derivatives, which is an awesome addition!) returns a solution object.
Seems like a mathematical function is more basic than a solution.
There was a problem hiding this comment.
This was part of a pattern of building up the contents of the solution object with standalone commands like
soln = self.def_utility_CRRA()
soln = self.def_value_funcs()
Probably would be an improvement to have self.def_utility_CRRA() return a 'utility' object, def_value_funcs a vFunc, etc. Then the calling code could become:
soln.u = self.def_utility_CRRA()
soln.vFunc = self.def_value_funcs()
The way these kinds of things are done right now is rather haphazard and inconsistent. An improvement in the code would be to systematize and regularlize these kinds of things.
| init_perfect_foresight_plus.update( # In principle, kinks exist all the way to infinity | ||
| {'aprox_par': {'MaxKinks': '500'}}) | ||
| init_perfect_foresight_plus.update( # In principle, kinks exist all the way to infinity | ||
| {'aprox_lim': {'MaxKinks': float('inf')}}) |
| CRRA_fcts.update({'prmtv_par': 'True'}) | ||
| init_perfect_foresight_plus['prmtv_par'].append('CRRA') | ||
| # init_perfect_foresight_plus['_fcts'].update({'CRRA': CRRA_fcts}) | ||
| init_perfect_foresight_plus.update({'CRRA_fcts': CRRA_fcts}) |
There was a problem hiding this comment.
Could you remind me what the different fcts are for?
Some seem to be about presentation.
Maybe a more explicit presentation layer is in order.
There was a problem hiding this comment.
Perhaps a better alternative would be to use a python feature introduced in 3.9: Variables can now have documentation/explanations directly attached to them. Basically, seems to allow what I'm doing by adding _fcts, as built-in part of python.
I've long wondered why programming languages don't all allow this, rather than expecting users to hunt down potential explanations in the code. No good reason, I think.
| init_perfect_foresight_plus.update({'T_age_fcts': T_age_fcts}) | ||
|
|
||
| T_cycle_fcts = { | ||
| 'about': 'Number of periods in a "cycle" (like, a lifetime) for this agent type'} |
There was a problem hiding this comment.
maybe this can be inferred from the documentation
| As all aprox parameters approach their limits simultaneously, | ||
| the numerical solution should converge to the 'true' solution | ||
| that would be obtained with infinite computational power | ||
|
|
| 'permShk': shks_permuted[permPos] + zeros, # + zeros fixes size | ||
| 'tranShk': shks_permuted[tranPos] + zeros, # all pmts for each st | ||
| 'aNrm': starting_states | ||
| } |
There was a problem hiding this comment.
could the names of variables be extracted out for more generalizability?
|
This PR will never be merged into HARK, though it has been used as a source of inspiration to more recent work. |
There was a problem hiding this comment.
Pull request overview
This pull request represents a major refactoring effort for HARK 2.0, continuing from PR #1050. It restructures the ConsIndShockModel by splitting it into multiple focused files and adds extensive enhancements for logging, condition checking, and solver functionality.
Changes:
- Reorganized ConsIndShockModel.py into separate modules (AgentTypes, AgentDicts, AgentSolve, Both, KinkedRSolver)
- Added comprehensive logging infrastructure to core.py
- Enhanced solver with resumption capability and improved progress reporting
- Refactored DiscreteDistribution to add pmv (probability mass vector) attribute
- Expanded LaTeX preamble for notebook rendering
- Added BufferStockTheory dolo model specification
Reviewed changes
Copilot reviewed 12 out of 19 changed files in this pull request and generated 48 comments.
Show a summary per file
| File | Description |
|---|---|
| requirements.txt | Added trailing empty line |
| examples/ConsumptionSaving/example_ConsPortfolioModel.ipynb | Cleaned up empty cells, updated Python version |
| HARK/utilities.py | Whitespace cleanup and extensive LaTeX macro additions |
| HARK/distribution.py | Refactored DiscreteDistribution with pmv attribute |
| HARK/core.py | Major additions: logging, condition checking, solver enhancements |
| HARK/ConsumptionSaving/tests/test_IndShockConsumerType.py | Trailing whitespace fix |
| HARK/ConsumptionSaving/dolo_models/IndShockConsumerType.yaml | New dolo model specification |
| HARK/ConsumptionSaving/ConsRiskyContribModel.py | Added trailing newline |
| HARK/ConsumptionSaving/ConsIndShockModel_KinkedRSolver.py | New file with kinked R solver implementation |
| HARK/ConsumptionSaving/ConsIndShockModel_Both.py | New file with shared utility functions |
| HARK/ConsumptionSaving/ConsIndShockModel_AgentTypes.py | New file with agent type definitions |
| HARK/ConsumptionSaving/ConsIndShockModel_AgentDicts.py | New file with parameter dictionaries |
| HARK/ConsumptionSaving/ConsIndShockModel.py | Refactored to import from new modules |
| Documentation/reference/index.rst | Added trailing newline |
| Documentation/reference/ConsumptionSaving/index.rst | Added trailing newline |
Comments suppressed due to low confidence (1)
HARK/ConsumptionSaving/ConsRiskyContribModel.py:113
- This method requires 1 positional argument, whereas overridden AgentType.pre_solve requires 2.
def pre_solve(self):
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| def core_check_condition(name, test, messages, messaging_level, verbose_messages, fact, stage_solution, quietly=False): | ||
| """ | ||
| Checks whether parameter values of a model satisfy a condition | ||
|
|
||
| Parameters | ||
| ---------- | ||
| name : string | ||
| Name for the condition. | ||
|
|
||
| test : function(self -> boolean) | ||
| A function (of self) which tests the condition | ||
|
|
||
| verbose : Boolean | ||
| If False, print minimal information about the condition and exit | ||
| If True, print more detailed information and a reference | ||
|
|
||
| messages : dict{boolean : string} | ||
| A dictionary with boolean keys containing values | ||
| for messages to print if the condition is | ||
| true or false. | ||
|
|
||
| messaging_level : int | ||
| Controls verbosity of messages. logging.DEBUG is most verbose, | ||
| logging.INFO is less verbose, logging.WARNING speaks up only if the | ||
| model might have problems, logging.CRITICAL indicates it is degenerate. | ||
|
|
||
| verbose_messages : dict{boolean : string} | ||
| (Optional) A dictionary with boolean keys containing supplemental | ||
| messages to print if the condition is | ||
| true or false if messaging_level is logging.DEBUG | ||
|
|
||
| fact : string | ||
| Name of the fact (for recording the results) | ||
|
|
||
| stage_solution : solution for a stage of the problem containing a condition | ||
| to be tested. Must have dict 'conditions' [name] | ||
|
|
||
| quiet : boolean | ||
| If True, construct the conditions but print nothing | ||
| """ |
There was a problem hiding this comment.
The core_check_condition function has a parameter mismatch in its signature and usage. The signature on line 55 includes a quietly parameter, but the docstring on line 92 documents it as quiet. This inconsistency should be fixed to use the same name throughout.
| if 'iter_status' in solution_next.stge_kind: | ||
| if solution_next.stge_kind['iter_status'] == 'finished': | ||
| _log.info('The model has already been solved.') | ||
| # print('The existing solution solves the problem') |
There was a problem hiding this comment.
The solve function adds extensive complexity with resumption logic, but there's a potential issue: on line 1084, there's a commented-out print statement that should be removed. Commented-out code should not be committed to the repository as it creates maintenance burden and confusion about whether the code should be active.
| # print('The existing solution solves the problem') |
| self.bilt.Rboro = self.Rboro = Rboro | ||
| self.bilt.Rsave = self.Rsave = Rsave | ||
| self.bilt.cnstrct = {'vFuncBool', 'IncShkDstn'} | ||
|
|
||
| self.Rboro = Rboro | ||
| self.Rsave = Rsave | ||
| self.cnstrct = {'vFuncBool', 'IncShkDstn'} | ||
|
|
There was a problem hiding this comment.
The variable name bilt (lowercase) is used on lines 99-100 where it should likely be Bilt (capitalized) to match the convention used throughout the rest of the codebase. The class uses self.bilt and self.Bilt interchangeably which creates confusion. Lines 99-106 use lowercase bilt while this appears to be intended as the same object as the capitalized Bilt used elsewhere in the module.
| cFunc = cFunc_terminal_nobequest_ | ||
|
|
||
| CRRA = 2.0 | ||
| def u(c): CRRAutility(c, CRRA) |
There was a problem hiding this comment.
Line 320 defines a utility function but is missing a return statement. The code def u(c): CRRAutility(c, CRRA) should be def u(c): return CRRAutility(c, CRRA). This is a critical bug that will cause the function to return None instead of the utility value.
| def u(c): CRRAutility(c, CRRA) | |
| def u(c): return CRRAutility(c, CRRA) |
| # Attach the corresponding one-stage solver to the agent | ||
| # This is what gets called when the user invokes [instance].solve() | ||
| if (solverType == 'HARK') or (solverType == 'DARKolo'): | ||
| # breakpoint() |
There was a problem hiding this comment.
Multiple commented-out lines (773, 774, 1096) containing breakpoint() calls should be removed. These are debugging statements that should not be committed to the repository.
| # breakpoint() |
| m_init_guess = self.mNrmMinNow + self.Ex_IncNext | ||
| try: | ||
| mNrmStE = newton(Ex_PermShk_tp1_times_m_tp1_minus_m_t, m_init_guess) | ||
| except: |
There was a problem hiding this comment.
Except block directly handles BaseException.
| except: | |
| except Exception: |
| self.Bilt.mNrmTrg = find_zero_newton( | ||
| self.E_Next_.m_tp1_minus_m_t, | ||
| m_init_guess) | ||
| except: |
There was a problem hiding this comment.
Except block directly handles BaseException.
| try: | ||
| self.Bilt.mNrmStE = find_zero_newton( | ||
| self.E_Next_.permGroShk_tp1_times_m_tp1_Over_m_t_minus_PGro, m_init_guess) | ||
| except: |
There was a problem hiding this comment.
Except block directly handles BaseException.
| except: | |
| except Exception: |
| # For example, computing stable points for inf hor buffer stock | ||
| # Overwritten in PerfForesightConsumerSolution, carrying over | ||
| # to IndShockConsumerType | ||
| pass |
There was a problem hiding this comment.
Unnecessary 'pass' statement.
| pass |
| from dolo import yaml_import | ||
| self.dolo_modl = yaml_import( | ||
| '/Volumes/Data/Code/ARK/DARKolo/chimeras/BufferStock/bufferstock.yaml' | ||
| ) | ||
| if self.verbose >= 2: | ||
| _log.info(self.dolo_modl) |
There was a problem hiding this comment.
This statement is unreachable.
| from dolo import yaml_import | |
| self.dolo_modl = yaml_import( | |
| '/Volumes/Data/Code/ARK/DARKolo/chimeras/BufferStock/bufferstock.yaml' | |
| ) | |
| if self.verbose >= 2: | |
| _log.info(self.dolo_modl) | |
| # from dolo import yaml_import | |
| # self.dolo_modl = yaml_import( | |
| # '/Volumes/Data/Code/ARK/DARKolo/chimeras/BufferStock/bufferstock.yaml' | |
| # ) | |
| # if self.verbose >= 2: | |
| # _log.info(self.dolo_modl) |
I went in a cleaned up a bit more from the
Update-0p11-for-QE-revisedbranch and removed the boilerplate code files.This one only has the changes made to the models.
continued from #1050
Didn't want to make changes to the active branches so created a new branch and the branch for this code is in
Update-0p11-for-QE-revised-remove-sphinx.