Error: duplicate keys found

[elliottslaughter]

I'm getting the following error with a large application I'm running. I'd be happy to send repro instructions privately, but so far I'm just not even sure where to even look for problems. Has anyone ever seen anything like this?

Fatal Error: duplicate keys found for _ZN6scitbx2af12boost_python12flex_wrapperIdN5boost6python19return_value_policyINS4_24copy_non_const_referenceENS4_21default_call_policiesEEEE10getitem_1dERNS0_5versaIdNS0_9flex_gridINS0_5smallIlLm10EEEEEEEl _ZN5boost6python6detail12caller_arityILj2EE4implIPFRdRN6scitbx2af5versaIdNS7_9flex_gridINS7_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3IS5_SE_lEEEclEP7_objectSQ_ _ZN5boost6python7objects23caller_py_function_implINS0_6detail6callerIPFRdRN6scitbx2af5versaIdNS7_9flex_gridINS7_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3IS5_SE_lEEEEEclEP7_objectSR_

(...and it continues for several pages...)

I thought the problem might be related to my use of shared libraries, and I suspected that there might be duplicate symbols included in multiple shared libraries all being loaded into the same execution. But I've tried that scenario in a smaller reproducer, and it works fine. So at the moment I don't have any ideas for what could be causing this.

For background, the application is running in Python, and Python has not been compiled with Byfl. Python loads one or more shared libraries---of which, if I'm doing things right, one and only one has been compiled with Byfl---and then runs some functions in those shared libraries. I've tested a similar scenario in my reproducer (main compiled with the system compiler, libraries compiled with Byfl) and haven't been able to get it to show any problems.

I'm on Ubuntu 16.04 with LLVM 6.0 (with the appropriate Byfl branch). I can provide repro instructions privately if desired.

Thanks.

[spakin]

Is the error repeatable across builds/runs? Specifically, does it name the same function each time? The error comes from a Byfl function that assigns a unique ID to each (possibly non-unique) function name. IDs are chosen such that collisions should be extremely unlikely. I'm thinking it's more likely that the ID-to-function name map is getting corrupted than that an ID conflict is genuinely occurring. Alas, I don't have any good ideas how to test that hypothesis. Maybe run the code through Valgrind and see if it balks at any of it?

[elliottslaughter]

Runs within the same build produce identical results (modulo cosmetic differences in application output). I'll kick off a new build and see if that makes any difference.

[elliottslaughter]

And a completely fresh clone and build produces the same result. So yeah, it looks deterministic.

[spakin]

Okay, thanks for testing that. Is Valgrind happy with the code, or does it balk at something before Byfl blows up? That'll help distinguish between Byfl's ID-to-function table having been corrupted and a legitimate but unanticipated naming situation (e.g., multiple functions in the same file with the exact same name).

[elliottslaughter]

I'll try to check this, but it looks like I may need a debug build of Python in order to make it play nice with Valgrind. The application is scripted in Python at the code I'm compiling with Byfl is being loaded in as a shared library.

https://stackoverflow.com/q/20112989/188046

[spakin]

Here's a simpler diagnostic that might be informative: Change the error message in lib/byfl/byfl.cpp, line 75 also to output keyID. If keyID is something fishy like 0 or FFF…F, that might imply data corruption. Otherwise, we'll need to look for some other possible culprit.

[elliottslaughter]

Ok, here's the output:

Fatal Error: duplicate keys found for _ZN6scitbx2af12boost_python12flex_wrapperIdN5boost6python19return_value_policyINS4_24copy_non_const_referenceENS4_21default_call_policiesEEEE10getitem_1dERNS0_5versaIdNS0_9flex_gridINS0_5smallIlLm10EEEEEEEl _ZN5boost6python6detail12caller_arityILj2EE4implIPFRdRN6scitbx2af5versaIdNS7_9flex_gridINS7_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3IS5_SE_lEEEclEP7_objectSQ_ _ZN5boost6python7objects23caller_py_function_implINS0_6detail6callerIPFRdRN6scitbx2af5versaIdNS7_9flex_gridINS7_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3IS5_SE_lEEEEEclEP7_objectSR_ _ZN6scitbx22positive_getitem_indexIlmEET0_RKT_RKS1_bPKc _ZN6scitbx2af12boost_python12flex_wrapperINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEN5boost6python19return_value_policyINSA_24copy_non_const_referenceENSA_21default_call_policiesEEEE10getitem_1dERNS0_5versaIS8_NS0_9flex_gridINS0_5smallIlLm10EEEEEEEl _ZN5boost6python6detail12caller_arityILj2EE4implIPFRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERN6scitbx2af5versaISA_NSD_9flex_gridINSD_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3ISB_SK_lEEEclEP7_objectSW_ _ZN5boost6python7objects23caller_py_function_implINS0_6detail6callerIPFRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERN6scitbx2af5versaISA_NSD_9flex_gridINSD_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3ISB_SK_lEEEEEclEP7_objectSX_ _ZN6scitbx22positive_getitem_indexIlmEET0_RKT_RKS1_bPKc _ZN6scitbx2af12boost_python12flex_wrapperIN5cctbx4xray9scattererIdNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESB_EEN5boost6python25return_internal_referenceILm1ENSE_21default_call_policiesEEEE10getitem_1dERNS0_5versaISC_NS0_9flex_gridINS0_5smallIlLm10EEEEEEEl _ZN5boost6python6detail12caller_arityILj2EE4implIPFRN5cctbx4xray9scattererIdNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESD_EERN6scitbx2af5versaISE_NSH_9flex_gridINSH_5smallIlLm10EEEEEEElENS0_25return_internal_referenceILm1ENS0_21default_call_policiesEEENS_3mpl7vector3ISF_SO_lEEEclEP7_objectSZ_ _ZN5boost6python7objects23caller_py_function_implINS0_6detail6callerIPFRN5cctbx4xray9scattererIdNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESD_EERN6scitbx2af5versaISE_NSH_9flex_gridINSH_5smallIlLm10EEEEEEElENS0_25return_internal_referenceILm1ENS0_21default_call_policiesEEENS_3mpl7vector3ISF_SO_lEEEEEclEP7_objectS10_ _ZN6scitbx22positive_getitem_indexIlmEET0_RKT_RKS1_bPKc _ZN6scitbx2af12boost_python12flex_wrapperINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEN5boost6python19return_value_policyINSA_24copy_non_const_referenceENSA_21default_call_policiesEEEE10getitem_1dERNS0_5versaIS8_NS0_9flex_gridINS0_5smallIlLm10EEEEEEEl _ZN5boost6python6detail12caller_arityILj2EE4implIPFRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERN6scitbx2af5versaISA_NSD_9flex_gridINSD_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3ISB_SK_lEEEclEP7_objectSW_ _ZN5boost6python7objects23caller_py_function_implINS0_6detail6callerIPFRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERN6scitbx2af5versaISA_NSD_9flex_gridINSD_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3ISB_SK_lEEEEEclEP7_objectSX_ _ZN6scitbx22positive_getitem_indexIlmEET0_RKT_RKS1_bPKc _ZN6scitbx2af12boost_python12flex_wrapperIN5cctbx4xray9scattererIdNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESB_EEN5boost6python25return_internal_referenceILm1ENSE_21default_call_policiesEEEE10getitem_1dERNS0_5versaISC_NS0_9flex_gridINS0_5smallIlLm10EEEEEEEl _ZN5boost6python6detail12caller_arityILj2EE4implIPFRN5cctbx4xray9scattererIdNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESD_EERN6scitbx2af5versaISE_NSH_9flex_gridINSH_5smallIlLm10EEEEEEElENS0_25return_internal_referenceILm1ENS0_21default_call_policiesEEENS_3mpl7vector3ISF_SO_lEEEclEP7_objectSZ_ _ZN5boost6python7objects23caller_py_function_implINS0_6detail6callerIPFRN5cctbx4xray9scattererIdNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESD_EERN6scitbx2af5versaISE_NSH_9flex_gridINSH_5smallIlLm10EEEEEEElENS0_25return_internal_referenceILm1ENS0_21default_call_policiesEEENS_3mpl7vector3ISF_SO_lEEEEEclEP7_objectS10_ _ZN6scitbx22positive_getitem_indexIlmEET0_RKT_RKS1_bPKc _ZN6scitbx2af12boost_python12flex_wrapperIdN5boost6python19return_value_policyINS4_24copy_non_const_referenceENS4_21default_call_policiesEEEE10getitem_1dERNS0_5versaIdNS0_9flex_gridINS0_5smallIlLm10EEEEEEEl _ZN5boost6python6detail12caller_arityILj2EE4implIPFRdRN6scitbx2af5versaIdNS7_9flex_gridINS7_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3IS5_SE_lEEEclEP7_objectSQ_ _ZN5boost6python7objects23caller_py_function_implINS0_6detail6callerIPFRdRN6scitbx2af5versaIdNS7_9flex_gridINS7_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3IS5_SE_lEEEEEclEP7_objectSR_ _ZN6scitbx22positive_getitem_indexIlmEET0_RKT_RKS1_bPKc _ZN6scitbx2af12boost_python12flex_wrapperIdN5boost6python19return_value_policyINS4_24copy_non_const_referenceENS4_21default_call_policiesEEEE10getitem_1dERNS0_5versaIdNS0_9flex_gridINS0_5smallIlLm10EEEEEEEl _ZN5boost6python6detail12caller_arityILj2EE4implIPFRdRN6scitbx2af5versaIdNS7_9flex_gridINS7_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3IS5_SE_lEEEclEP7_objectSQ_ _ZN5boost6python7objects23caller_py_function_implINS0_6detail6callerIPFRdRN6scitbx2af5versaIdNS7_9flex_gridINS7_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3IS5_SE_lEEEEEclEP7_objectSR_ _ZN6scitbx22positive_getitem_indexIlmEET0_RKT_RKS1_bPKc _ZN6scitbx2af12boost_python12flex_wrapperIdN5boost6python19return_value_policyINS4_24copy_non_const_referenceENS4_21default_call_policiesEEEE10getitem_1dERNS0_5versaIdNS0_9flex_gridINS0_5smallIlLm10EEEEEEEl _ZN5boost6python6detail12caller_arityILj2EE4implIPFRdRN6scitbx2af5versaIdNS7_9flex_gridINS7_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3IS5_SE_lEEEclEP7_objectSQ_ _ZN5boost6python7objects23caller_py_function_implINS0_6detail6callerIPFRdRN6scitbx2af5versaIdNS7_9flex_gridINS7_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3IS5_SE_lEEEEEclEP7_objectSR_ _ZN6scitbx22positive_getitem_indexIlmEET0_RKT_RKS1_bPKc _ZN6scitbx2af12boost_python12flex_wrapperIdN5boost6python19return_value_policyINS4_24copy_non_const_referenceENS4_21default_call_policiesEEEE10getitem_1dERNS0_5versaIdNS0_9flex_gridINS0_5smallIlLm10EEEEEEEl _ZN5boost6python6detail12caller_arityILj2EE4implIPFRdRN6scitbx2af5versaIdNS7_9flex_gridINS7_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3IS5_SE_lEEEclEP7_objectSQ_ _ZN5boost6python7objects23caller_py_function_implINS0_6detail6callerIPFRdRN6scitbx2af5versaIdNS7_9flex_gridINS7_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3IS5_SE_lEEEEEclEP7_objectSR_ _ZN6scitbx22positive_getitem_indexIlmEET0_RKT_RKS1_bPKc _ZN6scitbx2af12boost_python12flex_wrapperIN5cctbx4xray9scattererIdNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESB_EEN5boost6python25return_internal_referenceILm1ENSE_21default_call_policiesEEEE10getitem_1dERNS0_5versaISC_NS0_9flex_gridINS0_5smallIlLm10EEEEEEEl _ZN5boost6python6detail12caller_arityILj2EE4implIPFRN5cctbx4xray9scattererIdNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESD_EERN6scitbx2af5versaISE_NSH_9flex_gridINSH_5smallIlLm10EEEEEEElENS0_25return_internal_referenceILm1ENS0_21default_call_policiesEEENS_3mpl7vector3ISF_SO_lEEEclEP7_objectSZ_ _ZN5boost6python7objects23caller_py_function_implINS0_6detail6callerIPFRN5cctbx4xray9scattererIdNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESD_EERN6scitbx2af5versaISE_NSH_9flex_gridINSH_5smallIlLm10EEEEEEElENS0_25return_internal_referenceILm1ENS0_21default_call_policiesEEENS_3mpl7vector3ISF_SO_lEEEEEclEP7_objectS10_ _ZN6scitbx22positive_getitem_indexIlmEET0_RKT_RKS1_bPKc _ZN6scitbx2af12boost_python12flex_wrapperIdN5boost6python19return_value_policyINS4_24copy_non_const_referenceENS4_21default_call_policiesEEEE10getitem_1dERNS0_5versaIdNS0_9flex_gridINS0_5smallIlLm10EEEEEEEl _ZN5boost6python6detail12caller_arityILj2EE4implIPFRdRN6scitbx2af5versaIdNS7_9flex_gridINS7_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3IS5_SE_lEEEclEP7_objectSQ_ _ZN5boost6python7objects23caller_py_function_implINS0_6detail6callerIPFRdRN6scitbx2af5versaIdNS7_9flex_gridINS7_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3IS5_SE_lEEEEEclEP7_objectSR_ _ZN6scitbx22positive_getitem_indexIlmEET0_RKT_RKS1_bPKc _ZN6scitbx2af12boost_python12flex_wrapperIdN5boost6python19return_value_policyINS4_24copy_non_const_referenceENS4_21default_call_policiesEEEE10getitem_1dERNS0_5versaIdNS0_9flex_gridINS0_5smallIlLm10EEEEEEEl _ZN5boost6python6detail12caller_arityILj2EE4implIPFRdRN6scitbx2af5versaIdNS7_9flex_gridINS7_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3IS5_SE_lEEEclEP7_objectSQ_ _ZN5boost6python7objects23caller_py_function_implINS0_6detail6callerIPFRdRN6scitbx2af5versaIdNS7_9flex_gridINS7_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3IS5_SE_lEEEEEclEP7_objectSR_ _ZN6scitbx22positive_getitem_indexIlmEET0_RKT_RKS1_bPKc _ZN6scitbx2af12boost_python12flex_wrapperIdN5boost6python19return_value_policyINS4_24copy_non_const_referenceENS4_21default_call_policiesEEEE10getitem_1dERNS0_5versaIdNS0_9flex_gridINS0_5smallIlLm10EEEEEEEl _ZN5boost6python6detail12caller_arityILj2EE4implIPFRdRN6scitbx2af5versaIdNS7_9flex_gridINS7_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3IS5_SE_lEEEclEP7_objectSQ_ _ZN5boost6python7objects23caller_py_function_implINS0_6detail6callerIPFRdRN6scitbx2af5versaIdNS7_9flex_gridINS7_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3IS5_SE_lEEEEEclEP7_objectSR_ _ZN6scitbx22positive_getitem_indexIlmEET0_RKT_RKS1_bPKc _ZN6scitbx2af12boost_python12flex_wrapperIdN5boost6python19return_value_policyINS4_24copy_non_const_referenceENS4_21default_call_policiesEEEE10getitem_1dERNS0_5versaIdNS0_9flex_gridINS0_5smallIlLm10EEEEEEEl _ZN5boost6python6detail12caller_arityILj2EE4implIPFRdRN6scitbx2af5versaIdNS7_9flex_gridINS7_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3IS5_SE_lEEEclEP7_objectSQ_ _ZN5boost6python7objects23caller_py_function_implINS0_6detail6callerIPFRdRN6scitbx2af5versaIdNS7_9flex_gridINS7_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3IS5_SE_lEEEEEclEP7_objectSR_ _ZN6scitbx22positive_getitem_indexIlmEET0_RKT_RKS1_bPKc _ZN6scitbx2af12boost_python12flex_wrapperIN5cctbx4xray9scattererIdNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESB_EEN5boost6python25return_internal_referenceILm1ENSE_21default_call_policiesEEEE10getitem_1dERNS0_5versaISC_NS0_9flex_gridINS0_5smallIlLm10EEEEEEEl _ZN5boost6python6detail12caller_arityILj2EE4implIPFRN5cctbx4xray9scattererIdNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESD_EERN6scitbx2af5versaISE_NSH_9flex_gridINSH_5smallIlLm10EEEEEEElENS0_25return_internal_referenceILm1ENS0_21default_call_policiesEEENS_3mpl7vector3ISF_SO_lEEEclEP7_objectSZ_ _ZN5boost6python7objects23caller_py_function_implINS0_6detail6callerIPFRN5cctbx4xray9scattererIdNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESD_EERN6scitbx2af5versaISE_NSH_9flex_gridINSH_5smallIlLm10EEEEEEElENS0_25return_internal_referenceILm1ENS0_21default_call_policiesEEENS_3mpl7vector3ISF_SO_lEEEEEclEP7_objectS10_ _ZN6scitbx22positive_getitem_indexIlmEET0_RKT_RKS1_bPKc _ZN6scitbx2af12boost_python12flex_wrapperIdN5boost6python19return_value_policyINS4_24copy_non_const_referenceENS4_21default_call_policiesEEEE10getitem_1dERNS0_5versaIdNS0_9flex_gridINS0_5smallIlLm10EEEEEEEl _ZN5boost6python6detail12caller_arityILj2EE4implIPFRdRN6scitbx2af5versaIdNS7_9flex_gridINS7_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3IS5_SE_lEEEclEP7_objectSQ_ _ZN5boost6python7objects23caller_py_function_implINS0_6detail6callerIPFRdRN6scitbx2af5versaIdNS7_9flex_gridINS7_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3IS5_SE_lEEEEEclEP7_objectSR_ _ZN6scitbx22positive_getitem_indexIlmEET0_RKT_RKS1_bPKc _ZN6scitbx2af12boost_python12flex_wrapperINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEN5boost6python19return_value_policyINSA_24copy_non_const_referenceENSA_21default_call_policiesEEEE10getitem_1dERNS0_5versaIS8_NS0_9flex_gridINS0_5smallIlLm10EEEEEEEl _ZN5boost6python6detail12caller_arityILj2EE4implIPFRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERN6scitbx2af5versaISA_NSD_9flex_gridINSD_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3ISB_SK_lEEEclEP7_objectSW_ _ZN5boost6python7objects23caller_py_function_implINS0_6detail6callerIPFRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERN6scitbx2af5versaISA_NSD_9flex_gridINSD_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3ISB_SK_lEEEEEclEP7_objectSX_ _ZN6scitbx22positive_getitem_indexIlmEET0_RKT_RKS1_bPKc _ZN6scitbx2af12boost_python12flex_wrapperIN5cctbx4xray9scattererIdNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESB_EEN5boost6python25return_internal_referenceILm1ENSE_21default_call_policiesEEEE10getitem_1dERNS0_5versaISC_NS0_9flex_gridINS0_5smallIlLm10EEEEEEEl _ZN5boost6python6detail12caller_arityILj2EE4implIPFRN5cctbx4xray9scattererIdNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESD_EERN6scitbx2af5versaISE_NSH_9flex_gridINSH_5smallIlLm10EEEEEEElENS0_25return_internal_referenceILm1ENS0_21default_call_policiesEEENS_3mpl7vector3ISF_SO_lEEEclEP7_objectSZ_ _ZN5boost6python7objects23caller_py_function_implINS0_6detail6callerIPFRN5cctbx4xray9scattererIdNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESD_EERN6scitbx2af5versaISE_NSH_9flex_gridINSH_5smallIlLm10EEEEEEElENS0_25return_internal_referenceILm1ENS0_21default_call_policiesEEENS_3mpl7vector3ISF_SO_lEEEEEclEP7_objectS10_ _ZN6scitbx22positive_getitem_indexIlmEET0_RKT_RKS1_bPKc _ZN6scitbx2af12boost_python12flex_wrapperINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEN5boost6python19return_value_policyINSA_24copy_non_const_referenceENSA_21default_call_policiesEEEE10getitem_1dERNS0_5versaIS8_NS0_9flex_gridINS0_5smallIlLm10EEEEEEEl _ZN5boost6python6detail12caller_arityILj2EE4implIPFRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERN6scitbx2af5versaISA_NSD_9flex_gridINSD_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3ISB_SK_lEEEclEP7_objectSW_ _ZN5boost6python7objects23caller_py_function_implINS0_6detail6callerIPFRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERN6scitbx2af5versaISA_NSD_9flex_gridINSD_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3ISB_SK_lEEEEEclEP7_objectSX_ _ZN6scitbx22positive_getitem_indexIlmEET0_RKT_RKS1_bPKc _ZN6scitbx2af12boost_python12flex_wrapperImN5boost6python19return_value_policyINS4_24copy_non_const_referenceENS4_21default_call_policiesEEEE10getitem_1dERNS0_5versaImNS0_9flex_gridINS0_5smallIlLm10EEEEEEEl _ZN5boost6python6detail12caller_arityILj2EE4implIPFRmRN6scitbx2af5versaImNS7_9flex_gridINS7_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3IS5_SE_lEEEclEP7_objectSQ_ _ZN5boost6python7objects23caller_py_function_implINS0_6detail6callerIPFRmRN6scitbx2af5versaImNS7_9flex_gridINS7_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3IS5_SE_lEEEEEclEP7_objectSR_ _ZN6scitbx22positive_getitem_indexIlmEET0_RKT_RKS1_bPKc _ZN6scitbx2af12boost_python12flex_wrapperIN5cctbx4xray9scattererIdNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESB_EEN5boost6python25return_internal_referenceILm1ENSE_21default_call_policiesEEEE10getitem_1dERNS0_5versaISC_NS0_9flex_gridINS0_5smallIlLm10EEEEEEEl _ZN5boost6python6detail12caller_arityILj2EE4implIPFRN5cctbx4xray9scattererIdNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESD_EERN6scitbx2af5versaISE_NSH_9flex_gridINSH_5smallIlLm10EEEEEEElENS0_25return_internal_referenceILm1ENS0_21default_call_policiesEEENS_3mpl7vector3ISF_SO_lEEEclEP7_objectSZ_ _ZN5boost6python7objects23caller_py_function_implINS0_6detail6callerIPFRN5cctbx4xray9scattererIdNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESD_EERN6scitbx2af5versaISE_NSH_9flex_gridINSH_5smallIlLm10EEEEEEElENS0_25return_internal_referenceILm1ENS0_21default_call_policiesEEENS_3mpl7vector3ISF_SO_lEEEEEclEP7_objectS10_ _ZN6scitbx22positive_getitem_indexIlmEET0_RKT_RKS1_bPKc _ZN6scitbx2af12boost_python12flex_wrapperINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEN5boost6python19return_value_policyINSA_24copy_non_const_referenceENSA_21default_call_policiesEEEE10getitem_1dERNS0_5versaIS8_NS0_9flex_gridINS0_5smallIlLm10EEEEEEEl _ZN5boost6python6detail12caller_arityILj2EE4implIPFRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERN6scitbx2af5versaISA_NSD_9flex_gridINSD_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3ISB_SK_lEEEclEP7_objectSW_ _ZN5boost6python7objects23caller_py_function_implINS0_6detail6callerIPFRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERN6scitbx2af5versaISA_NSD_9flex_gridINSD_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3ISB_SK_lEEEEEclEP7_objectSX_ with key value 9965986033708389444
The colliding key is _ZN6scitbx2af12boost_python12flex_wrapperINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEN5boost6python19return_value_policyINSA_24copy_non_const_referenceENSA_21default_call_policiesEEEE10getitem_1dERNS0_5versaIS8_NS0_9flex_gridINS0_5smallIlLm10EEEEEEEl _ZN5boost6python6detail12caller_arityILj2EE4implIPFRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERN6scitbx2af5versaISA_NSD_9flex_gridINSD_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3ISB_SK_lEEEclEP7_objectSW_ _ZN5boost6python7objects23caller_py_function_implINS0_6detail6callerIPFRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERN6scitbx2af5versaISA_NSD_9flex_gridINSD_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3ISB_SK_lEEEEEclEP7_objectSX_ _ZN6scitbx22positive_getitem_indexIlmEET0_RKT_RKS1_bPKc _ZN6scitbx2af12boost_python12flex_wrapperIN5cctbx4xray9scattererIdNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESB_EEN5boost6python25return_internal_referenceILm1ENSE_21default_call_policiesEEEE10getitem_1dERNS0_5versaISC_NS0_9flex_gridINS0_5smallIlLm10EEEEEEEl _ZN5boost6python6detail12caller_arityILj2EE4implIPFRN5cctbx4xray9scattererIdNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESD_EERN6scitbx2af5versaISE_NSH_9flex_gridINSH_5smallIlLm10EEEEEEElENS0_25return_internal_referenceILm1ENS0_21default_call_policiesEEENS_3mpl7vector3ISF_SO_lEEEclEP7_objectSZ_ _ZN5boost6python7objects23caller_py_function_implINS0_6detail6callerIPFRN5cctbx4xray9scattererIdNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESD_EERN6scitbx2af5versaISE_NSH_9flex_gridINSH_5smallIlLm10EEEEEEElENS0_25return_internal_referenceILm1ENS0_21default_call_policiesEEENS_3mpl7vector3ISF_SO_lEEEEEclEP7_objectS10_ _ZN6scitbx22positive_getitem_indexIlmEET0_RKT_RKS1_bPKc _ZN6scitbx2af12boost_python12flex_wrapperINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEN5boost6python19return_value_policyINSA_24copy_non_const_referenceENSA_21default_call_policiesEEEE10getitem_1dERNS0_5versaIS8_NS0_9flex_gridINS0_5smallIlLm10EEEEEEEl _ZN5boost6python6detail12caller_arityILj2EE4implIPFRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERN6scitbx2af5versaISA_NSD_9flex_gridINSD_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3ISB_SK_lEEEclEP7_objectSW_ _ZN5boost6python7objects23caller_py_function_implINS0_6detail6callerIPFRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERN6scitbx2af5versaISA_NSD_9flex_gridINSD_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3ISB_SK_lEEEEEclEP7_objectSX_ _ZN6scitbx22positive_getitem_indexIlmEET0_RKT_RKS1_bPKc _ZN6scitbx2af12boost_python12flex_wrapperImN5boost6python19return_value_policyINS4_24copy_non_const_referenceENS4_21default_call_policiesEEEE10getitem_1dERNS0_5versaImNS0_9flex_gridINS0_5smallIlLm10EEEEEEEl _ZN5boost6python6detail12caller_arityILj2EE4implIPFRmRN6scitbx2af5versaImNS7_9flex_gridINS7_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3IS5_SE_lEEEclEP7_objectSQ_ _ZN5boost6python7objects23caller_py_function_implINS0_6detail6callerIPFRmRN6scitbx2af5versaImNS7_9flex_gridINS7_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3IS5_SE_lEEEEEclEP7_objectSR_ _ZN6scitbx22positive_getitem_indexIlmEET0_RKT_RKS1_bPKc _ZN6scitbx2af12boost_python12flex_wrapperIN5cctbx4xray9scattererIdNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESB_EEN5boost6python25return_internal_referenceILm1ENSE_21default_call_policiesEEEE10getitem_1dERNS0_5versaISC_NS0_9flex_gridINS0_5smallIlLm10EEEEEEEl _ZN5boost6python6detail12caller_arityILj2EE4implIPFRN5cctbx4xray9scattererIdNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESD_EERN6scitbx2af5versaISE_NSH_9flex_gridINSH_5smallIlLm10EEEEEEElENS0_25return_internal_referenceILm1ENS0_21default_call_policiesEEENS_3mpl7vector3ISF_SO_lEEEclEP7_objectSZ_ _ZN5boost6python7objects23caller_py_function_implINS0_6detail6callerIPFRN5cctbx4xray9scattererIdNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESD_EERN6scitbx2af5versaISE_NSH_9flex_gridINSH_5smallIlLm10EEEEEEElENS0_25return_internal_referenceILm1ENS0_21default_call_policiesEEENS_3mpl7vector3ISF_SO_lEEEEEclEP7_objectS10_ _ZN6scitbx22positive_getitem_indexIlmEET0_RKT_RKS1_bPKc _ZN6scitbx2af12boost_python12flex_wrapperINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEN5boost6python19return_value_policyINSA_24copy_non_const_referenceENSA_21default_call_policiesEEEE10getitem_1dERNS0_5versaIS8_NS0_9flex_gridINS0_5smallIlLm10EEEEEEEl _ZN5boost6python6detail12caller_arityILj2EE4implIPFRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERN6scitbx2af5versaISA_NSD_9flex_gridINSD_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3ISB_SK_lEEEclEP7_objectSW_ _ZN5boost6python7objects23caller_py_function_implINS0_6detail6callerIPFRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERN6scitbx2af5versaISA_NSD_9flex_gridINSD_5smallIlLm10EEEEEEElENS0_19return_value_policyINS0_24copy_non_const_referenceENS0_21default_call_policiesEEENS_3mpl7vector3ISB_SK_lEEEEEclEP7_objectSX_
And it has keyID 9965986033708389444

A couple things:

1. It's a bit hard to see, but the massive name on the first line has spaces in it. It looks like it contains multiple symbol names. Isn't it supposed to be the name of a single function? Could there be an issue with not inserting a \0 when a string overflows some buffer?
2. That key rendered in hex is 0x8a4e4b7edfb15444. It doesn't look especially meaningful to me, except it's large and might be a pointer.

[spakin]

1. In fact, both names appear to contain spaces. Thanks for pointing that out. Are you compiling with -bf-call-stack? Spaces are used to separate call-stack entries.
2. Keys are generated randomly at instrumentation time so don't read too much into that 9965986033708389444 value.

[elliottslaughter]

That's a good point, all my builds so far have been with -bf-by-func -bf-call-stack. In fact, a new build with just -bf-by-func seems to work just fine. So context does seem to be responsible somehow.

For my purposes, -bf-by-func is probably fine, so I'm happy to more forward with that for now, though I'm also happy to debug with you if there are other things you'd like me to try.

This is speculation, but I wonder if somehow having Python on the stack above the C++ calls is causing issues of some sort. Python might be doing strange things like using it's own stack; I'm not 100% sure about this.

[spakin]

I'm also happy to debug with you if there are other things you'd like me to try.

It'll probably produce an overwhelming amount of output, but I suppose you can modify bf_record_key() to output all funcname/keyID pairs. Perhaps that will reveal some pattern in how call stacks are getting mapped to IDs and why two different calls are receiving the same ID.

[mdewing]

I ran into this issue with on a code compiled with -bf-call-stack.
I think the issue is the hash mixing in bf_push_function:

  bf_func_and_parents_id = bf_func_and_parents_id^ depth ^ keyID;

If this is changed to something like 17 * bf_func_and_parents_id^depth^keyID; then the program gets past this error.

[spakin]

@markdewing: Wow, that's really strange. I can't imagine how two call stacks would see the same bf_func_and_parents_id^depth^keyID but different 17 * bf_func_and_parents_id^depth^keyID values. Multiplying by a constant shouldn't do anything to resolve a key collision.

[mdewing]

Oops - I missed the parenthesis in what I wrote - should be (17*bf_func_and_parents_id)^depth_keyID

[spakin]

Still, multiplying by a constant shouldn't do anything to resolve a key collision. I'm baffled how this could work.