get_references does not retrieve new references

[cplee1]

The get_references utility does not retreive references in the new v2 naming format.

For example, adl+24 is not in the reference dictionary returned by psrqpy.get_references(version='2.6.0', updaterefcache=True). I can see on the webpage that the new references have a "v1ref" field, but I'm not sure how to modify psrqpy to make use of this.

[mattpitkin]

Hi @cplee1, thanks for bringing this to my attention. I think I'll need to rewrite my get_references function to parse the new style of references in https://www.atnf.csiro.au/research/pulsar/psrcat/psrcat_ref.html - currently they are extracted from the psrcat_ref file in the catalogue download tarball, which doesn't seem to contain some of the newer references, e.g., adl+24. Given that they are now in bibTeX format this should hopefully make the function simpler!

[mattpitkin]

I contacted the ATNF catalogue maintainers and they know about the issue of missing references in the current release of the catalogue. They will be making sure these are in the catalogue's next minor release. So, I may not actually have to update the interface.

[cplee1]

Hi Matt, psrcat v2.6.1 has been released and I can see on the ref webpage that the reference names have been fixed (e.g. adl+24). However, I'm still getting an error with psrqpy:

import psrqpy

print(f"psrqpy v{psrqpy.__version__}")

q = psrqpy.QueryATNF(version='2.6.1')
print(f"psrcat v{q.get_version}")

ref_dict = psrqpy.get_references(version='2.6.1', updaterefcache=True)
print(ref_dict["adl+24"])

This results in:

psrqpy v1.3.1
psrcat v2.6.1
Traceback (most recent call last):
  File "/home/chris/software/pulsar_spectra/psrcat_testing.py", line 9, in <module>
    print(ref_dict["adl+24"])
          ~~~~~~~~^^^^^^^^^^
KeyError: 'adl+24'

[cplee1]

Hi @mattpitkin
Just following up on this. I tried with psrqpy 1.3.2 and psrcat 2.6.1, and still get the same error for (e.g.) adl+24. Any idea why?

[mattpitkin]

That reference still doesn't seem to have been added to the psrcat_ref file within https://www.atnf.csiro.au/research/pulsar/psrcat/downloads/psrcat_pkg.tar.gz. It's probably worth using https://www.atnf.csiro.au/research/pulsar/psrcat/feedback.html to submit a request to try and get this reference added in (having more requests might bump the priority).

If I get the time, I'll keep working on #151, to get reference info from the website (and maybe use the psrcat_ref file as backup).

[cplee1]

Thanks Matt, I requested feedback and they said they will aim to fix it for the next release.

[cplee1]

The missing refs have been added to psrcat_ref as of v2.6.2 :)

[mattpitkin]

The psrcat_ref file in the latest version has now changed format to look like:

beginthebibliography1226
expandafterifxcsname natexlabendcsnamerelaxdefnatexlab#1#1fi

bibem[Abbate et al.(2023natexlaba)Abbate, Noutsos,
  Desvignes, Wharton, Torne, Kramer, Eatough, Karuppusamy, Liu,
  Shao, & Wongphechauxsorn]and+23
Abbate, F., Noutsos, A., Desvignes, G., Wharton, R. S., Torne, P.,
  Kramer, M., Eatough, R. P., Karuppusamy, R., Liu, K., Shao, L., &
  Wongphechauxsorn, J. 2023natexlaba, mnras, 524, 2966

bibem[Abbate et al.(2023natexlabb)Abbate, Possenti, Ridolfi,
  Venkatraman Krishnan, Buchner, Barr, Bailes, Kramer, Cameron,
  Parthasarathy, van Straten, Chen, Camilo, Padmanabh, Mao,
  Freire, Ransom, Vleeschower, Geyer, & Zhang]apr+23
Abbate, F., Possenti, A., Ridolfi, A., Venkatraman Krishnan, V.,
  Buchner, S., Barr, E. D., Bailes, M., Kramer, M., Cameron, A.,
  Parthasarathy, A., van Straten, W., Chen, W., Camilo, F.,
  Padmanabh, P. V., Mao, S. A., Freire, P. C. C., Ransom, S. M.,
  Vleeschower, L., Geyer, M., & Zhang, L. 2023natexlabb, mnras,
  518, 1642

...

so reference retrieval is broken again. I'll need to make #151 a priority now.