parser: parse from simple-head too

[ioannistsanaktsidis]

• ref Elsevier parser should extract metadata from simple-head too cern-sis/issues-inspire#429

[ioannistsanaktsidis]

A note related to this PR. The repo is not python 3 compatible, meaning we can build it but tests are not running etc... Yet we release it with python3 -> https://github.com/inspirehep/hepcrawl/pull/358/files#diff-20e0e050358c0425896e7b9edb659fec4ed949bb350a35841c0d616e1971cc6bR22. What was the reason for that? Was it just to release it for now as we are using it only in python2(inspire-next) and fix it in the future? cc @drjova

[ioannistsanaktsidis]

Also, usecase -> https://github.com/inspirehep/hepcrawl/pull/358/files#diff-c9b75f1849ee2f819f04cabf4463702f66d1c748fb8ef795ea2a3ea81d06309eR165. Fails on CI, succeeds locally with exactly the same setup. Issue seems to be here https://github.com/inspirehep/hepcrawl/blob/master/hepcrawl/spiders/pos_spider.py#L112-L117. For some reason it is not extracted correctly on the CI. Could be the way we are generating the fake_response_file -> https://github.com/inspirehep/hepcrawl/blob/master/hepcrawl/testlib/fixtures.py#L50-L56, but not sure. To be checked further as this is irrelevant to the changes of this PR

[michamos]

Not directly related to simple-head, by I noticed two additional issues:

• ORCIDs don't seem to be extracted
• In the example file, the DOI of the original paper is mentioned in

<ce:document-thread><ce:refers-to-document id="rd0010"><ce:pii>S0550-3213(22)00251-6</ce:pii><ce:doi>10.1016/j.nuclphysb.2022.115900</ce:doi></ce:refers-to-document></ce:document-thread>

It would be good to extract this DOI too so the erratum gets matched with the original publication, otherwise it will create new records in INSPIRE which we don't want. So in this case, DOIs should look like

{
    "dois": [
        {"material": "erratum", "value": "10.1016/j.nuclphysb.2022.115991"},
        {"material": "publication", "value": "10.1016/j.nuclphysb.2022.115900"}
    ]
}

(order doesn't matter)

[ioannistsanaktsidis]

Thanks @michamos , will create another ticket to handle these findings.

[michamos]

OK, but note that this should not go to prod without the related DOI extraction, as it will cause a mess.

[ioannistsanaktsidis]

But isn’t that the case already ? Don’t we crawl elsevier papers like that one ?

[michamos]

We crawl them, but they don't get added because they don't pass the should_record_be_harvested() check due to the lack of title currently.

[drjova]

Thanks @ioannistsanaktsidis few comment for clarification and formatting

[drjova]

why the path here is different from abstract?

[ioannistsanaktsidis]

You mean from the line above ? only difference is that it tries to extract from simple-head if nothing found on head . Or I misunderstood the question ?

[michamos]

👍