Option to search substances in PubChem

[stanstrup]

Could we have an option in get_cid to also search through substances?

For example the CAS number "152957-32-9" is listed as a synonym for SID 349678457 but not in the CID entry. Yet the SID can point me to the CID.
So the following URL would give me what I was after:
https://pubchem.ncbi.nlm.nih.gov/rest/pug/substance/name/152957-32-9/cids/JSON

So I suggested an option that if TRUE would also search "substance". You might get duplicate CIDs returned from compounds and substance and those should be de-duplicated with unique.

[Aariq]

Sounds good to me. I assume not all SIDs have corresponding CIDs though, right? So even when search_substances = TRUE (or whatever), and there is a SID, it would return NA if there's no CID. If we ever wanted a get_sid function, that should be implemented separately I think.

[stanstrup]

I accidentally pushed directly to this repo. I didn't even know I had write access. Should the commit be rolled back? 417d1c8

Alternatively if you are fine with the suggestion I guess it can stay.

[Aariq]

If you can roll it back and submit as a pull request that would be best I think—partly because I have some suggestions and they are easier to make in PRs

[stitam]

Hi, I also think it would be better to separate this functionality into get_sid(). Apart from compounds and substances, PubChem also has assays, genes, and patents. Maybe it would be better to deprecate get_cid() and use something like get_pcid() to query all sorts of PubChem ids?

[stitam]

I started working on a highly customizable function that can assemble and execute all sorts of PubChem PUG-REST queries. The PR would revert get_cid() to its original functionality and introduce this new function. Would this be an acceptable solution for now?

[Aariq]

I do like the idea of having the option to search substances like in the original suggestion. It's definitely not the same as having a separate function to get SIDs. The problem is that there are some chemicals with CIDs that are not listed as compounds but rather as substances in pubchem. (i.e. they are at a https://pubchem.ncbi.nlm.nih.gov/substance/ URL). Would this new function solve this problem?

library(webchem)

# I want a CID for Cortisol 21-Sulfate Potassium Salt

get_cid("Cortisol 21-Sulfate Potassium Salt")
#> https://pubchem.ncbi.nlm.nih.gov/rest/pug/compound/name/cids/JSON
#> Warning in foo(query = .x, from = from, match = match, verbose = verbose, : No
#> CID found that matches the given name. Returning NA.
#> # A tibble: 1 x 2
#>   query                              cid  
#> * <chr>                              <lgl>
#> 1 Cortisol 21-Sulfate Potassium Salt NA

# No CID?  That's not true though!

get_cid("Cortisol 21-Sulfate Potassium Salt", search_substances = TRUE)
#> https://pubchem.ncbi.nlm.nih.gov/rest/pug/compound/name/cids/JSON
#> Warning in foo(query = .x, from = from, match = match, verbose = verbose, : No
#> CID found that matches the given name. Returning NA.
#> https://pubchem.ncbi.nlm.nih.gov/rest/pug/substance/name/cids/JSON
#> More then one Link found for ''.
#> Returning all matches.
#> # A tibble: 3 x 2
#>   query                              cid      
#> * <chr>                              <chr>    
#> 1 Cortisol 21-Sulfate Potassium Salt <NA>     
#> 2 Cortisol 21-Sulfate Potassium Salt 129011998
#> 3 Cortisol 21-Sulfate Potassium Salt 131874189

# It does have a CID associated with it.

^{Created on 2020-05-04 by the reprex package (v0.3.0)}

[stitam]

My main concern is that the get_cid() function would return either CIDs or SIDs, which is inconsistent with the name of the function and might be confusing. On the other hand, we would get away with one less exported functions. Is this the reason you suggest the search_substances argument instead of a separate get_sid() function? I am ok with this, we will need more complex function documentation and match.arg() conditions because the two domains accept different input types, but this is manageable. One small thing, I would rather call the argument domain with values "compound" and "substance" so if we later decide we want this function to handle AIDs as well, it will be much simpler. Let's reach consensus and I will implement it in PR #239.

[Aariq]

I think you misunderstand. 131874189 is the CID for the substance, not the SID.

[stitam]

Oh yes, I definitely misunderstood, thanks @Aariq! In this case there is no function name inconsitency, which is great. However, as CIDs can be acquired through queries from all three domains (compound, substance, assay), naming the new argument domain instead of search_substances still has benefits. Also whatever I wrote about documentation and match.arg() is still valid I think, e.g. get_cid() will not work when you search a formula in the substance domain. What do you think?

[Aariq]

What I like about search_substances is that it searches both the compound and substance domains and then collapses the results by removing duplicate CIDs. So, what would be cool is a similar behavior with something like domain = c("compound", "substance").

[stitam]

Then why not always search all valid domains for the given query type and collapse the results? This way we wouldn't need any new arguments. Is there any situation in which this woud be undesirable?

[Aariq]

Then why not always search all valid domains for the given query type and collapse the results? This way we wouldn't need any new arguments. Is there any situation in which this woud be undesirable?

It's slower. For a lot of compounds, it may be noticeably slower.