Identifying best matching code

[chapmags]

In applying the package (I’m going from HS6 to ISIC4), and doing so for all HS codes, my results often match to 2 or more ISIC codes (with equal weights across the codes). However, in the available datasets that you provide (looking specifically at hs6_isic3), you have a one-to-one match.

Having looked at my outputs (thank you to previous questions/answers about using purr::pluck to extract), it’s clearly not always the first or second match that is the best fit to the HS description; and with the weights being equal, I can’t simply identify the largest weight and use that for my match. Therefore, is there another step in this process that is available through your concordance package, so that I can similarly end with a one-to-one match?

If there isn’t an additional step available through the package, could you perhaps explain how you went about creating your one-to-one match for the hs6_isic3 data?

[stevenliaotw]

Thanks for the question. We use direct concordance tables from authoritative sources whenever possible (e.g., https://wits.worldbank.org/product_concordance.html). When such direct concordance tables are not available, we use the concordance tables that are close to the user's source/destination classification as intermediate "bridges" to produce the concordances.

For example, to get HS6 to ISIC3 concordances, we first concord HS6 to HS3 and then HS3 to ISIC3 (see lines 619-644 of https://github.com/insongkim/concordance/blob/master/data-raw/clean-hs-to-isic.R). Given the precise concordances within the HS classification, more one-to-one matches are produced. In contrast, concordance between HS6 to ISIC4 is rougher because we have to concord HS6 to HS3, HS3 to ISIC3, and then ISIC3 to ISIC4, which increases the likelihood of multiple matches (see lines 346-362 of https://github.com/insongkim/concordance/blob/master/R/concord_hs_isic.R).

We document all our data cleaning for HS-ISIC concordance here https://github.com/insongkim/concordance/blob/master/data-raw/clean-hs-to-isic.R. Hope this helps answer your question!

[chapmags]

Thanks Steven, that makes sense and explains the process well.

I started looking through some of your code/files but because there are so many, I stopped (because of time constraints). Credit to you and the others for documenting everything so well, and thank you for directing me to the specific scripts/lines. This is a great resource.