Backwards concordance not the same as forward concordance tables

[zachariah-frost]

Hi Insongkim,

I noticed when using your concordance package that when concording forwards, for example code 681510, it is identified as not changing between HS4 (2012) and HS5 (2017). However, when using the UN's HS tracker tool, it says that in this period, 681510 is split into 681510 and 962000. My suspicion is that this is due to which tables are used on the UN concordance tables website - HS2017 - HS2012 does not identify this change (there is a reference in the comments column, but not in the HS code column), and it is not recognised in your database within the package when looking at the hs5_hs4 data. If i'm correct in this, the code will have to be adapted to consider both forwards and backwards concordance - so will have to use as well the HS2012 - HS2017?

Kind regards,

ZF

[stevenliaotw]

Hi ZF,

Thanks for using our package and flagging this! Yes, we are just directly relying on UN's concordance official tables here https://unstats.un.org/unsd/trade/classifications/correspondence-tables.asp, which seems to only provide backward concordance the last time I checked. Further details of cleaning the data here https://github.com/insongkim/concordance/blob/master/data-raw/clean-hs.R. Do you know where we can find similar forward concordance tables? Would really appreciate it if you know of a good source to systematically identify them!

Best,
Steven

[zachariah-frost]

Hi Steven,

No problem! Yeah unfortunately from my experience the only source of the forwards concordance tables are in a series of disaggregated .pdf files on the World Customs Organisation site, e.g. here. Unfortunately, as per usual, the main distribution method is .pdf - it would be possible to extract the data (I think, although you never know with .pdf), but would require a lot of patience. I also noticed with your cleaning the data code, have you thought about functionalizing it, it seems the same bit of code repeats a lot (sorry if I've missed something)?

Kind regards,

ZF

[stevenliaotw]

Hi ZF,

Thanks! We will look into that source and figure out the best way to incorporate them.

Re the data cleaning code, they are just for documentation/replication purposes and are not part of the compiled package (everything under ~/data-raw/ is excluded when compiling). So we could functionalize the code, but there won't be efficiency gains in the actual package in doing so if that's the concern. :)

Best,
Steven