Recipient organisation type chart does not recognise Church codes as an org type

[KDuerden]

This funder has published 637 grants. 433 of these use newly adopted org ID type, GB-COE - https://org-id.guide/list/GB-COE

However these do not appear in the chart, and are instead the chart states that they "did not include information to determine recipient organisation type."

Data for reference is here:
https://grantvis.threesixtygiving.org/?query=*&default_field=*&sort=_score+desc&fundingOrganization=GB-CHC-1074857

[mariongalley]

Thanks Katherine, that's really helpful. @michaelwood Is there a way to add churches to our list of recognised org types now that the church codes are in org-id.guide?

[michaelwood]

Those organisation type categories are defined by Find That Charity so need to be added there.

[mariongalley]

@drkane Can you look into this one on the FTC side please? Happy for the issue to be moved to that repo instead

[drkane]

Not to ping this around different repos but I think this might need to be a change to the datastore.

Specifically this script: https://github.com/ThreeSixtyGiving/datastore/blob/live/datastore/additional_data/sources/load_all_org_data.sh needs to have a line added:

./manage.py load_org_data https://findthatcharity.uk/orgid/source/coe.csv

And then needs to be rerun (can't remember if it's rerun periodically anyway?) - the data should then filter through to grantnav and grantvis. The organisations will be shown as "Church".

[michaelwood]

Thanks David, great to hear those orgs are available. I'll move this issue to the datastore.

[drkane]

I took the opportunity to see if there are any other sources that are missing from the import list. I've set up a dashboard which shows the sources used, how many organisations and when they were last updated.

This seems to suggest there are three missing sources:

• coe - church codes as discussed above
• lamysociety - local authorities as maintained by mysociety - this replaces the registers which are now longer updated
• officeforstudents - UKPRN codes for higher education institutions

There are 4 sources which are deprecated so could probably be removed: lae, lani, las, pla - these are the local authority registers.

The nideptofeducation source has been broken since last year, but is still in theory an active source so I think should be kept.

So the updated list would look like:

#!/bin/bash
# Load all the different types of org data from Find That Charity.

./manage.py load_org_data https://findthatcharity.uk/orgid/source/casc.csv
./manage.py load_org_data https://findthatcharity.uk/orgid/source/ccew.csv
./manage.py load_org_data https://findthatcharity.uk/orgid/source/ccni.csv
./manage.py load_org_data https://findthatcharity.uk/orgid/source/oscr.csv
./manage.py load_org_data https://findthatcharity.uk/orgid/source/companies.csv
./manage.py load_org_data https://findthatcharity.uk/orgid/source/mutuals.csv
./manage.py load_org_data https://findthatcharity.uk/orgid/source/gor.csv
./manage.py load_org_data https://findthatcharity.uk/orgid/source/ror.csv
./manage.py load_org_data https://findthatcharity.uk/orgid/source/hesa.csv
./manage.py load_org_data https://findthatcharity.uk/orgid/source/coe.csv
./manage.py load_org_data https://findthatcharity.uk/orgid/source/lamysociety.csv
./manage.py load_org_data https://findthatcharity.uk/orgid/source/officeforstudents.csv
./manage.py load_org_data https://findthatcharity.uk/orgid/source/nhsods-epraccur.csv
./manage.py load_org_data https://findthatcharity.uk/orgid/source/nhsods-etr.csv
./manage.py load_org_data https://findthatcharity.uk/orgid/source/nhsods-ensa.csv
./manage.py load_org_data https://findthatcharity.uk/orgid/source/nhsods-eccg.csv
./manage.py load_org_data https://findthatcharity.uk/orgid/source/nhsods-ecsu.csv
./manage.py load_org_data https://findthatcharity.uk/orgid/source/nhsods-espha.csv
./manage.py load_org_data https://findthatcharity.uk/orgid/source/nhsods-wlhb.csv
./manage.py load_org_data https://findthatcharity.uk/orgid/source/nhsods-ect.csv
./manage.py load_org_data https://findthatcharity.uk/orgid/source/rsl.csv
./manage.py load_org_data https://findthatcharity.uk/orgid/source/gias.csv schools_gias
./manage.py load_org_data https://findthatcharity.uk/orgid/source/nideptofeducation.csv schools_ni
./manage.py load_org_data https://findthatcharity.uk/orgid/source/schoolsscotland.csv schools_scotland
./manage.py load_org_data https://findthatcharity.uk/orgid/source/walesschools.csv schools_wales

[KDuerden]

Thanks for sorting this out.

There are 4 sources which are deprecated so could probably be removed: lae, lani, las, pla - these are the local authority registers.

Can we please discuss this before actioning any depreciation or removal. We are still recommending using these, and the new Mysociety org ID prefix/type hasn't been submitted to org-id.guide yet so we should agree next steps and put alternatives in place before removing the existing org ID types from our systems.

[drkane]

Thanks Katherine - that's a good point. I'm not planning to remove them from find-that-charity, and they wouldn't be removed from the datastore I don't think (just no longer updated) - but it's probably a good idea to keep them in this list anyway in case we ever needed to rebuild the list from scratch to make sure they're in there.

[KDuerden]

thanks for clarifying that it relates to the updates, rather than the org IDs themselves - all understood!

[mariongalley]

We need to look at the scale of legitimate org ids that aren't getting assigned org types - it looks like this includes Companies Limited by Shares, but also there seems to be a bunch of other company and charity and other types of IDs not getting mapped to an org type (~60k records for 2020-23 data)

[michaelwood]

Just to pull this back a bit to specifics, I think the first part of addressing this would be to add a GB-COE Org type matcher to the table with the appropriate type label:

id	regex	priority	tsg_org_type
1	GB-(LAE|LANI|LAS|PLA)-.*	10	Local Government
2	GB-GOR-.*	20	Central Government
3	GB-COH-RC000766|360G-blf|GB-CHC-1036733|GB-GOR-PC390|GB-GOR-PB188	30	Lottery Distributor
4	.*	0	Grantmaking Organisation
5	GB-GOR-DA1020	25	Devolved Government