Subsetting creates edges pointing to non existing nodes

[Paco-Danes]

Hello, i think that the subsetting procedure doesn't filter out edges to the nodes that actually exist in the subset.
If i try to load the csv to neo4j with the command, it gives an error by finding edges pointing to nodes (id) that do not exist (since by subsetting a lot of nodes are excluded).
I don't know if this should not happen and i did something wrong with the import, but if this is not the case i think we should ensure that during the subset option, each edge is retained only if both source and also target nodes are present in the nodes subset.

[slobentanzer]

Hi @Paco-Danes,
Thanks for opening that issue. First off, we have not worked on this for a while, so it may have become de-synchronised with BioCypher updates. Main reason is that the from-neo4j-to-neo4j process I tried here is a little overcomplex, in my opinion. I have not officially deprecated this repo / approach, but considering it.

Are you hoping to use it downstream; would it make sense to "rescue" the source data? I think the original repo has been quiet since 2022. There is a question about updates in any case, since running it from the Neo4j dump file is also static.

1 Yes, there is no filter in the subsetting procedure. We have BioCypher settings to add the Neo4j admin commands to ignore these;

neo4j:
  skip_duplicate_nodes: true
  skip_bad_relationships: true

https://biocypher.org/BioCypher/reference/biocypher-config/

we should ensure that during the subset option, each edge is retained only if both source and also target nodes are present in the nodes subset

2 I think it depends on the absolute number, because this filtering during subsetting is also computationally expensive. What is your use case?