Cell search

[ChasingNeutrons]

I've implemented neighbour lists and cell search sorting (as in Serpent) to speed up particle transport in large cell universes.

The cell search reordering, on an ATR model, gave a >2x speed-up for 200k particles per cycle when doing a pure delta tracking run. The neighbour list gives a ~60x speed-up for the same system when doing a pure surface tracking run.

The neighbour list required implementing a linked list (following the recommendations from Harper et al). This is likely not implemented optimally.

Additionally, cell reordering is challenging to do in a thread-safe manner. This is because, to prevent breaking interfaces, it needs to be done during particle transport. I have done it by implementing something like a semaphore: a flag that says when reordering is happening, use naive cell ordering. This feels like it could be somewhat risky... Would be glad to hear any further thoughts on this.

[ChasingNeutrons]

I'm sorry to make this bigger. I have added overlaps. However, these were closely associated with the cellUniverse. Additional modifications to other files are very small, I promise.

[valeriaRaffuzzi]

I confirm that anything geometry related just breaks me......

Other than minor aesthetic comments, a couple of more high level questions:

• what is the point of linked lists? I haven't read Harper's paper, but to me it seems like the same function could have been done with our dynArr, for example.

• I understand why the semaphore thing is a bit uncomfortable.. Ideally, the reordering would happen occasionally at the end of cycle so that only one thread looks at it - but the interface change is annoying. I'm not sure I have a better idea there.

• I am also not fully convinced by the criterion for reordering. You check the frequency of the local cell being visited, rather than total number of cell visits.

• Could you maybe save an old cell seach list and a new one? Now, during cell reordering, the other threads go back to an ordered list. If you could provisionally save the old list, then the other threads could use that instead during reordering. This however might be a waste of memory and buy very little if the cell reordering is fast.

Also, I haven't comment on the BEAVRS inputs but I leave it to you to fix the conflicts!

[ChasingNeutrons]

Sorry, forgot to respond to comments above.
We don't use dynArray because we may add to the neighbour list frequently. For an array, this implies allocating and deallocating frequently, which may have a performance impact. It may also hit performance in that, when adding a new entry, other threads cannot use the neighbour list. Neither of these are the case with the linked list, which can add entries easily. The penalty is the usual one for linked lists versus arrays, i.e., slower access time (especially for large element numbers). As I flagged, accessing large elements numbers is not implemented at all efficiently at the moment, but it doesn't seem to impact performance significantly.

Agreed on not really knowing what to do with the geometry update. I suggested in one of the comments that we might want to make a generic 'geometryUpdate' subroutine for time-dependent things, but even that isn't perfect - for time-dependent we probably want to update geometry (or have a time-dependent geometry) that would look different during a cycle.

I am not convinced about the reordering criterion either. This was plucked from the air. There may be better ways of doing it, but for now this offered a meaningful improvement. This could be optimised further later.

That is a good idea on what to do during reordering. However, I think that may also be worth investigating later since for now everything seems to perform well.

[valeriaRaffuzzi]

This is ok for now then! Thanks for the implementation :)