Fix MSv3b ELEMENT_OFFSET dimensions

[calgray]

According to the MSv3b spec the PHASED_ARRAY/ELEMENT_OFFSET column has shape (3, Nant), but OSKAR outputs in the reversed dimension order.

[fdulwich]

Thanks! I'd like @tammojan to review this if possible. (I thought the OSKAR MS writer was now consistent with what EveryBeam was expecting in the PHASED_ARRAY table, so it would be good to double check.)

[tammojan]

I think we have used EveryBeam successfully on MSs produced by OSKAR, so I'm very wary of changing this. However, just from looking at the code, this change looks correct. In EveryBeam, we have here:

  assert(aips_offset.shape().isEqual(casacore::IPosition(2, 3, aips_offset.ncolumn())));

where aips_offset is a casacore::Matrix holding the element offsets for a single station, straight from the MS.

To validate the change, I strongly suggest taking a MS produced by OSKAR in the current state, and comparing that with a LOFAR MeasurementSet. It is very easy to get confused about transposes since casacore uses Fortran-order internally.

[fdulwich]

This is what the current element offset column looks like from OSKAR.

[calgray]

It was noticed OSKAR and EveryBeam have the reverse order compared to the format document when testing SKA phased array outputs. Not urgent for us as we can also output in reverse order, but ideally either the handful software packages eventually correct for it, or a new MS definition gets released.

[fdulwich]

Thanks @calgray that's good to know. It would probably be good just to check a LOFAR MS as well, if @tammojan has one available. If there are lots of LOFAR MSes around with the current ordering, and if there aren't any other packages using that sub-table at the moment, it might be a case of updating the documentation to match what currently exists than risk breaking lots of other things - but definitely good to review it. I'll defer to the experts here!

[tammojan]

Yuk. A LOFAR MS indeed has the opposite ordering:

However, looking closer at EveryBeam, the ELEMENT_OFFSET used for a MSv3 and a LOFAR MS is indeed transposed:
LOFAR:

assert(aips_offset.shape().isEqual(casacore::IPosition(2, 3, aips_offset.ncolumn())));

MSv3:

assert(aips_offset.shape().isEqual(casacore::IPosition(2, aips_offset.nrow(), 3)));

My apologies for stating that it should be 'just like LOFAR' and causing confusion there.

I'm wary of making code changes at this point since OSKAR is not the only package creating MSv3's that may not be compliant with the draft MSv3 standard or EveryBeam.

[fdulwich]

Thanks @tammojan for checking a LOFAR MS, and interesting to see that it's different there. It seems that OSKAR is compliant with EveryBeam, but not with the MSv3 document. We can certainly change what OSKAR is doing if that's a good idea, but presumably EveryBeam would need to be changed as well. I'm wondering if there was a reason for the change in the MSv3 implementation there compared with the LOFAR MS that we may be overlooking, though?