Race condition in PIC model

[parthy]

During MP safety tests I stumbled across spurious interrupts which I think are caused by a race condition between triggering and EOI. The scenario leading to an error is the following:

    Trigger Thread                Receiver Thread
    ------------------------------------------------------
    Set IRR
    Prioritize -> yes
    INTR
                                  INTA
                                  DEASS
                                  EOI: Clear ISR
    Set IRR
    Prioritize -> yes
                                  EOI: Prioritize -> yes
                                  INTR
                                  INTA
                                  DEASS
                                  EOI
    INTR
                                  INTA -> spurious!

This effect can be emphasized by adding some artificial delay between the trigger thread's prioritize function and the actual INTR message. In a very basic standalone PIC model I created to simplify hunting this issue, I found that the optimization setting is also important (with -O0 I have to add the delay to see anything at all).

Now I have basically two questions:

1. Does the above scenario make sense or am I missing something?
2. What is an appropriate solution to it? I don't think this is a MP-related problem, because the same course of events could occur between different threads on the same CPU. It is just highly unlikely to happen.

@vmmon, @blitz, @udosteinberg, any suggestions as to how this can be solved elegantly are very much appreciated.

[vmmon]

1. Does the above scenario make sense or am I missing something?

Your scenario makes sense. While the PIC state is synchronized via atomic operations, the calculation+sending+receiving of its output is not.

1. What is an appropriate solution to it?

• It is not sufficient to lock the PIC or the BUS, since the very same can happen when receiving the message at the LAPIC or the CPU.
• An inelegant solution would be to have a global lock serializing all threads.
• Adding a version number to the legacy messages could solve the problem as well. Each thread would atomically increment the version number in propagate_irq(). The LAPIC and the VCPU could then ignore outdated messages.

BTW, the very same could happen between the LAPIC and the VCPU, where fixing this race seems to be much more tricky...

[vmmon]

Another solution would be to move from a push to a pull model.

The device threads would still update the IRRs but would not propagate the INTR state.
Instead they would just notify the LAPIC and the VCPU that the INTR state has changed,
so that the VCPU thread could synchronize its state afterwards.