I think I have finally fixed the Xen issues. After slowly
wading through the basics of getting the syscall provider, the
fbt provider, and GPF's in kernel space resolved, it appears to work.
Life has been difficult because the key to the issues resolve around
paravirtualising certain instructions, but also, the way interrupts
are handled. The normal interrupt routine sequence doesnt work for
INT1, INT3 and PageFault interrupts.
What Linux does is provide two distinct interrupt routines - one for
normal hardware, and one for Xen. At a low level in the IDT handler,
it decides which one to use. (This is buried in the Xen handler
Part of the work is to have a similar mechanism - autodetect if we
are on a Xen host, and use the correct interrupt handlers. Fortunately,
the code in intr_x86-64.S is amenable to parameterisation via the
macro assembler, so the code for all the interrupts is one macro,
with some conditional assembly.
Another problem area is that when the system dies, hard, Xen is very
unforgiving and reports an issue but no easy way to diagnose, in simple
terms what happened. (Before modifying the page fault handler to
use the correct Xen calling sequence, Xen would kill the guest due
to issues in the page table; this appears to be bogus, and not
the true cause of the issue - that the interrupt stack wasnt
I now need to merge the Xen changes back to the mainline code, and
check it still compiles/works on the older kernels.