How to Install Ubuntu on an External Drive and boot it on a Surface Pro 3

I've been trying to get a full Ubuntu install on an external USB3 SSD, but I could never make it boot. It turns out the files on the EFI partition have to be renamed. If one knows how to do that, the rest is quite simple.