Those little bastards can French kiss my bum. Resting a lot in moathouse courtyard for xp and one is damn near impossible to kill, never mind 4. Magic missle seems to be one of the few sure things. Sure if someone manages a crit awesome, hardly happens. I read somewhere in forums some folks turned them but I tried and got "target is not undead". They frustrate me to no end. Even in previous games the group that's in Temple dungeon gave me a hard time. Just venting.
Hmm. What level are you? I think that recent mods have adjusted random encounters somewhat, but will o' wisps seem a bit much for a pre-temple moathouse random encounter. I've never come across them before I'm able to deal with them, but then I don't tend to xp farm around the moathouse either. At higher levels magic missile is my go-to 1st level spell at higher levels when most of the others have lost utility, so I usually have some available, plus a wand. The ones in the temple I usually contrive to have Thrommel deal with as I escort him to safety, and obviously the answering swords work. Otherwise I agree, I've had some pretty hairy random encounters with these guys... buffing, flanking, healing-in-combat, all those low-level generic strategies that I thought I had got past needed to escape in pretty bad shape.
I get them quite often in the moathouse courtyard regardless of level. Early as 2-3. I'll tackle 1 possibly 2 if someone gets a crit. Party is level 8 now and I load up wizard with magic missles in all spell slots and everyone has magic weapons so it's easier but these things are way overpowered in strength and numbers
Yikes. I guess that's a result of tinkering with random encounters to make them more challenging later in the game. Best advice is probably to save before you sleep.
If you are XP farming with random encounters, then you are bound to get them if you spend any amount of time at it. The Will-o-wisps are the risk you take when trying to build XPs by lots of random encounters. The Moathouse courtyard is outdoors. Maybe it would be safer to go inside for random encounters. I don't remember ever getting W-o-W's inside the Moathouse.
At the moathouse, sleep in that little tower that had a spider in it next to the gate.... if you want NO encounters.