It turns out that mouse detection in Windows is normally handled by the serenum.sys filter driver. This driver implements support for legacy serial mice along with serial plug-and-play. Microsoft has even provided the sourcecode as a WDK sample.
During detection the ports switches to 1200-7-N-1 mode while asserting DTR+RTS to which a response is expected within 200 ms, with a couple of retries in case of failure. Unfortunately for a legacy mouse a single M or B character suffices as identification.
In our case the protocol was reworked to avoid these characters and now appears not to be misidentified anymore.
However we were using a virtual USB serial port and for a traditional serial port this approach may be somewhat difficult as anything sent at a different baud rate is liable to look like line noise. In this case I suppose the easiest workaround is probably, as has already been suggested, to avoid making any unsolicited transmissions.
Alternatively with the serial control signals actually hooked up, or intercepted by a USB CDC device, processing the DTR or RTS signals and holding off on output. Actually implementing the plug-and-play protocol would be an even niftier option. Supposedly there are cheap RS232 cables around without a full complement of control signals though so this approach might still fail.