Just my opinion. I'd start looking at AV and any Malware programs you run. The lag could be any one of those items along with firewalls on the local machine. The reason I say this is that the latency seems to be with the packets themselves having to go through those programs.
Just a thought.