It's simple physics. Because the front guy is pushing through the air aerodynamically on his own - the air bulges round him, then comes together smoothly (see fig A), causing no drag and good speed. This is the view from above the cyclist...
When a drafter comes along he disrupts the air-flow behind the lead rider, which in effect pulls him backwards and slows him down. (see fig B).
Fig C is the lead rider doing a little skid because a taxi has pulled over in front of him without indicating. See the disrupted airflow and therefore reduction in speed.
How on earth has cyclist C attached a slotted leading-edge flap to his bike??
How on earth has cyclist C attached a slotted leading-edge flap to his bike??