Another brake related question, does it make much of a difference which side the cable enters a v brake from (i.e. cable pulling from left or right, with a right hand lever).
V-brakes are notorious for being pushed off centre by the force exerted by the routing of the outer cable. The arms may need recentering, using the 2 small screws that act on the springs.
Whatever force you apply to the lever is applied in an equal and opposite way to the handlebar. The lever clamp is what closes that kinematic loop