You could try to head down Portland st which runs parallel to Walworth > through Burgess > Southhampton way > Benhill and then up Camberwell Grove. It's quite a detour and the grove is a much steeper hill, but definitely quieter. And you can then head through lordship lane or the village from then on.
Edit: stevo's suggestion is probably less "quiet way"
You could try to head down Portland st which runs parallel to Walworth > through Burgess > Southhampton way > Benhill and then up Camberwell Grove. It's quite a detour and the grove is a much steeper hill, but definitely quieter. And you can then head through lordship lane or the village from then on.
Edit: stevo's suggestion is probably less "quiet way"