I replaced the battery on my 4 (I think it would be similar for the 4S).
£5 for a battery and tools off ebay and about 15 minutes using the iFixit guide. Definitely not an official battery but it worked fine and didn't explode.
Very easy to do yourself, especially on the 4s. If you google iPhone parts a few of the top sites will usually have genuine batteries for no more then £10. Replacing it takes 10-15 mins, just follow the ifixit guide.
How much does it cost to replace the battery in an iPhone 4s?
Got a hand-me-down and the battery is only 75% design capacity.