I would put it on the undersides yes - just as a precaution. I just tend to use up the full bottle/can of the stuff and slap it everywhere. You can either use a brush or a pump spray if you have one. I opted for brush in the hope the treatment was less airborne than the spray.
I would also get some treatment for woodworm on those joists to be safe just incase any of it is active.