-
It's not just fine, it's what you should do. If you aren't hitting numbers because you're tired from training or you have external stress impacting your performance then you should definitely back off, either by skipping sessions, reducing length or reducing intensity (depending on what the nature of the sessions are). Listen to your body as well as what coach/training plan/broscience/whatever says.
I think it's absolutely fine to drop the odd session if you don't want to / can't do it /aren't in the right frame of mind. I started one and wasn't enjoying it, had time pressure, so I just stopped and had a big lunch.