It's not fenced so deer and rabbits have free access which could be an issue for fruit trees / bushes.
Might give currants a go and see how they do. I like the smell of the leaves, would be nice in summer. And I don't really mind if the birds eat all the fruit!
South aspect with heat-retaining brick wall suggests espaliered fruit trees to me,
or,
at least a selection of 'currants.