CV joints tend to knock more when placed under the most load when turning through the tightest angle.
To make this easier to visualise, imagine a group of soldiers maching in ranks, and the leader asks them to turn. the soldiers on the inside if they are turning right, tend to just mark time, whereas the soldiers on the outer ranks carry on marching at pace to make the turn.
In your case, the joint on the o/s/f is turning a tighter angle at a lower speed, while the n/s/f turns at a lesser angle and higher speed assisted by the diff, otherwise the inner tyre would destroy itself.
As has already been said, joints usually fail when the protective boot splits, but it is not unheard of a joint to fail just due to excess wear and tear.
Best plan is to jack the car up safely, use an axle stand to support the car just in case of movement, then put the steering on lock with the car switched off and rotate the wheel by hand while looking at the joint and boot to see if anything is not right under there.
Worn joint also tend to lock and release when rotated rather than smoothly rotate (difficult to explain, but you'll know it when you see it, it's that obvious).
Hope it helps, difficult subject to explain in a single post, good luck