For most data elements, if you exclude them, there is no assumption about what the value is.  If you exclude/ignore Observation.performer, then you don't know who performed it.  If you exclude/ignore Observation.effective, then you don't know when the Observation was deemed to be true.

However, if you strip out Observation.focus (e.g. because you don't support it), the presumption of someone looking at the resource is that the focus is the same as Observation.subject.  That obviously changes the meaning of the instance.  If you have an observation of heart rate with a focus of fetus#1 and subject of mom, then ignoring the focus is going to change the interpretation of the Observation, and that's not safe.

It doesn't matter whether the definition of the Observation explicitly says that focus matters in interpreting the resource.  That's not enough to change the element from being a modifier.