We'd have to clarify what free speech is, and also account for damage done by deliberate lies. In other words, opinions are free speech but facts which are demonstrable false are not.

The only time anyone would bother to take action on it would be if actual damage were done. And that's why yelling "Fire" in a theatre when there is no fire is not free speech.

We already know there needs to be a distinction made here--it's not as hard as you make it sound and it's not about point of view either.