You can micro-analyse it all you want, it's not a foul. You can apply the rule book back down to a T, it still doesn't matter. Referees have the power of discretion, context is everything in football, it's a completely different sport to American football or basketball in how the rules are applied. This is a fact, it's been this way forever. That was not enough to give a penalty, the tug was minimal. Contact like that happens ALL the time, you can't give frees for it or it would destroy the game. Especially not penalties which are traditionally harder to win (that's a debate for another time).
Incidental contact is usually not a foul and shouldn't be. Clashing of heads, arms across the face, even elbows, hard tackles. They all depend on context. Again, it's a contact sport. That means [some] contact is allowed depending on how it happens. I have no desire to see football turn into a sport where simply touching someone is a foul. It's heading that way unfortunately. I agree that defenders do get the benefit of the doubt btw, and keepers especially (as seen by Ceaser getting a free kick undeservedly after Olic headed it and Croatia scored - another injustice). But that doesn't mean overcompensating is right. It was never a penalty. Refs need to be consistent and that just wasn't close enough to be a penalty.