refactor($scope): prevent multiple calls to listener on `$destroy`

Prevent isolated scopes from having listeners that get called
multiple times when on `$destroy`