Building on historic Wortley sports pitch like 'constructing houses on City Square', claims former MP