Coming to the nuisance is no defence
The principle, established in Sturges v Bridgman that if something is a nuisance, the fact that it has been a nuisance for a long time without anyone complaining about it doesn't stop it being a nuisance. In Sturges v Bridgman it was an apothecary’s noisy mortar and pestle.
So if someone moves into the neighborhood and decides this thing everyone else doesn't consider a nuisance is a nuisance, then saying "the nuisance was here first" is no defence.
Can cricket ever be a nuisance?
There is a suggestion that either (a) this general principle doesn't apply to cricket or (b) cricket is, at law, not a nuisance (per Lord Denning MR's judgment in Miller v Jackson. Sadly Lord Denning articulated his famous view in the course of a dissenting judgment (the remainder of the court was sympathetic to Lord Denning’s excellent arguments but felt itself bound by the superior court judgment in Sturges v Bridgman, and for whatever reason, the Lintz Cricket Club did not appeal) so sadly, in the eyes of the common law, cricket remains susceptible to nuisance actions.