Polaritons, quasiparticles formed from the collective strong coupling of light and matter, have been shown for their capability to modify chemical reactions, energy and charge transport – amazing features that can revolutionize the way we control molecular properties. Many of these features originate from the delocalization of polaritons, i.e., polaritons possess delocalized wavefunctions, which is one of their hallmarks. Furthermore, polariton delocalization has long been assumed to be robust against disorder that is ubiquitous in chemical systems, yet, without being fully checked. Herein, we examined the criteria to ensure delocalization in molecular polaritons, and this study reveals that transition energy disorder destroys delocalization of polaritons. In order to mitigate the impact of disorder and restore delocalization, a collective coupling strength needs to exceed four times the standard deviation of energy disorder linewidth. This observation indicates a more stringent criterion for preserving the unique delocalization characteristics of polaritons compared to the conventionally adopted standard (Rabi splittings larger than photonic and molecular spectral linewidths). This work sheds light on previous polariton dynamic studies done by our group and others, explaining why the onset of modified dynamics is larger than the strong coupling criteria, and it provides an important threshold to reach polariton delocalization for chemical and material research under strong coupling.