The people (mostly) were workers and they would come for Friday prayer in the same condition, so it was said to them: If only you were to perform Ghusl.
كَانَ النَّاسُ مُهَّانَ أَنْفُسِهِمْ فَيَرُوحُونَ إِلَى الْجُمُعَةِ بِهَيْئَتِهِمْ فَقِيلَ لَهُمْ لَوِ اغْتَسَلْتُمْ