The Messenger of Allah ﷺ said: "If you say to your companion: 'Be quiet and listen' when the imam is delivering the Khutbah, you have engaged in idle speech." (Using translation from Nasāʾī 1577)
أَنَّ رَسُولَ اللهِ ﷺ قَالَ إِذَا قُلْتَ لِصَاحِبِكَ وَالْإِمَامُ يَخْطُبُ أَنْصِتْ فَقَدْ لَغَوْتَ