If you (even) ask your companion to be quiet on Friday while the Imam is delivering the sermon, you have in fact talked irrelevance.
أَخْبَرَهُ أَنَّ رَسُولَ اللَّهِ ﷺ قَالَ إِذَا قُلْتَ لِصَاحِبِكَ أَنْصِتْ يَوْمَ الْجُمُعَةِ وَالإِمَامُ يَخْطُبُ فَقَدْ لَغَوْتَ