I attended the 'Id prayer along with the Messenger of Allah ﷺ. When he finished the prayer, he said: We shall deliver the sermon; he who likes to sit for listening to it may sit and he who likes to go away may go away. Abu Dawud said: this is a mursal tradition (i.e. the successor 'Ata directly reporting from the Prophet ﷺ and omitting the link of the Companions).  

شَهِدْتُ مَعَ رَسُولِ اللَّهِ ﷺ الْعِيدَ فَلَمَّا قَضَى الصَّلاَةَ قَالَ إِنَّا نَخْطُبُ فَمَنْ أَحَبَّ أَنْ يَجْلِسَ لِلْخُطْبَةِ فَلْيَجْلِسْ وَمَنْ أَحَبَّ أَنْ يَذْهَبَ فَلْيَذْهَبْ  

“I attended the ‘Eid prayer with the Messenger of Allah ﷺ. He led us in offering the ‘Eid prayer, then he said: ‘I have finished the prayer. Whoever wants to sit (and listen to) the sermon, then let him sit, and whoever wants to leave, then let him leave.’”  

حَضَرْتُ الْعِيدَ مَعَ رَسُولِ اللَّهِ ﷺ فَصَلَّى بِنَا الْعِيدَ ثُمَّ قَالَ قَدْ قَضَيْنَا الصَّلاَةَ فَمَنْ أَحَبَّ أَنْ يَجْلِسَ لِلْخُطْبَةِ فَلْيَجْلِسْ وَمَنْ أَحَبَّ أَنْ يَذْهَبَ فَلْيَذْهَبْ  

"إِنّا نَخْطُبُ فمن أحَبَّ أن يَجْلِسَ للخطبة فليجِلسْ، ومن أَحبَّ أن يذهبَ فليذهبْ".