Yahya related to me that Malik said, "The sunna concerning the time of prayer on the ids of Fitr and Adha - and there is no disagreement amongst us about it - is that the imam leaves his house and as soon as he has reached the place of prayer the prayer falls due."

Yahya said that Malik was asked whether a man who prayed with the imam could leave before the khutba, and he said, "He should not leave until the imam leaves."  

قالَ يَحْيى، قالَ مالِكٌ: مَضَتِ السُّنَّةُ الَّتِي لاَ اخْتِلاَفَ فِيها عِنْدَنا، فِي وقْتِ الفِطْرِ والأضْحى، أنَّ الإمامَ يَخْرُجُ مِن مَنزِلِهِ قَدْرَ ما يَبْلُغُ مُصَلاَّهُ، وقَدْ حَلَّتِ الصَّلاَةُ.

قالَ يَحْيى: وسُئِلَ مالِكٌ عَنْ رَجُلٍ صَلّى مَعَ الإمامِ يَوْمَ الفِطْرِ، هَلْ لَهُ أنْ يَنْصَرِفَ قَبْلَ أنْ يَسْمَعَ الخُطْبَةَ؟ فَقالَ: لاَ يَنْصَرِفُ حَتّى يَنْصَرِفَ الإمامُ.