When Messenger of Allah ﷺ arrived in Mecca, he refused to enter the Kaʿba while there were idols in it. So he ordered that they be taken out. The pictures of the (Prophets) Abraham and Ishmael, holding arrows of divination in their hands, were carried out. The Prophet ﷺ said, "May Allah ruin them (i.e. the infidels) for they knew very well that they (i.e. Abraham and Ishmael) never drew lots by these (divination arrows). Then the Prophet ﷺ entered the Kaʿba and said. "Allahu Akbar" in all its directions and came out and not offer any prayer therein.
أَنَّ رَسُولَ اللَّهِ ﷺ لَمَّا قَدِمَ مَكَّةَ أَبَى أَنْ يَدْخُلَ الْبَيْتَ وَفِيهِ الآلِهَةُ فَأَمَرَ بِهَا فَأُخْرِجَتْ فَأُخْرِجَ صُورَةُ إِبْرَاهِيمَ وَإِسْمَاعِيلَ فِي أَيْدِيهِمَا مِنَ الأَزْلاَمِ فَقَالَ النَّبِيُّ ﷺ قَاتَلَهُمُ اللَّهُ لَقَدْ عَلِمُوا مَا اسْتَقْسَمَا بِهَا قَطُّ ثُمَّ دَخَلَ الْبَيْتَ فَكَبَّرَ فِي نَوَاحِي الْبَيْتِ وَخَرَجَ وَلَمْ يُصَلِّ فِيهِ