When the Kaʿba was rebuilt, the Prophet ﷺ and ʿAbbas went to carry stones. ʿAbbas said to the Prophet ﷺ "(Take off and) put your waist sheet over your neck so that the stones may not hurt you." (But as soon as he took off his waist sheet) he fell unconscious on the ground with both his eyes towards the sky. When he came to his senses, he said, "My waist sheet! My waist sheet!" Then he tied his waist sheet (round his waist).  

لَمَّا بُنِيَتِ الْكَعْبَةُ ذَهَبَ النَّبِيُّ ﷺ وَعَبَّاسٌ يَنْقُلاَنِ الْحِجَارَةَ فَقَالَ عَبَّاسٌ لِلنَّبِيِّ ﷺ اجْعَلْ إِزَارَكَ عَلَى رَقَبَتِكَ يَقِيكَ مِنَ الْحِجَارَةِ فَخَرَّ إِلَى الأَرْضِ وَطَمَحَتْ عَيْنَاهُ إِلَى السَّمَاءِ ثُمَّ أَفَاقَ فَقَالَ إِزَارِي إِزَارِي فَشَدَّ عَلَيْهِ إِزَارَهُ  

In the lifetime of the Prophet ﷺ there was no wall around the Kaʿba and the people used to pray around the Kaʿba till ʿUmar became the Caliph and he built the wall around it. 'Ubaidullah further said, "Its wall was low, so Ibn Az-Zubair built it."  

عَنْ عَمْرِو بْنِ دِينَارٍ وَعُبَيْدِ اللَّهِ بْنِ أَبِي يَزِيدَ قَالاَ لَمْ يَكُنْ عَلَى عَهْدِ النَّبِيِّ ﷺ حَوْلَ الْبَيْتِ حَائِطٌ كَانُوا يُصَلُّونَ حَوْلَ الْبَيْتِ حَتَّى كَانَ عُمَرُ فَبَنَى حَوْلَهُ حَائِطًا قَالَ عُبَيْدُ اللَّهِ جَدْرُهُ قَصِيرٌ فَبَنَاهُ ابْنُ الزُّبَيْرِ