The Verse of Hajj-at-Tamatu was revealed in Allah's Book, so we performed it with Messenger of Allah ﷺ, and nothing was revealed in Qur'an to make it illegal, nor did the Prophet ﷺ prohibit it till he died. But the man (who regarded it illegal) just expressed what his own mind suggested.
أُنْزِلَتْ آيَةُ الْمُتْعَةِ فِي كِتَابِ اللَّهِ فَفَعَلْنَاهَا مَعَ رَسُولِ اللَّهِ ﷺ وَلَمْ يُنْزَلْ قُرْآنٌ يُحَرِّمُهُ وَلَمْ يَنْهَ عَنْهَا حَتَّى مَاتَ قَالَ رَجُلٌ بِرَأْيِهِ مَا شَاءَ
There was revealed the verse of Tamattu' in Hajj in the Book of Allah and the Messenger of Allah ﷺ commanded us to perform it. and then no verse was revealed abrogating the Tamattu' (form of Hajj), and the Messenger of Allah ﷺ did not forbid to do it till he died. So whatever a person said was his personal opinion. (Using translation from Muslim 1226i)
نَزَلَتْ آيَةُ الْمُتْعَةِ يَعْنِي مُتْعَةَ الْحَجِّ فِي كِتَابِ اللهِ وَأَمَرَ بِهَا رَسُولُ اللهِ ﷺ لَمْ تَنْزِلْ آيَةٌ تَنْسَخُ آيَةَ مُتْعَةِ الْحَجِّ وَلَمْ يَنْهَ عَنْهَا رَسُولُ اللهِ ﷺ حَتَّى مَاتَ قَالَ رَجُلٌ بِرَأْيِهِ مَا شَاءَ