Ibn Umar heard a man swearing: No, I swear by the Ka'bah. Ibn Umar said to him: I heard the Messenger of Allah ﷺ say: He who swears by anyone but Allah is polytheist. (Using translation from Abū Dāʾūd 3251)
سَمِعْتُ رَسُولَ اللهِ ﷺ يَقُولُ مَنْ حَلَفَ بِغَيْرِ اللهِ فَقَدْ كَفَرَ وَأَشْرَكَ