Allah ﷻ prescribed the prayer, when He prescribed it, as two rakahs each, both while resident and while traveling. The prayer while traveling was left as it was, and the prayer while resident was increased.

فَرَضَ اللَّهُ الصَّلاَةَ حِينَ فَرَضَهَا رَكْعَتَيْنِ رَكْعَتَيْنِ فِي الْحَضَرِ وَالسَّفَرِ، فَأُقِرَّتْ صَلاَةُ السَّفَرِ، وَزِيدَ فِي صَلاَةِ الْحَضَرِ.

Ibn ʿUmar went on a journey and said, "I accompanied the Prophet ﷺ and he did not offer optional prayers during the journey, and Allah says: 'Verily! In Messenger of Allah ﷺ you have a good example to follow.' " (33.21)

صَحِبْتُ النَّبِيَّ ﷺ فَلَمْ أَرَهُ يُسَبِّحُ فِي السَّفَرِ وَقَالَ اللَّهُ جَلَّ ذِكْرُهُ {لَقَدْ كَانَ لَكُمْ فِي رَسُولِ اللَّهِ أُسْوَةٌ حَسَنَةٌ}

offered four rakʿat of Zuhr prayer with the Prophet ﷺ at Medina and two rakʿat at Dhul-Hulaifa. (i.e. shortened the ʿAsr prayer).

صَلَّيْتُ الظُّهْرَ مَعَ النَّبِيِّ ﷺ بِالْمَدِينَةِ أَرْبَعًا وَبِذِي الْحُلَيْفَةِ رَكْعَتَيْنِ

We set out with the Prophet ﷺ from Medinah to Makkah, and he would pray two rakahs at a time until we returned to Madinah. I said, “Did you stay in Makkah for any time?” He said, “We stayed there for ten days.”

خَرَجْنَا مَعَ النَّبِيِّ ﷺ مِنَ الْمَدِينَةِ إِلَى مَكَّةَ، فَكَانَ يُصَلِّي رَكْعَتَيْنِ رَكْعَتَيْنِ حَتَّى رَجَعْنَا إِلَى الْمَدِينَةِ. قُلْتُ: أَقَمْتُمْ بِمَكَّةَ شَيْئًا؟ قَالَ: أَقَمْنَا بِهَا عَشْرًا.