"A Cane And A High Starched Collar" bears a remarkable resemblance to the far older song, known variously as "Brighton Camp," "Blyth Camp," "Waxie's Dargle," but is perhaps best known as "The Girl I Left Behind Me."
The tune probably originates from Ireland, dating as far back as the 18th and perhaps even the 17th century.
The first recording of "The Girl I Left Behind Me" was probably made by the Columbia Drum, Fife and Bugle Corps in 1899 and released on a brown wax cylinder, number 12800.
https://davidneale.eu/elvis/originals/list1.html#S1721
Listen to a sample on David Neale’s site (see link above).
Another source tells us of a later recording:
‘The girl I left behind me’ (medley) recorded [by US Marine Fife and Drum Corps] on an Edison black wax cylinder circa 1904.
"Although probably an old Irish tune, "The Girl I Left Behind Me" became a popular British marching song under the title "Brighton Camp." In the years before the American revolution, it was often played when a British naval vessel set sail or an army unit left for service abroad. "The Girl I Left Behind Me" was adopted by the Americans and has become a traditional army song especially associated with the Seventh Infantry. It was also a favorite with the troops at Fort Snelling in the 19th century. Even today it is played at the United States Military Academy at West Point as part of the medley for the cadet's final formation for graduation."
https://archive.org/details/thegirl1904
Listen to a sample by tapping the link above.