According to information given on the Abbey list of Vicars, this structure is the steeple of St. Paul's Church which used to be the Parish church at Malmesbury. Birdcage Walk is where the body of the church stood. It is now the Abbey Belltower.
According to the British Listed Buildings website, this structure is Grade I listed - for more information see their website at: British Listed Buildings.
When we next visit Malmesbury we will take more photographs of this steeple.