Just got stuck on this twice now. First time, he refused to blow up the tower - this was possibly triggered by my speaking with the Wyld Hunt tactician before reaching the tower? Second time around, I didn't interact with any of the npc's and we got the cutscene after the tower... but then he just stood there waiting for me to "talk" to him while I was only allowed to "greet".