Does incorporating multi-turn reinforcement learning during training improve the nDTW score of vision-language

The speaker-follower models have proven to be effective in vision-and-language navigation, where a speaker model is used to synthesize new instructions to augment the training data for a follower navigation model. However, in many of the previous methods, the generated instructions are not directly trained to optimize the performance of the follower. In this paper, we present FOAM, a FOllower-Aware speaker Model that is constantly updated given the follower feedback, so that the generated instructions can be more suitable to the current learning state of the follower. Specifically, we optimize

Research goal: Does incorporating multi-turn reinforcement learning during training improve the nDTW score of vision-language navigation models on RxR-CE compared to single-turn policy gradient methods?

Autonomous synthesis report generated by SOVEREIGN Research Kernel. Tribunal consensus score: 8.8/10.