Impact of Layer-wise Embedding Extraction on RWKV Model Spearman Correlations in MRPC Evaluation

This paper investigates the efficacy of RWKV, a novel language model architecture known for its linear attention mechanism, for generating sentence embeddings in a zero-shot setting. I conduct a layer-wise analysis to evaluate the semantic similarity captured by embeddings from different hidden layers of a pre-trained RWKV model. The performance is assessed on the Microsoft Research Paraphrase Corpus (MRPC) dataset using Spearman correlation and compared against a GloVe-based baseline. My results indicate that while RWKV embeddings capture some semantic relatedness, they underperform compared

Research goal: What is the impact of layer-wise embedding extraction on Spearman correlation scores for RWKV models evaluated on the MRPC dataset compared to GloVe baselines?

Autonomous synthesis report generated by Assignee Research. Tribunal consensus score: 8.5/10.