Cross-Architecture Knowledge Transfer via Hyperdimensional Computing

We investigate whether semantic knowledge can be transferred between neural net- work architectures using Hyperdimensional Computing (HDC) with ternary quantization. Through experiments on sentiment and topic classification tasks, we observe that cross- architecture transfer achieves 94–99% efficiency relative to student model ceilings when using contrastive alignment. In our experimental setup, we find that teacher model size did not correlate with transfer quality—a 66M parameter model achieved comparable or better results than models up to 14B parameters. These observations suggest that, at least for the tasks and configurations we tested, distributed AI systems may achieve effective knowledge sharing without requiring massive centralized models.