Large Language Model Embeddings from SNOMED CT

Embeddings for SNOMED CT concepts produced by Large Language Models (LLMs). Each NPZ file encodes a dictionary, which links the ID of a SNOMED CT concept to its corresponding embedding. The embeddings from the LLMs were obtained using the method described in LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders (https://arxiv.org/abs/2404.05961) by prompting the model using the Fully-Specified Name of the concept.

File llm2vec_llama3_sct_dict.npz contains the embeddings extracted using Meta-Llama-3-8B-Instruct-mntp, whereas llm2vec_mistral7B_sct_dict.npz contains the embeddings generated using Mistral-7B-Instruct-v2-mntp. 

The metadata version of the files encodes the embeddings of metadata from SNOMED CT, such as relationships.

These embeddings were generated and studied in the paper Assessing the Effectiveness of Embedding Methods in Capturing Clinical Information from SNOMED CT () and more information can also be found in the following repository: https://github.com/JavierCastellD/AssessingSNOMEDEmbeddings.