ChronoRAG : Detecting Events from Emails using Time-based Retrieval for RAG

Nowadays, a lot of decision-making occurs through email communication and online meetings. A project plan might change over time and the final deliverable might look different than what was planned during the starting phase of the project. These changes might be a result of multiple events which lead to different decisions and plans. Post-project retrospectives often require understanding the guiding factors behind these decisions, the timing of each decision, and the events that triggered them. This involves a time-based analysis of internal documents, email communications, meeting notes, and other data points pertaining to the project.

We demonstrate a time-based retrieval approach for RAG(Retrieval Augmented Generation) to detect events from a massive textual knowledge base. RAG has proven to be the technique for enhancing the accuracy and reliability of LLM-generated responses with facts fetched from external sources [1]. Without RAG, the LLM takes the user input and creates a response based on information it was trained on—or what it already knows. With RAG, an information retrieval component is introduced that utilizes the user input to first pull information from a given data source. The user query and the relevant information are both given to the LLM. The LLM uses the new knowledge and its training data to create better responses [2]. A simple retrieval approach retrieves data that are most semantically similar to a given user query. The time-based retrieval approach retrieves data that are semantically most similar and pertinent to a specific time frame. In time-based retrieval for RAG, time filtering is done before performing semantic similarity. Data that match the time filter are used for semantic retrieval. 

We used Qdrant [3] for our knowledge store and LangChain [4] framework to implement document ingestion, prompt management, and RAG. The knowledge store included emails, internal documents, meeting transcripts, and in-depth interviews of team members. After some data cleaning, the email data was divided into weeks (“week 1” to “week 100”) to track weekly changes. To track the decision changes and events, we used a chained RAG approach, wherein RAG was first applied to “week 1”  and the responses were used as context for RAG with a filter for “week 2”. To classify the events and to extract the entities from the events, we use Marvin[5]. In this poster, we will present the architecture used to track events from email data leveraging time-based retrieval for RAG.

References

[1] Lewis, P., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks”, arXiv e-prints, 2020. doi:10.48550/arXiv.2005.11401.

[2] https://aws.amazon.com/what-is/retrieval-augmented-generation/

[3] https://qdrant.tech/

[4] https://www.langchain.com/ 

[5] https://www.askmarvin.ai/