[FIXED] Langchain RetrievalQA: Missing some input keys

Issue

I have created a RetrievalQA Chain, but facing an issue. When calling the Chain, I get the following error: ValueError: Missing some input keys: {'query', 'typescript_string'}

My code looks as follows:

from langchain_community.embeddings import HuggingFaceEmbeddings

embeddings = HuggingFaceEmbeddings(model_name="intfloat/multilingual-e5-large",
                                       model_kwargs={'device': 'mps'}, encode_kwargs={'device': 'mps', 'batch_size': 32})

vectorstore = FAISS.load_local("vectorstore/db_faiss_bkb", embeddings)
retriever = vectorstore.as_retriever(search_kwargs={'k': 1, 'score_treshold': 0.75}, search_type="similarity")
llm = build_llm("modelle/mixtral-8x7b-instruct-v0.1.Q5_K_M.gguf")

def build_retrieval_qa(llm, prompt, vectordb):
    chain_type_kwargs={ "prompt": prompt,
                        "verbose": False
                        }
    dbqa = RetrievalQA.from_chain_type(llm=llm,
                                        chain_type='stuff',
                                        retriever=vectordb,
                                        return_source_documents=True,
                                        chain_type_kwargs=chain_type_kwargs,
                                        verbose=True
                                        )
    return dbqa

prompt = """
<s> [INST] You are getting a task and a User input. If there is relevant information in the context, please add this information to your answer.

### Here the Task: ###
{typescript_string}

### Here the context: ###
{context}

### Here the User Input: ###
{query}

Answer: [/INST]
"""

prompt_temp = PromptTemplate(template=prompt, input_variables=["typescript_string", "context", "query"])

dbqa1 = build_retrieval_qa(llm=llm,prompt=prompt_temp,vectordb=retriever)

question = "What is IGB?"
types = "Answer shortly!"

dbqa1({"query": question, "typescript_string": types})

With this code the error from above occurs in the last line here.

The weird thing is, that it is working with a LLM-Chain from Langchain without Retrieval:

from langchain.chains import LLMChain

llm_chain = LLMChain(
        llm=llm,
        prompt= prompt_temp,
        verbose=True,
)

test = llm_chain({"type_string": types, "input": question})
test

This works and I am getting a correct response. I am using

Langchain == 0.1.0

So is there something wrong with my PromptTemplate?

Solution

Found a way to solve the issue posted here in this discussion: https://github.com/langchain-ai/langchain/discussions/11542#discussioncomment-8184412

Here the answer if the webpage changes: So got it working but figured out a weird behaviour. My prompt & code looks as follows and it works:

rag_prompt= """
<s> [INST] Im folgenden bekommst du eine Aufgabe. Erledige diese anhand des User Inputs.
Falls im Kontext relevante Informationen enthalten sind, die deine Antwort verbessern, füge diese in deine Antwort ein. 
Wichitg ist, dass du keine zusätzlichen Informationen erfindest, sondern gegebenenfalls nur die im Kontext enthaltenen.
Die Antwort soll sich nicht wiederholen.

### Hier die Aufgabe: ###
{typescript_string}

### Hier Kontext zum User Input: ###
{{context}}

### Hier der User Input: ###
{{question}}

Antwort: [/INST]
"""

rag_prompt = rag_prompt.format(typescript_string="Schreibe die Stichpunkte in einen Volltext um.")
def set_rag_prompt():
    return PromptTemplate(template=rag_prompt, input_variables=['typescript_string', 'context', 'question'])

from langchain_community.embeddings import HuggingFaceEmbeddings

embeddings = HuggingFaceEmbeddings(model_name="intfloat/multilingual-e5-large",
                                       model_kwargs={'device': 'mps'}, encode_kwargs={'device': 'mps', 'batch_size': 32})

vectorstore = FAISS.load_local("vectorstore/db_faiss_bkb", embeddings)
retriever = vectorstore.as_retriever(search_kwargs={'k': 1, 'score_treshold': 0.75}, search_type="similarity")
llm = build_llm("modelle/mixtral-8x7b-instruct-v0.1.Q5_K_M.gguf")
qa_prompt = set_rag_prompt()
def build_retrieval_qa(llm, prompt, vectordb):
    chain_type_kwargs={ "prompt": prompt,
                        "verbose": False
                        }
    dbqa = RetrievalQA.from_chain_type(llm=llm,
                                        chain_type='stuff',
                                        retriever=vectordb,
                                        return_source_documents=True,
                                        chain_type_kwargs=chain_type_kwargs,
                                        verbose=True
                                        )
    return dbqa


dbqa = build_retrieval_qa(llm=llm,prompt=qa_prompt,vectordb=retriever)
dbqa({"query": "Test Query"})

The weird behaviour is happening in naming the Prompt input variables. It is only working if the input_variable pair is "context" and "question". Before (when it did not work), "{question}" was named "{query}" in the Prompt and when calling the RetrievalQA chain with dbqa({"query": "Test Query"}) it gave me a "Key Error {query}". After renaming it to "question" the code runs through.

Answered By - Maxl Gemeinderat