A Hybrid Search and RAG prompting solution using Python OpenAI API Embeddings persisted to a Pinecone vector database index and managed by LangChain. Demonstrates the following:
- System Prompting. How do use the system prompt to modify LLM text completion behavior.
- Templates. How to create templates in order keep your prompts DRY.
- LangChain. How to setup a project using LangChain as an alternative to vendor specific LLM PyPi packages.
- PDF Loader. a command-line pdf loader program that extracts text, vectorizes, and loads into a Pinecone dot product vector database that is dimensioned to match OpenAI embeddings.
Pinecone. How to create, load, and query a Pinecone vector database.
- Retrieval Augmented Generation (RAG). A chatGPT prompt based on a hybrid search retriever that locates relevant documents from the vector database and includes these in OpenAI prompts.
- System Prompting. How do use the system prompt to modify LLM text completion behavior.
- Templates. How to create templates in order keep your prompts DRY.
- LangChain. How to setup a project using LangChain as an alternative to vendor specific LLM PyPi packages.
- PDF Loader. a command-line pdf loader program that extracts text, vectorizes, and loads into a Pinecone dot product vector database that is dimensioned to match OpenAI embeddings. Pinecone. How to create, load, and query a Pinecone vector database.
- Retrieval Augmented Generation (RAG). A chatGPT prompt based on a hybrid search retriever that locates relevant documents from the vector database and includes these in OpenAI prompts.