Group Assignment 4

Portfolio Exercise 4: Advanced AI Applications

Note: M4 - Final Assignment Deadline: Friday, 8 March 2024, 12:00 PM

LangChain Cheat Sheet

Introduction

This assignment is designed to explore the frontier of AI applications, focusing on the integration of Retrieval-Augmented Generation (RAG) with vector databases such as ChromDB and LanceDB, and the comparison of various prompt engineering techniques. The goal is to build an application that not only showcases advanced AI and DL capabilities but also evaluates the impact of different prompt strategies on model performance.

Objective

Task Description

Create an application that utilizes RAG and vector databases, and systematically compares the effectiveness of at least three distinct prompt engineering techniques.

Key Components

  • RAG and Vector Database Integration: Implement RAG with ChromDB and LanceDB to enhance information retrieval and content generation.
  • Transformer Model Adaptation: Use transformer models (SBERT or BERT)
  • Prompt Engineering Comparison: Experiment with and evaluate at least three different prompt engineering techniques to determine their impact on the model’s performance.
  • Platform Integration: The model should include a Gradio app (in-notebook) for demonstration purposes. Deployment on Hugging Face Spaces is optional for exploring additional features.

Additional Features (Nice-to-Have)

  • Fine-Tuning Capabilities: If possible, fine-tune a GPT model specific to your application’s needs, detailing the process and its impact on application performance.
  • Streamlit Application: Develop a Streamlit app hosted on the HF Hub, offering a richer, more interactive user experience.

Data

  • You may use open-source datasets or create your own data for the application.
  • Ensure that your data choice effectively demonstrates the capabilities of your application.

Submission

  • Create a GitHub repository specifically for this assignment.
  • Include all necessary materials, such as code, datasets, and a descriptive README.md.
  • Submissions can be individual or in groups of up to three members.
  • Submission also via DigitalExam, where you compile all your previous assignments and submit in one file for the overall portfolio for the module exam. You are welcome to tweak/improve previous module submissions for that.