Building a pipeline for large language model fine-tuning, with a semantic search application
Date:
Presented project building a pipeline for document ingestion and encoding as part of the process for preparing to fine-tune a large language model for aviation-specific applications. Project demonstrated semantic search over documents, ensuring that responses are grounded in truth and preventing hallucination. Demonstrated dramatic performance improvements over state of the art language models under aviation-specific questioning.