UE_Bot is a personal research and development project focused on creating AI-powered conversational characters for Unreal Engine. The project was developed to explore the complete AI pipeline required for natural voice interactions between players and virtual characters while maintaining control over the underlying systems through locally hosted models and self-managed infrastructure.

The original goal was to better understand how modern AI technologies could be integrated into interactive applications and to create a reusable framework capable of supporting intelligent NPCs within Unreal Engine. Rather than relying entirely on third-party services, the project investigates locally hosted language models, speech recognition, vector databases, retrieval systems, and middleware solutions that can be tailored to the needs of individual projects.

The system combines multiple technologies into a unified workflow:

• Real-time voice capture in Unreal Engine using C++.

• Speech-to-text processing through locally hosted transcription services.

• Large Language Model integration using locally hosted AI models through PrivateGPT.

• FastAPI middleware for communication between Unreal Engine and backend AI services.

• Text-to-speech generation and audio playback.

• MetaHuman facial animation and viseme generation driven by AI-generated speech.

• Retrieval-Augmented Generation (RAG) pipelines for custom knowledge integration.

• Local vector database experimentation for document retrieval and contextual memory.

From a technical perspective, the project required development across several disciplines including Unreal Engine C++, Blueprint systems, Python backend services, API integration, networking, audio processing, and AI infrastructure deployment. The backend environment was configured using WSL, local GPU acceleration, PrivateGPT, Faster-Whisper, and supporting services connected through custom middleware.

Beyond creating a functional chatbot, the project served as a platform for learning and experimentation with emerging AI technologies, conversational systems, and scalable NPC architectures. The long-term objective is to evolve the framework into a reusable solution capable of supporting multiple AI-driven characters with individualized knowledge, memory systems, function-calling capabilities, and simulation-aware behaviors.