Evaluating and Enhancing LLMs for Multi-turn Text-to-SQL with Multiple Question Types

Recent advancements in large language models (LLMs) have boosted text-to-SQL systems, but many miss the mark on handling real-world conversations. This can lead to issues with tricky questions that SQL alone can't solve. To tackle this, we created MMSQL, a test suite that checks how well LLMs handle different question types and multi-turn chats. We tested popular LLMs and found what affects their performance. Plus, we developed a multi-agent system to better identify question types and choose the right strategies. Our experiments show this enhances model's ability to navigate the complexities of conversational dynamics. For a more detailed presentation, refer to the Page. The paper was accepted for presentation at IJCNN 2025.