insightql稿子

We’ll talk about the infrastructure requirement to support the InsightQL product in Phase 2. As is shown in breakdown here, we’ve identified a major limitation regarding throughput.

At the current stage, we are sharing one machine across all our environments. But in Phase 2, the volume increases a lot due to the large number of LLM request from CURA cases. As our estimation, there’ll be around 100 cases or 150 participants from CURA to be processed by InsightQL.

If we estimate 5 prompt for each participant and each prompt takes 1 minutes to complete, the CURA tasks will occupy LLM server for 12.5 hours per day. Again, the actual need is not cleared yet and need to be confirmed in later discussion.

To avoid the bottleneck that cause long waiting time and affect user experience, we need to use more AI servers to handle the LLM tasks parallelly.

Once we have InsightQL established, here is long-term roadmap for 2027 and later.

We have two main goals. The first is about Quality. We will adopt Multi-layer AI models to achieve a ‘Chain of Thought’ architecture. This means the AI can break the problem down into multiple steps, and deliver to different models.

For example, for a patient analysis tasks for specific disease, one model can preprocess and filter the HPO terms that user are interested. And another model use those result to do the analyzation. It allows us to handle these complicate task where the single prompt cannot.

The second goal is about Accessibility. We want to improve InsightQL by making use of data from Data Warehouse. Currently, InsightQL is a tool only for patient curation. In the future, it can become a business intelligence tool.

With MCP server, AI can fetch the real time patient statistic, or pipeline status, by querying from Data Warehouse. In this case, user can use InsightQL to do the auditing task or pipeline monitor task, where currently they can only check from statistic dashboard. In addition, if they have any new requirement, for example generating report to list patient with specific disease, they can use natural language to query from InsightQL, instead of waiting for developer to implement it.

It’ll boost user’s efficiency and also save developer’s effort to do such adhoc task, so that they can focus on more important system development.