The 'Internal ChatGPT' Challenge: Why AI Search is Essential for Enterprises Now

In this article, based on an interview with Arvind Jain, CEO and co-founder of the enterprise AI search platform Glean, we explain everything from the evolution of search technology to the forefront of leveraging internal corporate knowledge. Drawing on his extensive experience at Google and Rubrik, we unravel how he led the paradigm shift from traditional keyword search to semantic search, and how he overcame the challenges of internal data acquisition and governance that were solved by the advent of the SaaS era.

1. Evolution of Search Technology via LLMs

1-1. From Keyword Matching to Semantic Matching

Traditional search was dominated by 'keyword matching,' which simply compares the words entered by a user with the words in a document. This had vulnerabilities such as 'keywords that don't hit' and 'inability to resolve ambiguous expressions,' but the emergence of Large Language Models (LLMs) has enabled deep understanding of the 'intent of the question' and the 'content of the document,' allowing for conceptual matching. Jain states, 'Search has entered a phase where it is no longer just a list of links, but generates direct answers to user questions.'

1-2. Transformer-based Embedding Search

In the early versions of Glean, we utilized transformer models like 'BERT' released by Google to quantify each document as an embedding. This method enabled conceptual search, known as 'vector search,' and was implemented as an internal embedding search even in an era when terms like 'RAG (Retrieval-Augmented Generation)' and 'Generative AI' were not yet generalized.

2. Building and Transforming the Glean Platform

2-1. Initial Phase: Introduction of Embedding Search

Glean, which began conceptualization at the end of 2018 and was founded in early 2019, placed embedding search at the core of its technology from the start. We mapped business content for each customer into a dedicated embedding space and utilized this as the foundation for our search engine.

2-2. Evolution of Models and Data Connectors

In recent years, Glean has evolved beyond a mere search function into a platform that allows users to cross-search internal and external information while interacting in a chat format, and to build business automation applications. By acquiring data from Slack, Google Drive, and various SaaS via APIs, it enables 'retrieval-augmented' models and 'custom app development.'

3. Challenges and Failure Factors in Enterprise Search

3-1. The Era of Hardware & On-Premise Dependency

In the past, methods like the 'Google Search Appliance,' which involved installing dedicated appliances on internal corporate servers to ingest data, were mainstream. However, installation costs were high and operations were complex, and with legacy systems, data acquisition and scaling became major barriers.

3-2. Solutions through the Rise of SaaS and APIs

With the spread of SaaS, an environment where all customers can use the same version and the same API has been established, realizing 'turnkey data connection.' Jain reflects, 'By connecting via APIs, we have become able to unify dispersed information and build search foundations rapidly.'

4. Security and Access Governance

4-1. Importance of Permission Management

Since 90% of corporate information is treated as confidential, simply returning search results carries a high risk of information leakage. Glean implements a function that links each document and user permission information at the time of indexing, and filters results according to the access rights of the logged-in user.

4-2. Strengthening Governance Functions

Furthermore, triggered by cases where 'confidential information was inadvertently exposed,' governance functions have been strengthened as a 'security layer' of the product, including document classification, sensitivity labeling, and the implementation of policies that query access permissions. As a result, Glean is increasingly being recognized as a governance platform itself.

5. Application Development and Business Process Application

5-1. From Personal Assistant to Custom Apps

Initially, under the banner of 'Google in your worklife,' Glean provided the Glean Assistant, which presents search results in a conversational format. However, as client companies requested curated apps for specific departments and tasks, such as 'HR-only' or 'Sales-only' tools, the company expanded into a suite of custom features called 'apps'.

5-2. Experiences for HR and Specific Business Tasks

For example, in the HR department, answers to questions regarding benefits and leave policies are generated by referencing only pre-approved documents. This goes beyond simple information retrieval to realize the automation and streamlining of business processes.

6. Product Roadmap and Future Outlook

6-1. Priority Products: Glean Assistant and AI Platform

Moving forward, the company will further advance the 'Glean Assistant' to pursue fully automated responses that leverage company-wide knowledge, while simultaneously enhancing its capabilities as a developer platform that allows companies to build their own unique AI applications.

6-2. Ongoing Challenges: Accuracy and Information Discovery

Mr. Jain points out that 'the biggest problem with AI is not hallucination, but the inability to find the correct information in the first place.' The key to future development lies in technology that can find a 'needle in a haystack' from vast amounts of internal content to provide the latest and most reliable knowledge.

Glean's challenge is an attempt to redefine the fundamental technical domain of 'search' by combining it with LLMs and SaaS-era technologies. In the future, it is expected that AI assistants will become a 'personal team' for individual employees, realizing a world where corporate knowledge is utilized to its fullest potential. The future, where knowledge discovery and business automation beyond the scope of search dramatically enhance the performance of people and organizations, is already steadily beginning to move.

Related Articles