The Importance of 'AI Onboarding' and the Challenge of the AI Workforce
I am Nakamura (@nrryuya_jp), Head of the AI/LLM Division at LayerX. Yesterday, we announced that our generative AI platform, 'Ai Workforce,' has been adopted by MUFG Bank. We have also launched a product introduction page.
LayerX is building its business by betting on the evolution of AI. AI will impact how many people work. However, even as AI becomes increasingly intelligent, it does not necessarily mean humans can use it exactly as they intend. We use the term 'onboarding' for when companies accept and integrate new employees, and for AI as well, 'AI onboarding' to create an environment where AI can work effectively is essential.
In this article, I will introduce this concept of 'AI onboarding' and the vision we want to realize with Ai Workforce.
The Necessity of 'AI Onboarding'
The Evolution of AI and LLMs
If we compare it to the human 'brain,' early machine learning was like a newborn baby suddenly learning a specific task (such as image recognition or fraud detection). It often required far more data than a human to master a single task, and (compared to today) the use cases that were actually practical were limited.
On the other hand, recent Large Language Models (LLMs), trained on vast amounts of information from the internet, could be described as 'world-class, highly knowledgeable new employees' in terms of sheer knowledge. Just like human new hires, they have the potential to possess general knowledge and common sense, and from there, bridge the gap to acquire company-specific knowledge and tasks. In terms of thinking and learning abilities beyond mere knowledge, they are currently inferior to humans in many ways. However, with technologies like OpenAI's recently announced o1, we believe the gap will gradually narrow as these technologies evolve.
AI is Not Being Properly Onboarded
However, we are still far from a world where LLMs are as active as human employees. Why is that? I believe it is because, compared to human new hires, LLMs have not yet received sufficient 'onboarding'.
Human employees grow by overcoming various joys and sorrows. Depending on the industry, it can take 5 to 20 years to become an 'ace' performer within a company. They discuss things with various people daily, receive guidance from supervisors, sometimes fail... compared to the experience humans gain over time, the information currently being provided to LLMs is very minimal.
I believe that even if AI can acquire the experience and skills that humans normally take 5 to 20 years to master in just a few months to a few years, it would bring a significant impact to many companies. It is natural that results will not appear immediately upon implementation, and shortening the lead time until AI becomes productive is what is important in AI onboarding.
Challenges of AI Onboarding
AI onboarding is still a new field, and we do not have all the answers ourselves. Below, I will introduce three specific challenges we are tackling.
How to Have AI Learn Work
First, here, I define 'learning work' as mastering the process of creating appropriate output from some form of input. For example, creating sales materials from customer information or making credit decisions from a company's financial statements.
I have organized the patterns of how AI learns human work into the following four quadrants. (Adapted from materials created for humans.)
First, the 'Manual Type' in the top left is about teaching AI a process that has been systematically organized from the start. A simple example would be instructing an LLM via a prompt based on an operations manual. However, because operations manuals created for humans often lack sufficient detail or are difficult for language models to reference, it is important to create 'operations manuals that are easy for AI to read.' The 'AI Workflow' feature in Ai Workforce corresponds to this.
The "Case Analysis" approach in the upper right corresponds to traditional supervised machine learning, as well as task planning from past cases using LLMs. It is necessary to design in advance so that input and output data are accumulated in a format that is easy for the AI to learn. Even in practical examples of an AI Workforce, it is rare to have a perfect business manual from the start, and business rules are often discovered from examples of human work.
In the "Feedback" approach in the lower left, humans review and correct the AI's output, and the AI improves based on that.
In an AI Workforce, since corrections made by humans are stored in a database, accuracy can be improved by referencing them in subsequent processing.
Finally, regarding the "Introspective" approach in the lower right, although it is generally the most difficult to implement, I believe it is the essence of AI's learning ability that surpasses humans and will become particularly important in the future. AlphaGo Zero, which gained its strength through self-play, is one example. I also view RLHF, which is already used for LLM training, as one method of performing "introspection" using human feedback data.
Just like humans, it is important to have the AI learn by combining and repeating the four approaches above. This means first mastering a somewhat systematized method, analyzing past cases from seniors to fill in the details, actually trying it out by thinking for oneself, receiving feedback on it, and learning while struggling on one's own.
How to convey intent and background to AI
When having an AI learn or take on work,correctly conveying the intent of "what you want the AI to do in the first place" is actually a very difficult problem.
Specifically, Specification-gaming is a known problem. The GIF animation below is a simulation of an AI robot that was supposed to have learned to stack a red block on top of a blue block.
As a result of defining the state of stacking blocks during training as "the bottom of the red block is at a higher position than the bottom of the blue block relative to the ground," it learned to simply flip the red block over instead of stacking the blocks.
Human work is far more complex than this and exists within a lot of "context." For example, "Create a proposal for Company XX" or "How can past internal cases and research results be utilized in developing a product like XX?" Even if you give the AI instructions in just one sentence,the expected results will not be achieved if implicit assumptions are not conveyed.
How to intervene in AI work
Even if it learns the work correctly, it is the same for both AI and humans that there is a possibility of making mistakes when actually doing it. Therefore, naturally, there is no such thing as leaving work entirely to the AI, and it is necessary for humans to "intervene" effectively.
In particular, the more authority and responsibility given to the AI, the higher the risk if a mistake occurs. Even now, for example, there are cases where it interacts directly with customers in customer support or links with other important systems, but this will become more pronounced as the scope of AI's activities expands in the future.
For humans to intervene in AI work and collaborate effectively, it is important that AI behavior is interpretable by humans. For example, when having an AI write code, if the AI uses an AI-specific programming language that cannot be converted into a language readable by humans, it becomes difficult for humans to ensure quality.
This 'ease of intervention' is also related to the ease of accumulating data necessary for the aforementioned 'feedback-type' learning.
Example: Is 'Doraemon' an ideal interface?
This is a slightly extreme example, but let's consider what would happen to the above problem if 'Doraemon' actually existed. Roughly speaking, Doraemon has a structure similar to a human (hands, feet, eyes, nose, mouth, and ears?). If it has a form close to a human, by definition, it would not be an interface that is harder to onboard than a new human employee. However, is it optimal? That is not necessarily the case.
For example, when teaching someone a job, have you ever thought, 'It would be so easy if I could just look inside this person's head'? Whether it's creating documents or pair programming, I think it would be easier to give advice if you knew which parts of the PC screen they were reading, in what order, and what they were thinking. However, as far as I know, Doraemon is like a human in that you cannot tell what it is thinking from the outside, so I do not believe it is an interface that surpasses humans in terms of 'ease of instruction'.Precisely because it is AI, we can provide a new interface for intelligence that is not bound by human bodies or communication methods.
Ai Workforce: An AI platform that grows with companies
As mentioned above, in order to utilize AI in business, we need to onboard AI ourselves rather than waiting for an AI that is excellent from day one to appear. Ai Workforce aims to be a platform that helps companies overcome this challenge and 'grow together with AI.' 'Growing together' does not just mean AI learning tasks; it is intended to mean thatbusiness methods and the business itself evolve through AI, and the company itself grows.
A platform that grows the more you use it
I believe an ideal AI platform is one that becomes smarter like a flywheel as its scope of use expands and data accumulates.
In the case of humans, knowledge and experience inevitably accumulate in each individual's head, and these basically become siloed, butAI can be consolidated as one massive intelligence for each company, so there is room for it to grow much faster and larger than humans.
Horizontality
Since AI is 'intelligence,' it will affect almost all tasks and businesses that humans are involved in. I believe it is important for an AI platform to be general-purpose rather than specialized in specific tasks or departments as it grows. Conventionally, many business software and SaaS products have been created in a form specialized for specific tasks. However,originally, each job is connected. For example, in the flow of planning a product, manufacturing it, marketing it to the market, sales selling it, accounting recording the revenue, and customer support handling any issues, various business contexts are constantly passed on, and tasks are linked. That is why Ai Workforce aims to be aplatform that transcends departments and tasks as a layer of data and intelligencebeyond mere system API integration.
The current status of Ai Workforce
Ai Workforce is a product that will continue to evolve in the future, and although it has just started, it has already begun to embody the above vision little by little. It is already being used for completely different tasks by customers in a wide range of completely different industries. Various AI workflows are being created, realizing knowledge sharing across teams and departments. From the data accumulated through these, 'agent functions' enable specialized problem solving and report generation.
If you are interested in using Ai Workforce, pleasedownload the materials from the product introduction page.
Challenges of the AI/LLM Division
In the AI/LLM Division, which develops and provides Ai Workforce, members of various teams and roles collaborate to take on the following challenges.
Building an 'AI-UX Factory'
As mentioned above, Ai Workforce aims to be a general-purpose platform, but I believe that the UX actually delivered to end users needs to be researched for each use case. As seen in the Doraemon example above, a single general-purpose form is not necessarily optimal for the interface between AI and humans. At LayerX, we want to be a company thatcontinues to propose ideal AI-UX (UX based on the premise of AI) in a reproducible form. To realize an ideal AI-UX for individual use cases while coexisting with a common foundation, and to maintain high development productivity and quality, advanced product management, the rhythm of the development team, and the power of design to deliver it to users are important.
Building a new way of working with AI
To realize the ideal AI-UX, not only AI and software but also the human side must change. In various tasks across different industries, we must work with customers who are experts in their fields to think about ideal operations and ways of working, verify hypotheses, and follow through until value is actually created for the user. At LayerX, we tackle this by collaborating with members from various roles, including account executives, project managers, sales engineers, and business architects.
AI Alignment for Enterprises
The concept of 'AI Onboarding' introduced here is, I believe, a business use-case perspective on the theme of "AI alignment," which has been studied for a long time in the AI field. We need R&D capabilities to create practical methods in light of new use cases and evolving foundation models, while referring to past research results.
If you are interested
The list of open positions is as follows! Please feel free to apply.
We also hold casual interviews with various members, including myself. (If you have a preference for which team member you would like to speak with, please let us know and we will arrange it.)