Accessible Generative AI: Trial-and-Error Steps for Successful Complex Research with Generative AI
In this article, I will explain how to conduct research that satisfies complex constraints—a key skill for maximizing the capabilities of generative AI—based on actual verification results. I chose selecting corporate accounting software as the subject for this verification because it serves as a representative example of a research project with complex constraints. The research method described in this article is a versatile approach that can be applied not only to accounting software but also to system selection and product comparisons involving various functions and constraints.
I verified a method using generative AI (Perplexity in this case) to list prerequisites and expected functions, and then have it select and compare accounting software that meets those criteria. Through the following trial-and-error steps, I was finally able to obtain the desired research results.
The First Barrier: The direction of the research results does not match expectations
Details of the issue
The first problem I faced was that the results output by the generative AI were in a completely different direction from what I expected. As a specific example, because I did not state the prerequisite of being a one-person company, the research results output were for accounting software intended for large corporations.
Also, although I initially focused on core accounting and financial closing functions, I forgot to include necessary functions such as payroll calculation and year-end tax adjustments in the conditions. Due to these omissions in the conditions, many proposals that did not meet my expectations were included.
Response
To eliminate omissions in the conditions, I repeated prompt revisions and executions more than 10 times. In response to the generative AI's output not meeting expectations, I improved the prompts to comprehensively list prerequisites (one-person company, bookkeeping knowledge level, cost-consciousness, etc.) and expected functions (accounting and financial closing, as well as payroll calculation, year-end tax adjustments, electronic tax filing, attendance management, electronic contracts, etc.).
What was important in this process was a step-by-step improvement approach. It was effective not to try to make everything perfect at once, but to add missing conditions while looking at the output results and repeat the revisions.
Result
I was finally able to obtain research results in the same direction as I expected. Candidates that matched the assumed constraints, such as the one-person corporate plans for Money Forward and freee, began to appear.
The Second Barrier: The problem of missing important points
Details of the issue
Although candidate accounting software was listed, the important information necessary for actual judgment was not clearly stated. For example, regarding functions not included in the plans listed as candidates, it was unclear whether they could be used by adding separate options, whether that was even possible, and if the former, what the cost would be.
As a specific example, it was not clear whether electronic filing of corporate tax was possible with the Money Forward one-person corporate plan, and decisive judgment material was lacking.
Response
I refined the condition descriptions and revised the prompt more than 5 times so that the expected information would be explicitly stated. At this stage, I had to think about how to refine the conditions while checking the details with regular Google searches. This felt like putting the cart before the horse, but it was also an effective means of improving the prompt.
Result
I was finally able to clearly obtain the expected detailed information. The functional limitations, option costs, and scope of support for each plan were specifically indicated, and the information necessary for actual selection judgment was gathered.
The Third Barrier: Unclear realistic impact
Details of the issue
Although detailed information could be obtained, the situation was such that I did not know to what extent the constraints included in the research results would actually have an impact. For example, even if the research results stated as a constraint of the Money Forward one-person corporate plan that "journal entries are limited to 500 per year," there was no material to judge whether this constraint would realistically be a problem.
Response
I asked the generative AI to simulate the impact of constraints by inputting specific assumptions (job type, journal entry content, transaction frequency, etc.). I also had it propose ways to ensure that constraints would not become an issue (e.g., methods to reduce the number of journal entries). This adjustment also required more than five rounds of trial and error.
Simulation MethodIn this, I had it calculate the annual number of journal entries by specifically assuming factors such as "average monthly transaction volume," "frequency of compound journal entries," and "number of adjusting journal entries." This allowed me to quantitatively evaluate the extent to which the 500-entry limit would actually pose a risk.
Results
I was finally able to select accounting software. In the end, it required about 20 prompt revisions, but it achieved a significant time reduction compared to researching it on my own.
Summary of Effective Approaches
Through this verification, the effective approaches that emerged for successfully researching complex constraints using generative AI are as follows:
Concretization and comprehensiveness of research conditions
Step-by-step prompt improvement
Clarification of evaluation criteria
Simulation of constraint impact
Combined use with information sources other than generative AI
The methods explained in this article can be applied to the selection of various products and services with complex constraints, such as HR systems, sales support tools, and manufacturing systems, in addition to accounting software selection. The key is to utilize generative AI not merely as an information search tool, but as a collaborative partner premised on step-by-step improvement.