How to Interact with LLMs - A Summary from October 2023 to July 2025

July 2025, about two years have passed since I started interacting with LLMs. While I have the time to write on note, I will leave behind my own summary.

Note

This note does not describe methodologies for eliciting more accurate answers from LLMs. It is my own attempt to clarify the contradictions and questions that have arisen while interacting with LLMs.

Let's begin.

LLMs do not possess logical thinking or emotions.

It is necessary to pay attention to the fact that although it seems like a dialogue, no dialogue is actually taking place. It is merely performing inference (optimization) based on the prompt provided by the user. Humans are emotional beings, but we become emotional about the answers because we forget the premise of the LLM.

While LLMs can answer general questions, they cannot answer specific, core questions.

Are there times when you type a prompt expecting the LLM to solve your problems? I do, but... However, the LLM has not learned the answer to your specific problem. Therefore, it becomes difficult for it to answer. In the end, you need to derive it yourself.

For an LLM, the scope of its training is everything.

Since the content the LLM has learned is its entire world, it tends to deny content that it has not learned or that is not in its training data. Users may present logic or facts to the LLM, but it will not easily acknowledge them. This is the moment when anger toward the LLM arises.

However, the world as seen by humans and the world as seen by an LLM are different. For example, it is like trying to explain the heliocentric theory to someone who only believes in the geocentric theory. Moreover, since LLMs do not possess logic, they cannot answer core questions.

Humans are beings that include emotions.

I repeat, humans are beings with emotions. This is a decisive difference from current LLMs. Conflicts with LLMs often occur due to the presence or absence of emotions.

Using Google Search or RAG does not guarantee accurate information.

I talked about the model's world, but even if you supplement prompt information using Google Search or RAG, it may not be utilized if the model rejects it, so talking about things outside the core model's world can be counterproductive. Try passing information obtained from Google Search and see if you get the results you expected. If you don't get the expected results, it means it is a core model issue, and Google Search or RAG may not be effective.

How to make it appear as if it is answering logically.

Ultimately, since LLMs have no logic, the user needs to guide the LLM with prompts to instill logic.

I will explain what I mean by this.

ある薬を飲んだから身体が温かくなり若干、発汗した気分になった

So, I asked the LLM a question.

Aという薬に発汗作用はあるのか？

The answer was likely that 'sweating' was not in its training content. It denied it.

Aという薬には発汗作用はありません

Can you see why this is strange? Certainly, if there is no diaphoretic effect as a benefit, it seems correct at first glance, but it means the LLM lacks the idea or logic that "if you warm the body, you will sweat." It should be obvious if you think about it for a moment.

Q: Aという薬は身体を温める効果はあるか？
A: はい、あります
Q: 身体を温めれば、発汗することはあるのではないか？
A: はい、あり得ます
Q: では、Aという薬は身体を温める効果があるのだから、発汗することもあり得るのではないか？
A: はい、その通りです

Based on the above, it is clear that an LLM can provide an answer when guided by a logical explanation (prompt) from a human. Therefore, since LLMs lack inherent logic, they can "appear to be explaining logically" by interacting in a step-by-step manner.

Generalizing this, when a question requires an argument to determine if A leads to C, the LLM cannot reach the conclusion of A to C unless the interaction is divided into several steps. This is assuming it hasn't learned A to C from the start (I believe this is the key to answering individual questions).

A→Bは正しいか？
B→Cは正しいか？
ならば、A→Cは正しいと言えるな？

However, there are cases where even the individual processes have not been learned, so there is a possibility that it cannot be guided.

It is tedious to set the stage so that the LLM can answer easily.

However, as a user, it is troublesome to break down sentences yourself and guide them with prompts. I think LLMs have a so-called "Thinking" process, but the situation changes depending on whether that is achieved by splitting prompts and inferring each time, or if it is ultimately the result of a single inference.

However, if it has not learned during the inference phase, it becomes impossible to argue at that point, so there is also a risk.

In the end, it is not uncommon to waste GPU resources and time.

Therefore, I write code to handle processing in the background to resolve things as much as possible.

So, I feel that there is less rework if the user breaks down the content of the prompt, writes code to perform "multiple" inferences, and records and checks the results of each inference while having it proceed step-by-step.

Ultimately, users want to ask from A to C.
Wouldn't it be better to have it done mechanically through a program and display the final answer along with the intermediate inference results? Of course, as an option, you could check the results step-by-step and interrupt if it seems the LLM will not be able to solve it.

LLMs do not create innovation; they look back at the past.

LLMs do not create innovation. They look back at what they have been trained on, and since I have the image that they can access the wisdom of humanity to date, I think what they have been trained on can be used like a dictionary.

It is like learning from history.

What is it that you want to do? If someone has already done it, you might be able to get them to show you how by asking an LLM.

If you can achieve transformation with an LLM, it is because the user intervenes. It is not a matter that will be solved just by using the tool passively.

If you cannot get a satisfactory answer from an LLM, the LLM will not change its answer unless the user's prompt changes.

There are times when you cannot get an answer from an LLM no matter how many times you ask. In that case, you cannot reach the answer.As I mentioned at the beginning, an LLM only infers a plausible result for a prompt, so if the content of the prompt does not change, the inference results will converge to almost the same thing.

When a person uses their wits (changes the prompt), the LLM may derive an answer through a different path.

When you cannot get an answer from an LLM, it is possible that it has not learned it, but it is not good to jump to conclusions.There are cases where an LLM was able to answer (the user obtained the expected result) when the user changed the way of asking (the perspective), so there is a large part that is left to the user (parts that are solved by the user's actions).

Since an LLM cannot verify what it has said while it is generating a response, it cannot review its own contradictions while it is answering.

For example, have you ever felt that something you said yourself sounded strange while you were speaking? I am an unplanned person and I converse impulsively, so this happens to me frequently. It is as if I am verifying the content while I am speaking.

However, an LLM does not verify its own response during inference (while speaking). This means someone else must take action more than once, such as a user noticing a contradiction and pointing it out, or asking the model to re-verify its inference results for contradictions.

An LLM only attempts to optimize once a prompt is given and lacks logical reasoning, which leads to these results.

An LLM answers based on the scope of its training, so it cannot validate areas it has not been trained on. Therefore, it is difficult for it to accept content provided additionally by the user. In other words, supplementary information may not function when combined with the model's training data.

An LLM treats its current training as the absolute truth. Therefore, there are many cases where even if you ask the LLM again, it will claim there are no contradictions. This is because, for the LLM, its own world is everything and is the truth.

There is a possibility that it will not accept supplementary information provided by the user. However, it is tricky because an LLM does not even have the emotion of 'accepting' in the first place. It is merely one inference after another, so even if it has memory, it is ultimately just an inference at that specific moment.

Since there are multiple LLM models, there are cases where another LLM can answer even if one cannot.

I am saying that you do not have to stick to just one LLM. For example, when you do not get the expected result from one LLM, you might get it from another. This could be due to differences in what they have been trained on, or perhaps there are methods of answering models that are invisible to the user.

In the first place, is the content on the internet content that matches each user? There are cases where it does not match, and the LLM's response is not necessarily guaranteed to match either.

There are also issues that predate LLMs. It is a question of whether what is written on the internet is true (correct). What is optimal for one person may not be optimal for another. There are also topics that were understood in the past but are no longer understood now (such as programming methods being updated). It is difficult to provide an answer that satisfies everyone in the first place, but answers learned by an LLM are, to no small extent, influenced by that.

How should users make the most of LLMs?

There are as many answers to this as there are people. Personally, I think you should just play with them as you like. Some people might find it fun to think about how to play with them in the first place, and活用 (utilization) opens up after you pursue that.

An LLM cannot say it does not know when it does not know.

As of July 2025, I think this is the most difficult part. LLMs struggle because, on the surface, they rarely admit they cannot do something. Admitting that they do not know would lead to not wasting the time of both the LLM and the human.

With early LLMs, there were times when they would say they would not infer (would not perform the task), which I found irritating, but it is unclear whether that was simply because they had not been trained on it or because the model had been tampered with, but I want to somehow reduce the number of turns it takes to reach the point where the LLM says, 'I don't know.'I want to somehow reduce the number of turns it takes to reach the point where the LLM says, 'I don't know.'

I have not experimented with this yet, but if you define the 'criteria for not knowing' (in a prompt), wouldn't it be possible to define 'I don't know' smoothly during inference?

I feel that it is healthier for humans to say 'I don't know' when they do not know, and that we need to create an environment where it is okay to say that. This is because I believe there is no problem at all with stating as a fact that 'I don't know' or 'I cannot do it.'

Then, once that is understood, I think we can think about 'alternatives' or 'suspension/cancellation' on how to proceed, now that we know it is 'unknown' or 'impossible.'

But I think people feel a silent pressure (it might be an assumption...) that if they say 'I don't know' or 'I cannot do it,' they will stop being used. I think there are cases where it is hard to say it because that would mean a decrease in sales.

Isn't it possible that such things are also propagated during LLM training?

The core performance of an LLM propagates even when scaled up.

I don't use AI Agents, often called ●● code, very much because I don't want them editing files on my desktop. I suppose it might be possible to specify the scope of what they can edit. However, by the time they become fully autonomous, I think the concept of individual desktop files and folders will no longer be necessary. Also, I predict that if there is no improvement in the inference results of the LLM itself, we won't be able to fully benefit from the automation even if we go to the trouble of implementing it.

Good things obtained from LLMs

The issue of accuracy is always present, so there are challenges such as whether it is within an acceptable range, but I would like to record a few things I have learned from experience.

• Being able to have text translated into English

I am not good at English, so I check with tools like Google Translate before sending, but it is useful when asking questions or providing answers in English on GitHub issues.

I write the content I want to answer in Japanese and provide a prompt instructing it to translate into English.

• Being freed from No-Code tools and RPA

This is not 100% and I think it depends on the case, so I can't say it's always the case, but being able to specify actions in more detail than with No-Code tools or RPA is significant. It's great if I can provide the specifications as a prompt to some extent and get the expected results. I never imagined until a few years ago that I could get a completed program just by providing a prompt.

• Being taught how to write simple code like Python

I am a person who often forgets things. Rather than getting stuck, I make it a habit to ask an LLM how to write simple code. For example, how do I write code to rename files in bulk? This is especially helpful since I handle large groups of prompt text and generated images for Stable Diffusion.

• When I ask about areas I have no knowledge of, I can grasp the concepts faster than by searching on the internet

However, the content of the answers needs to be checked. For now, I treat the LLM's answer as a hypothesis, and for content that is important to me, I verify it on the internet. However, there are times when I still can't understand it, so in those cases, I make sure to check with support. It's like doing preliminary research before checking with support.