I want students to acquire the ability to verify the sources of information from AI chats using scientific methods

Introduction

Working in the research sector, I often think about this. I want future students to be able to use AI as one of their tools. The key is to master it as a tool, rather than being used by it.

Regarding the rapidly emerging AI chat technology, I want students to acquire a solid foundation and methodology in science so they can verify the sources of the information it provides for themselves.

Today, I would like to talk about that.

In a world overflowing with information

It is not just about AI. The world is full of suspicious, pseudo-scientific expressions. Of course, I want to respect freedom of expression. On the other hand, we must not mislead people.

To acquire the ability to expose such things, we learn in places like schools from teachers who hold degrees such as associate, bachelor, master, or doctoral degrees—which serve as a guarantee that 'this person, while their field may be narrow, knows the truth, falsehoods, and definitions within that area.' Through learning, we can understand the style and methodology of science. Some people continue to pursue science and make it their profession.

From the perspective of someone with a doctoral degree, what skills are required to distinguish between what is scientific and what is not?

Immersed in the world of science

I earned my doctoral degree by thesis while working. I completed it while flying back and forth to visit my supervisor who was far away. Once I received my diploma from the university president, I began to notice suspicious expressions when reading papers in related fields, thinking, 'Wait, is that really true?'

If I read them like a daily routine, it becomes easier to judge, but if I step away for a while, my skills get rusty. This is because the world of science advances day by day. Yesterday's common knowledge could be overturned by tomorrow. I keep my ears perked like an antenna and search my research area thoroughly. Neglecting this is dangerous.

When moving outside of my specialty into peripheral areas, I have to read back through decades of reviews to understand the content before I can finally reach recent trends. Since it is like this even in areas near my narrow specialty, it is even harder in more distant fields. I go to the experts in those fields and ask for their guidance on my questions. Lectures and academic conferences are such opportunities.

Regardless of status or position, as long as you are a participant in the same conference or event, even a student can ask questions without hesitation, and if asked, one must answer sincerely and without discrimination. This is the rule.

What happens then?

Sometimes, you can gradually see the situation around you. It is like a momentary clearing in the fog. You can notice, 'Oh, this is strange.' I am particularly sensitive to descriptions in the very narrow area I have worked on myself. When I think, 'Where is the evidence for this statement? I can't find it...', it is usually suspicious. Even if I search, I cannot trace it back to primary information. Chat AI also does this quite often.

When I carefully inquire, the response is often vague, or I receive a thank you for pointing it out, saying they have corrected it immediately. If you are in a position to face science, it is easy for both parties to quickly sense whether it was a careless mistake or something else. You can understand it well by trying to replicate the experiment.

When you follow the procedures of solid, widely cited, and well-regarded work, the experimental results come out clearly. You can be convinced, 'I see, this is the point.' I am sure they must have struggled immensely to break through this.

Descriptions not based on empirical experiments or observational results are immediately apparent to those involved in the field. Even if you try to reproduce them, they do not work well. You might think, 'Is this where the misunderstanding lies?' It is common even in top-tier journals that experiments cannot be replicated or do not work. In fact, this happens often.

If something is written based only on belief rather than experiments or observations, the logic is likely to collapse. Even if it looks sophisticated, there may be leaps in logic, or it may be confusingly and strangely difficult. Since that is no longer science, I do not engage with it.

Conclusion

The sentence immediately above has become a typical example of "-nai" (negative) thinking. There is no point in getting worked up about it. Let's talk more positively. That is why I tell students that even if the initial results are clumsy, they should just observe them carefully as they are and record them exactly as they appear.

In fact, there are often interesting phenomena hidden within them that have never been seen before. How can one find them without overlooking them? Perhaps it requires acquiring the scientific practices mentioned above, while simultaneously maintaining an attitude of being honestly straightforward toward nature and repeating observations, along with the necessary patience for a while.

It could be said that it is just that kind of unglamorous work.

Through that process, one can leave behind something for oneself and those around them that rewards the effort of accumulating observations. Like a fragrant essence. That is certain, and there is no falsehood in it. When the truth cannot be understood from the data, one should simply leave it as a clear record and entrust it to future generations.

Related Articles

Advertisement