Generative AI is bad at maps. I thought about how to fix that.
For visually impaired people, every voice navigation system is difficult to use. On the other hand, generative AI cannot handle maps. Even so, we want to walk alone. This is a challenge and a proposal regarding that.
[Background Explanation]
As my handle suggests, I have built a voice navigation system for myself and have been using it to walk for about eight years.
And I have been thinking about the fundamental reason why voice navigation systems are not spreading among visually impaired people.
To put the conclusion simply, it comes down to the fact that "sighted people and visually impaired people want different functions from voice navigation. That is not being taken into account."
If the developers tried walking around blindfolded themselves, they would surely understand a great deal.
[Differences in required functions for walking navigation, Sighted vs. Visually Impaired]
Voice navigation for sighted people:
Provide information that cannot be understood by looking. For example, the route to a first-time destination.
Voice navigation for visually impaired people:
I want you to supplement the information that should be obtained from vision.
General walking navigation does not provide information that you can understand just by looking. Therefore, simply converting that into voice does not provide the necessary information, and unnecessary information flows in, wasting time on the timeline.
[Uses and requirements for navigation for the visually impaired]
I want to go to the dentist I visit by myself. (There are no sighted people who use navigation to go to a dentist they visit regularly.)
・ I want to use a safe and familiar route to get to the dentist, even if it is a detour.
・ I want to use a route that has tactile paving or pedestrian bridges, even if it is a detour.
・ I want to walk the familiar routes I used to walk when I could see.
These are the urgent needs.
[MUST items for considering navigation specifications]
① The user cannot see the surroundings or the map.
② Button operations are performed relying on the voice reactions generated by the screen reader. The surroundings are noisy, and I am holding a white cane in my right hand.
[Realization of required specifications]
To solve the above, I believe that interacting with devices through natural language dialogue is the most suitable approach. And that is where expectations for generative AI are growing.
[Generative AI is bad at handling maps]
I looked into the fact that Google's Gemini and Google Maps work in tandem, but I was disappointed with the results. I will omit the details, but current generative AI cannot handle map data directly.
I think it would be wonderful if you could change your driving route while driving using a car navigation system. The reason that does not exist is that it is not yet possible.
Below, I will describe how to realize interactive voice navigation using generative AI that cannot handle maps directly.
[I want to customize routes through dialogue]
Any navigation system will suggest the shortest route or a few additional options when you enter a destination.
The following problems occur here:
(1) If you cannot see the map, it is almost impossible to confirm (understand) the route in advance.
(2) It is almost impossible to modify the proposed route.
Therefore, I thought of an interactive solution using generative AI.
[A preliminary solution using generative AI]
Below, I will use the term Google Maps as a representative example, but other map databases would also work.
■ Outline
Have it suggest many routes connecting the starting point and the destination. For example, several dozen.
Select the desired route from among those routes through dialogue. The idea is that if there are a large number of candidate routes, the route you want will surely be included.
You use interactive generative AI to select your preferred route from these many candidate routes.
[I tried creating a walking route in Matsumoto City]
The Metoba River flows between Matsumoto Station and Matsumoto Castle, and many roads converge at Metoba Bridge. Here, I will describe a method for suggesting many walking routes from the station to Metoba Bridge.
(1) Set the starting point to in front of Matsumoto Station and the destination to Metoba Bridge.
2. Connect the start and end points with a straight line. Create a straight line about 500m long that intersects perpendicularly at the midpoint. The overall shape becomes a cross.
3. Create 'intermediate waypoints' at intervals of about 50m on the perpendicularly created line segment. Create them mechanically using trigonometry.
With this, about 10 sets of start-intermediate waypoint-end point paths are created, and I have the system search for those routes. It does not matter if the intermediate waypoints are outside the path, such as inside a building.
As an actual task, about 12 routes were created, and of those, about 6 remained as practical routes.
As a result, these 6 routes covered almost all realistic walking paths.
I will introduce two of them with images.
The first image shows:
The second image shows:
On this route, tactile paving is laid along the entire path. Note that many white cane users generally remember which roads have tactile paving.
[Procedure for customizing routes through dialogue]
Google Maps outputs route data as search results via API.
If you apply appropriate transformations to that route data, you can walk from the start to the end while stopping every 3m to watch the surroundings. Of course, this is virtual.
With this, you can obtain additional relevant information for the route in question.
For example: crosswalks, railroad crossings, names of facilities within 20m, bridges, underpasses, turning angles at corners, etc.
If you analyze satellite photos, you can extract zebra crossings and tactile paving.
Associate this vast amount of incidental information with the route and store it in memory. This is a task that probably finishes in a few seconds.
[Starting a dialogue with the user]
Generative AI searches and responds to that massive amount of information (text information) while interacting with the user.
For example: 'Please exclude routes that go through railroad crossings.' '5 routes remain.' 'Please choose the route with the fewest turns.' '1 route matches.' 'How many meters is that route?' 'It is 650m.'
With that said, I will start the navigation if you are ready.
[Utilization after starting to walk]
Current navigation systems provide various guidance information in a push-based manner. Because it is transmitted serially, important information can sometimes come after unnecessary information.
People who use white canes do not walk while chatting. Information from their ears is that important, and they are prepared not to miss anything.
That is precisely why it is desirable to obtain information on-demand, using a pull-based method. For example, it should answer when you ask, "How many more meters until Lawson?"
The app that receives the question will retrieve the relevant answer from the information acquired at the time of departure and speak it. This is exactly where generative AI comes in.
[Conclusion]
For walking navigation intended for visually impaired people, natural language dialogue is the best user interface method. Generative AI is useful for that.
Below is the link to the route introduced in the second image.