Thinking about new interactions from the history of operations

I still remember the excitement I felt when I was in elementary school and experienced the novel controls of swinging and pointing on the Wii.

I believe that such novel interactions are born by stepping outside the framework of video games and incorporating operations from various other things.

So, this time, I thought that if I tried out interactions that have been born in various fields but haven't been tried on smartphones, something interesting might come of it,

and I roughly brainstormed some new interaction ideas. (Some have been prototyped.)

What kind of interactions have been born in each field?

First, I roughly researched what kind of interactions have been created so far for PCs, smartphones, vehicles, and games.

Based on my research, it looked roughly like this. (See the figure below.)

I realized that if you include auxiliary controllers, you really can do anything in games.

On the other hand, it seems there is still room for exploration on smartphones.

Trying out interactions that haven't been tried

From here on, I roughly brainstormed various ideas about what could be done using the interactions identified above that haven't been tried on smartphones.

Some of these have been prototyped, so please take a look.

Tapping the smartphone

I thought it would be interesting if something happened when you used two smartphones and bumped them together, so I prototyped "Tap to Airdrop".

Other idea notes:

• Exchanging contact information by tapping smartphones together in a toast gesture at a drinking party

• Clapping by tapping your smartphone at a live concert, where something happens depending on the number of claps

• The sound of tapping the smartphone changes to drum sounds or similar, allowing you to play music anytime, anywhere

• etc...

Tilting the smartphone

I created "Tilt Pay", where if you tilt the phone downward as if showing it to a clerk during a payment, the card enlarges to make it easier to scan.

Other consideration notes ↓

• Operations using subtle tilts of a device like a gyro sensor

  • Seems like it could be used for training to improve sense of balance

• etc...

Pulling/pushing the smartphone

I thought this could be an action related to something spatial, so I felt it could be used for something in AR.

• When opening a secret door in an AR city-walking RPG

  • Pull the smartphone while touching the screen

• etc...

Stepping on the smartphone

I wondered what kind of situation would involve stepping on a smartphone, but I thought about it anyway.
Consideration notes ↓

• If you step on it, the smartphone gets angry and says it hurts

• Can measure body weight

• etc...

Gripping the smartphone

I thought about this while realizing that the action of taking a screenshot is basically 'squeezing'.
Consideration notes below

• Input based on the strength of squeezing a smartphone

  • Measuring grip strength

  • Changing the degree of favorites/likes based on squeezing strength

  • etc...

Rotating a smartphone

When I looked it up, I found something interesting like this.
↓ A music video you watch while rotating it on a record player

Other consideration notes below

• Input based on rotation angle and direction

  • Input like clockwise for Yes and counter-clockwise for No?

Bonus: 'Pushing/Pulling' a PC

I thought of one for PC, so I'll post it here.

• Interaction based on opening/closing a PC

  • Closing and opening it switches movies randomly

  • It has the fun of opening a treasure chest

Conclusion

After thinking about it, I felt that the method of 'deciding on an interaction first and then thinking about what can be done with it' is a way to come up with surprisingly interesting ideas even when doing it yourself.

Also, this time I used an approach of 'taking existing interactions from elsewhere' limited to UI, but I thought it would be even more interesting if I could bring them from various things in the world, not just UI.

I realized that new things are born by stepping outside and learning by looking at things in various fields.

I did this one quite roughly, so I think there are many other untried interactions. I hope you all try thinking about it 'roughly' as well.