Internet age , Computers are the core of computing platforms , As the first generation of Internet interaction, mouse and keyboard , Once completely ignited the computer age . In the era of mobile Internet , Smart phones are beginning to stand at the core of the computing platform , Multi touch operation has become the symbol of the second generation of revolutionary interactive mode .
The interaction mode starts from one-dimensional keys 、 Click on the interaction , Evolved into two-dimensional multi touch interaction . We see , Towards the interconnected digital world of all things , Three dimensional multimode interaction means will be used in the new computing platform XR（AR/VR/MR Is called ） Play an extremely important role in , Create the ultimate surreal user experience for the smart world . if ,Windows The graphical interface of the system takes people from Dos Liberated from the system , It is a new technological leap to let people operate computers in a more intuitive and human way , Multimode interactive surreal experience is another more far-reaching Leap . And computer technology , Finally, the digital world and the real world are fully integrated , Human beings are upgraded to digital people in a real sense .
What is the premise of realizing surreal experience ？
Multi mode interactive surreal experience , We need a seamless integration of the virtual world and the real world , And can accurately perceive 、 Restore the world , Understand the user's intention in the mixed real world of virtual reality . Besides , Multimode interaction needs to open up vision 、 auditory 、 Tactile sensation 、 Smell and other sensory experiences , Realize the interaction between people and thousands of edge devices . To achieve this goal , We need to treat the overall environment of users as a super digital world , Relying on language 、 body 、 Tactile sensation 、 Light sense 、 Brain machine 、 expression 、 Consciousness, etc , Collect and transmit information , And identify the user's intention 、 Then through the naked eye 3D、 Holographic projection 、AR glasses 、 Technologies such as digital smell and digital touch are presented to users , So as to realize the surreal interactive experience .
In a surreal experience , Traditional mouse 、 Keyboard and multi touch interaction have been unable to meet XR Interaction requirements of equipment . To put it simply , Without physical screens and physical control devices , Humans cannot live in a virtual three-dimensional environment , Through a simple keyboard 、 Mouse or multi touch to complete complex command output . Take gesture manipulation as an example , The terminal device needs to capture the movement of fingers in three-dimensional space , Complete the judgment of time and space at the same time , And send instructions synchronously to complete the user's intention . The terminal equipment must also clearly judge the voice instructions sent by the user , Even judge the user's next intention according to the user's behavior habits . This way of interaction , It's what we often call instinctive interaction -- Maximize the user's mental model , Use consensus or easy to learn 、 Easy to use operation , Let users think less 、 Less doubt 、 Less repetition .
The real challenge is coming
Although the surreal experience can be expected in the future , But the existing interactive equipment and interactive environment , High operating accuracy cannot be achieved yet , Products for general consumers , Technological and commercial breakthroughs are still needed .
- Mobile oriented
For the control of moving position , The equipment needs to complete the original liberation of users in three-dimensional space . In a virtual space that is almost infinite , Users can move freely in the virtual three-dimensional space . This unrestricted action , Although the user's senses have been separated from the real world , But I still can't get away from the physical world . Absolute freedom of movement , It's a denial of environmental rules . Therefore, the physical world and the digital world still need to be further integrated , In order to let users get rid of the shackles of the real world and really enter the virtual world . The premise of this integration is ： Support and follow-up are realized through an external frame . - Facing hand operation
The user's hand operation generally accounts for more than 80% of the overall daily action , Without accurate hand motion capture 、 Feature tracking 、 Action recognition 、 Space positioning system , The entire hand based operating environment is difficult to establish . The functionality of hand operation is reflected in the service execution of the terminal ; The expressiveness of hand operation is reflected in the output of text content in the window ; The sociality of hand manipulation is reflected in body language , Composed of gestures and body movements .
therefore , In addition to meeting the above basic conditions, hand interactive devices should also comply with lightweight 、 Scalable 、 Strong openness and other characteristics . - Data input oriented
In a way ,XR It is the beginning of mankind's real digitization of itself . At this starting point , All behaviors of users can be collected by data . meanwhile , The scope of data monitoring has also been effectively expanded , Predictably, , The future must be the triumph of behavioral psychology and statistics . Remove data formed by user's limb behavior , The user's stay at any location 、 Visual choice 、 Hand operation , Can become the collection category of users' personal data . And the top priority , Can form a set of action based 、 Data input interactive system of language , Continue to effectively promote at this stage XR Development and evolution are of great significance . With an efficient data input system , Users can improve expression efficiency , Efficient input means that users can follow more reliable input interaction logic .
At the end
MR( Mixed reality )、VR( Virtual reality )、AR（ Augmented reality ） Are the three important organizational parts of the future digital world . Relying on new computing platforms and more advanced 5G/6G technology , The boundary between the physical world and the virtual digital world is blurring , and XR The development of technology brings great development opportunities and unprecedented challenges to practitioners .
Zhanrui, as the ecological bearer of the digital world , The builder of the ground floor of the digital world , Shoulder the important task of building the base of the future digital world . In the field of multimode interaction , Zhanrui's smartphone platform passes Sensor Hub Sensor system , Support gesture operation 、 Somatosensory movement 、 Various interaction modes such as facial movements . in addition , Zhan Rui is in speech recognition 、 Semantic understanding 、 There is also a deep layout in natural language processing such as multilingual system . The terminal categories based on zhanrui core are mobile phones 、 Tablet watches 、 Translation pen, etc , Continue to expand to a richer field of the Internet of things . stay 5G、 Short distance connection 、 Based on innovative achievements such as high-performance computing platform , Zhan Rui is constantly breaking through the border , Explore a wider space ……
