Recently, a company called Veeso developed a social VR headset that can track players’ facial expressions. The official said it is the world’s first VR device that captures facial expressions and converts them into digital characters in real time. Currently, it’s crowdfunding on Kickstarter. .
The real-time conversion effect is as follows:
At this stage of VR social networking, the direction, position, basic posture, and head tracking of social objects have been completed, but this only satisfies the most basic needs: communicating with each other and seeing each other's body. At present, Altspace VR, High Fidelity, and vTime have developed VR social headlines. The three provide players with a virtual avatar representing themselves in the virtual world. However, the image modeling is too “rough†to reflect the user’s actions. And facial expressions, and the VR social environment built inside is based on the interaction of traditional social applications.
In order to make VR social networking more interesting, Altspace VR began to implant various functions and mini-games, such as web browsing, art galleries, and tabletop casinos.
However, the core of VR social networking is the interaction between people, not games and the environment.
The social interaction of human beings in the real world is not only verbal dialogue but also the expression of the other party’s true thoughts and feelings. According to statistics, only 30% of the information communicated by human beings in society is realized through language, and the remaining 70% is achieved through facial expressions and body movements. Therefore, facial expression capture is particularly important in VR social networking. Create virtual scenes that are fully synchronized with real expressions, and players can improve the social feelings of social objects through expression changes.
 Facial capture technology1. Common facial capture technology
The common facial expression capture technology can present the real facial expressions in an animated manner on the customized virtualized body, making the avatar characters come to life, creating a more immersive virtual reality experience, which is applied in Hollywood movie special effects production. widely. Faceshift, which Apple acquired last year, is a facial expression capture company that captures actors' facial movements and replicates them in animated films and virtual characters in games.
However, there are two problems with this type of identification technology. The first is that the user needs to mark the face with a mark. The mark is the position identification point of the PCB in the circuit board design applied to the automatic placement machine. The high-speed camera captures the human face. Mark point to count back facial actions. The process of placing mark points is cumbersome and complicated. It is only suitable for professional performances and is not suitable for ordinary players.
The second problem is that the above recognition technology does not have real-time performance, and it needs to be synthesized later by MAYA and 3D MAX.
2. Veeso's facial capture features
The feature of Veeso is that users do not need to draw marks on their faces, and they can capture them in real time and synchronize with virtual figures in real time. Veeso's VR head prototype is equipped with two cameras, one hidden in the head display to record the player's eyeball and eyebrow movements, and the other to protrude from below and record the user's mouth and chin movement. The two cameras record the movements of the eyeball, eyebrows, mouth, and chin at the same time and map the facial expressions to the avatar's face.
Since the tracking of facial expressions is synchronized in the virtual picture, it is difficult to implement the technology. The main points are: tracking and reproduction (mapped to the avatar) delay, device data processing capabilities, tracking sensor sensitivity , social network bandwidth, and more. Moreover, facial tracking involves a wide range of disciplines, including CV, electronics, mechanics, psychological perception, machine learning, facial animation, and sensors. The running process of real-time facial capture is roughly as follows. The human expression is first tracked by the sensor, and after data processing, it is transmitted to the VR device of the social object through the high-speed broadband network and is reproduced in its virtual environment.
 Existing solutionsIn fact, several teams have already tried VR facial capture technology. Assistant Professor at the University of Southern California, Li Wei, is working with Oculus on virtual reality head tracking and facial tracking. The initial prototype of the device and the research paper "Facial Performance Sensing Head-Mounted Display" were published in 2015. The year's SIGGRAPH (Graphics and Interactive Technology Special Enthusiast Group) was first made public.
In a collaborative study, they found that because the VR player's head is blocked by the HMD part, this can cause great trouble for the VR to achieve facial table tracking. So, the researchers tried to embed the sensor in the foam board (yellow mark in the figure below) to capture the facial expression of the player by sensing the twitch of the facial muscles . The front camera was used to track the user's chin and mouth. action.
Li Xi said that this prototype proves that it is possible to extrapolate facial expressions using a strain sensor and a machine learning algorithm under the head of virtual reality, but such a device is still a certain distance away from achieving perfect tracking and reproduction of facial expressions.
Veeso is similar to the above-mentioned prototype in achieving mouth and chin tracking, with an outrigger and camera attached to the front of the product. And some people have tried to use the Lip Sync sound calculated by the avatar's lips, the shape of the mouth and let the words match the player said. The way to match the movement of the mouth by sound is useful in the most basic dialogue scenario, but if the player wants to make a grimace without making a sound, it will be stretched.
A foreign technology performs facial expression reproduction according to events occurring in a virtual scene. When a player encounters an unexpected event in a virtual scene, the user's interest focus is estimated based on the event content, and an eye-opening action is finally generated. In general, it is based on the specific scene to guess the player's eye expression . Another new technology is to use a camera to capture the lower half of the face, and then use deep learning to analyze the lower half of the facial expression to restore the full face. Although these two technologies cannot be used as the mainstream capture technology, they can be used to polish mainstream solutions and make it more accurate to capture and reproduce based on speculative algorithms.
The official said, Veeso head display can adapt to all iOS and Android devices, not only support the current market VR content, there are several independently developed games available to players to choose. Currently, from the Demo video released on Kickstarter, the facial expression tracking system in the head and the facial expression demonstration in the VR scene are not accurate enough, and there are still many technical problems to be solved. Therefore, the so-called "world's first VR device with built-in facial tracking technology" does not mean that it has much subversiveness.
The current pre-emptive experience for this project is $70, and developers can purchase equipment and development kits for $80.
21-75Kva Diesel Generator,Portable Type Diesel Generator,Home Use Diesel Generator,Shanghai Power Generator Set
Shanghai Kosta Electric Co., Ltd. , https://www.ksdpower.com