It looks like Sony is heavily investing in PSVR (PlayStation VR) and VR as part of the company’s future. From real-world positioning, facial tracking, finger tracking, to even built-in headphones, Sony has taken a large interest in VR, as a large number of recently filed Sony VR patents have surfaced.

Do note that these are patents, and by no means suggests that the features found in them will make it into any commercial product. So take the information as you will, though one can’t deny that Sony is definitely not through with VR. Some of these have been filed just this year, while others from a bit back. They have all gone public this month, so all information found in this article is accessible to everyone.

The first patent we’ll look at is the most recent one and it pertains to Sony looking into creating a system that is able to detect and display boundaries associated with player movements. 

“Systems and methods for indicating that a boundary is crossed during execution of a video game are described. One of the methods includes generating, by a camera, a field-of-view, the field-of-view encompassing a plurality of boundaries that define a volume. The method further includes capturing, by the camera, one or more images of a head-mounted display (HMD) within the field-of-view. The method includes identifying, by a processor, a position of the HMD in a real-world environment from the one or more images. The method includes determining, by the processor, whether the HMD at the position has crossed one of the boundaries. The method includes generating one or more video frames including a guard rail upon determining that the HMD has crossed the one of the boundaries and sending the one or more video frames to the HMD for display.”

What exactly does this mean? Well, it means that if you have a large enough space to play in, the camera will be able to track your movements within the capture space. This meaning you could walk around within a VR environment as long as it’s within range of the camera. Here are some screenshots to accompany this for a better visual representation.

That’s some pretty exciting stuff right there, as the current method of movement requires a controller on PlayStation. You can lean and do a few other things to adjust your view, but physically walking in PlayStation VR yields no results as of now.

The next patent also is related to movement tracking, however, it expands and includes the ability to properly display the same VR scene across multiple users. In the examples provided, we see two VR users standing in different positions, along with their viewpoints showing each player in their corresponding position in the VR application. You may also recognize the characters in the following images as they are Stormtroopers from the Star Wars series.

Systems and methods for processing operations for head mounted display (HMD) users to join virtual reality (VR) scenes are provided. A computer-implemented method includes providing a first perspective of a VR scene to a first HMD of a first user and receiving an indication that a second user is requesting to join the VR scene provided to the first HMD. The method further includes obtaining real-world position and orientation data of the second HMD relative to the first HMD and then providing, based on said data, a second perspective of the VR scene. The method also provides that the first and second perspectives are each controlled by respective position and orientation changes while viewing the VR scene.

Virtual reality (VR) presented through head mounted displays (HMDs) are becoming a more and more popular way for consumers to interact with content. However, while much of the computer entertainment industry has seen a shift toward offering more social and inter-personal platforms of interacting with content, VR has been a notable exception. Indeed, VR users often find that virtual environments can be isolating and lacking in a social aspect. As VR continues to gain popularity, there is an opportunity to incorporate social and inter-personal components to enhance and humanize the VR consumer experience.

This is especially true when HMD users are co-located. For example, two co-located HMD users each within their own VR may wish to share their VR with the other and otherwise interact with one another within the VR environment. That is, there is an opportunity to not only enable co-located users to witness the same VR, but also to allow for the interaction between the avatars of the co-located users within a VR environment. Such inter-personal and “inter-avatar” interaction enhances the VR experience for users and allows content creators to incorporate social and multi-user dimensions into the VR content universe.

There was another tracking-based patent filed by the company back in 2018 that also went public this month, though unlike the other it doesn’t specify VR applications. It does however describe the ability of tracking movement and users starting points.

An electronic device for object tracking based on a user-specified initialization point is provided. The electronic device stores a sequence of image frames, which includes a first image frame and a second image frame. The electronic device estimates a set of feature correspondences between a first set of features points in the first image frame and a second set of feature points in the second image frame. The electronic device generates different first motion-estimate models for different groups of feature correspondences of the set of feature correspondences and further estimates, from different groups of feature correspondences, a plurality of inlier feature correspondences that correspond to the object of interest in the first image frame and the second image frame. The electronic device generates a second motion-estimate model as an optimal motion-estimate model and tracks the object of interest in the sequence of frames, based on the second motion-estimate model.

The next two are related to eye tracking. The first one doesn’t indicate any gaming application, however, the second one that was filed much later builds off that one and applies itself to VR. There’s a third one that was filed back in 2016 that also went public this month that pertains to eye tracking and performing actual movements. 

(first patent) A method for tracking eye movement is provided. One embodiment of the method includes receiving a first measurement from a first sensor configured to detect a gaze location, determining an initial gaze location based at least on the first measurement, receiving at least one of eye motion amplitude and eye motion direction measurement from a second sensor, and determining an estimated gaze location based at least on the initial gaze location and the at least one of eye motion amplitude and eye motion direction. Systems perform similar steps, and non-transitory computer readable storage mediums each store one or more computer programs.

(second patent) Methods, systems, and computer programs are presented for rendering images on a head mounted display (HMD). One method includes operations for tracking, with one or more first cameras inside the HMD, the gaze of a user and for tracking motion of the HMD. The motion of the HMD is tracked by analyzing images of the HMD taken with a second camera that is not in the HMD. Further, the method includes an operation for predicting the motion of the gaze of the user based on the gaze and the motion of the HMD. Rendering policies for a plurality of regions, defined on a view rendered by the HMD, are determined based on the predicted motion of the gaze. The images are rendered on the view based on the rendering policies.

(third patent) There is provided an information processing device, an information processing method, and a program that can facilitate a user to perceive a stereoscopic vision object, the information processing device including: a display control unit configured to perform movement control of a stereoscopic vision object perceived by a user from a start depth that is different from a target depth to the target depth on a basis of mode specifying information that specifies a mode of the movement control that supports stereoscopic vision by the user, and an information processing method including: performing movement control of a stereoscopic vision object perceived by a user from a start depth that is different from a target depth to the target depth on a basis of mode specifying information that specifies a mode of the movement control that supports stereoscopic vision by the user.

Finally, the last one showcases a slight change to the PSVR headset, as it now sports a pair of earphones, built right into the headset itself. What’s more, while the PSVR features the option to directly plug in a headset, Sony is looking for ways to improve the ease of use as the patent details the ability to easily locate the earphone without possibly removing the VR set.

To propose a head-mounted display capable of enhancing convenience with respect to earphones. A head-mounted display (1) includes: a device main body (10) with a display device (11) incorporated therein; a mounting band (20) which extends toward the rear side from the device main body (10) and is used for mounting to a user’s head; a right earphone holding section (24R) which is provided at a right portion of the mounting band (20) and to and from which a right earphone (31R) can be attached and detached; and a left earphone holding section (24L) which is provided at a left portion of the mounting band (20) and to and from which a left earphone (31L) can be attached and detached. According to the HMD 1, the user can recognize the positions of the earphones (31R) and (31L) without visual confirmation.

There are a few other patents by the company that expand a bit more on the tracking that is seen above, though we didn’t include them as they were just reciting the same information. That, and we’ve already reported on a handful of them from this month. Anyway, what do you all think about these patents? Is the company going in the right direction with tracking? Should they maybe opt for some sort of stationary conveyer belt system? Or perhaps they should continue with their current offerings, that being the 3Drudder that released last year? 

Again, stressing what we’ve mentioned previously, Sony filing these patents do not guarantee that they will be in the next iteration of the PSVR. Rather, it’s more conclusive proof that the company is investing significantly in VR, and these Sony VR patents prove it.