Video Text Content

AT-CES2023 2022/12/26

 

Electronic music plays throughout. 

 

ONSCREEN TEXT:

Virtual Production 

Where express your creativity without limits

 

Voice Over

“Sony believes the technology to instantly display high-definition, photo-realistic computer graphics in real time will give creators more freedom than ever before.”

 

ONSCREEN TEXT:

BROADEN YOUR WORLD

Present and Future of Virtual Production

 

Time lapse video, installing the frames for huge LED wall in the studio.

Camera operator and the team check the system of Virtual Production in front of big screen.

 

Voice Over

“Sony Innovation Studios was one of the first to use a game engine and LED wall for Virtual Production.”

 

ONSCREEN TEXT:

Sony Innovation Studios

 

A scene of demonstration of Virtual Production at Sony booth at CES2020.

The camera captures a real automobile in front of a large LED wall, which displays alleyway.

The camera cuts out images in real time, as if the cars are really parked in the city.

 

Voice Over

“At CES 2020, a prototype was unveiled.

The LED display showed a photo-realistic 3D computer graphics set, and the way the background synched with the camera movement suggested to many creators the possibility of new shooting techniques.”

 

ONSCREEN TEXT:

VIRTUAL PRODUCTION

In-Camera VFX ＋ LED WALL

 

Motion graphics starts. it shows how In-camera VFX technology works in Virtual Production.

The CG background displayed on the LED wall and changes with the camera's movements.

 

Voice Over

“In-camera visual effects use a game engine to display 3D computer graphics on a large LED wall.

This technique enables users to create a seamless integration between foreground actors and virtual backgrounds.

The background image changes in real time according to the camera's positional information, making it possible to shoot a 3D CG space as if shooting on an existing location.”

 

In a scene from behind the scene of “THE DESERT LIGHT (2021)” .  it shows Virtual Production filmmaking scene.

A lady exploring in a dark cave. The background scenes change from the cave to desert.

The operating PC screen shows that the background is made of computer graphics.

 

ONSCREEN TEXT:

THE DESERT LIGHT (2021)

 

Logo of Sony Innovation Studios is placed left under the corner.

 

Voice Over

“Sony proposes the combination of its "Crystal LED" technology which realistically projects space with rich color expression, and its "VENICE" digital cinema camera, which can capture all light present, as the ultimate Virtual Production shooting environment.

Ancient ruins, deserts, any location in the world and even fictional worlds can be filmed freely in the Virtual Production studio as if filming on site. Creating virtual assets in high-definition and projecting in real time greatly expands the possibilities for a studio shoot.”

 

In a scene from “BTS of Sony Innovation Studios”.

It shows people work in their new Virtual Production studio. The team adjusts 3DCG background using PC.

 

Voice Over

“In October 2022, Sony Innovation Studios installed the world's largest stage for Virtual Production using its “Crystal LED” technology.

We will expand Virtual Production workflows in various forms of entertainment and create works that seamlessly integrate the real world and virtual spaces.”

 

Time lapse video.

The scene of filming in a huge LED wall studio. LED wall displays mountain scene.

And then LED wall displays the logo of Sony Innovation Studios.

The screen fades to black.

 

ONSCREEN TEXT:

EXPANDING DIVERSITY IN VIRTUAL PRODUCTION

A case in Music Video / King Gnu "Stardom"

 

In a scene from the music video of King Gnu “Stardom”.

The vocalist sings in the midst of many gnus herds on a red plain.

 

ONSCREEN TEXT:

King Gnu-Stardom

＠2022 Sony Music Entertainment (Japan) Inc.

 

The scene changes to a behind of the scene.

Vocalist sings in front of huge wall and tells it was shot in the Virtual Production studio.

 

ONSCREEN TEXT:

Sony PCL Inc. KIYOSUMI-SHIRAKAWA BASE”

 

Shu Sasaki is interviewed in front of LED wall.

It displays one of the CG background from “Stardom”.

 

ONSCREEN TEXT:

SHU SASAKI

Creative Director of “Stardom”

PERIMETRON

 

Shu Sasaki

“Talking about the Music Video, this song, "Stardom", has a huge range of sound image.

Initially, I thought that the locations I used in Japan did not seem realistic. We wanted to create something seamless for the audience by combining computer graphics with live action shots.

In a scene from “Stardom”, the car is driving very fast through the city at night then change to the footage of behind the scene.

King Gnu members play instruments in front of a display, showing a geometric background graphic.

 

OSRIN is interviewed in front of LED wall.

It displays one of the CG background from “Stardom”.

 

ONSCREEN TEXT:

OSRIN

Director of “Stardom”

PERIMETRON

 

OSRIN

“I think it all started with us talking about whether we could create a visual that can convey the speed.

Which lead us to use a car as the main motif.

However, this will make filming really complicated because we have to consider the actor's driving skills and also have to avoid accidents.

Considering those factors, we decided to construct a computer graphic world that matches

both the car we're going to use as well as our band image.”

 

Screen shows the video of behind the scene of “Stardom”

The red car is illuminated to create the impression that it is driving fast.

Back to Interview video.

 

Shu Sasaki

“What I expected from CG was to be able to create a large field that is unthinkable in reality,

and to put band members in a space, which is impossible in reality.”

 

OSRIN

“It showed us something between realistic and un-realistic.

We thought we were going to create a very realistic tunnel for the scene,

but something more was added to it and that was really exciting.”

 

In a scene from behind the scene footage.

It shows tunnel scene.

The Virtual Production team adjusted the computer graphics background using Unreal Engine, a real-time 3D creation tool on PC.

Shooting scene, the car is placed in front of the tunnel image on the LED wall.

 

OSRIN

“Mach was the main concept, and we wanted to create something speedy and everything fell in the right place.

What I had the most fun when we were shooting was, when everything came together,

for example the acting, the art design and even each actor's personality, when all those came together.

When what we shoot turns out few times or even ten times better than what I had in my storyboard,

I almost cried.

 

Shu Sasaki

“I had so much fun during the production. Especially during Virtual Production shooting,

me and my band members were able to see what get captured on camera and talk things through while shooting.”

So that made it feel like a regular live action shoot and was very fun.”

 

OSRIN

“I want to make sure the camera gets what's happening in front of me, but I also want to make sure I see it with my own eyes.

Maybe I'm being greedy, but I want to take time to register the scenery in front of me as a special view of that day even when we're in production.”

 

The screen fades to black.

 

ONSCREEN TEXT:

POTENTIAL OF REAL-TIME RENDERING

”BACKDROP LIBRARY“ gives creators freedom

 

A couple of footages of a large number of colored balls flying through the streets of Shibuya.

Mr. Ushiro, who wears yellow cardigan is interviewed in front of the virtual asset of Shibuya Spain-Zaka.

 

ONSCREEN TEXT:

HIDENORI USHIRO

Creative Director of “BACKDROP LIBRARY”

Sony PCL Inc.

 

Hidenori Ushiro

“When we as SONY create the background assets, the most important thing to pay attention to is definitely the absolute reality.

 If we have a city as the background asset, in this case it's Shibuya,

the ultimate purpose for that asset is to make it possible to shoot in that environment without physically going to that location.”

 

The scene of Virtual Production stage.

The team filmed in front of the large LED wall which displays virtual asset of Shibuya.

A girl on a bicycle was in front of the LED wall and it looked as if she is at Shibuya.

 

The screen changes to the scene of Shibuya Spain-Zaka.

The CG production team is using a point cloud scanner to measure the city.

They also take tons of picture of Shibuya with still camera for making virtual asset.

 

Mr. Higa, who wears gray shirt  is interviewed at his office.

 

ONSCREEN TEXT:

SATORU HIGA

VFX Supervisor / Technical Director

BACKSPACE Productions Inc.

 

Satoru Higa

“We created Shibuya Spain-zaka based on 3D scan data by FARO and tons of pictures of a location.

 We combined those huge photo data with the 3D scan data to create a mesh model with realistic texture by using Photogrammetry technology.”

 

Screen shows 3D model of Shibuya Spain-Zaka.

 

Satoru Higa

“What's important here is how it looks when we film what is projected on the LED wall,

and when I saw the result, it really looked like

Shibuya Spain-zaka and I was stunned.”

 

In a scene from the behind the scene of “Make Colors, Make Shibuya”.

Mr. Higa adjusts the Shibuya Spain-Zaka 3D model at Virtual Production stage and LED wall displays it.

 

Back to interview scene.

Mr. Tsukamoto, wearing light blue shirt is interviewed.

 

ONSCREEN TEXT:

TOKIHIKO TSUKAMOTO

CG Producer

jitto inc.

 

Tokihiko Tsukamoto

“Real-time rendering takes absolutely shorter time.

It takes less time to render even a photo-real space and is able to output in real time.“

 

Satoru Higa

“I'll get feedback for what I just did immediately, and that allows me for more try and error opportunity.

I believe that gives a creator much more freedom.”

 

Tokihiko Tsukamoto

“When we're working on the data in Virtual Production studio, we could be like "hey let's fix this part a bit" and

we were able to reflect that change immediately.

It was not possible with pre-rendering.”

 

In a scene from behind the scene from “Make Colors, Make SHIBUYA”.

A girl acts in front of the LED wall that displays 3D computer graphics background of Shibuya.

Mr. Higa adjusts the 3D model using his PC and Mr. Ushiro checks the girl’s positioning  with a small monitor.

 

Hidenori Ushiro

“The huge advantage of real-time rendering for creators is that they do not have to wait for hours or days to see the composited image in real time on the set.”

 

Back to Interview scene.

Mr. Yoong, wearing a black sweatshirts is interviewed in front of LED wall.

 

ONSCREEN TEXT:

TAKESHI YOONG

CG Director

jitto inc.

 

Takeshi Yoong

“Whether we wanted to fix camera or lighting, both elements would affect each other, rather than simply changing or adding one thing.

For example, if we want to have lighting in certain way, we'd also have to change things in Virtual Production asset data.

When dealing those issues, the lighting and CG specialist were able to talk things through and make decisions on the go.

I think that kind of live interactions can only happen in Virtual Production.

When I was able to witness them creating the backdrop within 3 or 4 days, I was shaking from excitement.

It is so interesting. And there is so much freedom.”

 

At Virtual Production studio.

Mr. Yoong and Mr. Ushiro discuss about 3D Shibuya model then having conversation with the team.

They start filming the scene, which the girl flies though Spain-Zaka.

The girl is suspended wires in front of LED wall.

 

Satoru Higa

“You could try an idea that happen to come to your mind or just try something that you usually wouldn’t even bother doing.

And that expands the possibility of creation or expression in so many degrees.”

 

In a scene from “Make Colors, Make SHIBUYA”.

A girl rides her bicycle through the narrow alleys of Shibuya, then the bicycle starts flying through the air.

She looks like she’s having fun.

 

ONSCREEN TEXT:

Make Colors, Make SHIBUYA.

©2022 Sony PCL Inc.