On-set virtual production

Short description: Technology for television and film production

This article is missing information about Error: you must specify what information is missing.. Please expand the article to include this information. Further details may exist on the talk page. (September 2023)

|

On-set virtual production (OSVP), also known as virtual production (VP), or In-Camera Visual Effects (ICVFX), and often called The Volume, is an entertainment technology for television and film production in which LED panels are used as a backdrop for a set, on which video or computer-generated imagery can be displayed in real-time. The use of OSVP became widespread after its use in the first season of The Mandalorian (2019), which used Unreal Engine, developed by Epic Games.

## Contents

* 1 History
* 2 Terminology
* 3 Technology
* 4 Examples
* 5 Productions using the technologies
* 5.1 Television series
* 5.2 Feature films
* 5.3 Short films
* 6 References

## History

Australian film director Clayton Jacobson first had the idea of improving the green screen that was then in use, when filming a TV ad for detergent in 2003. Watching his son playing videogames and seeing the 3D technology used in them gave him the idea. Eventually, in 2016, Jacobson and his son made one of the first prototypes for a virtual production stage in their shed, using a set of LED screens. However, he could not get anyone to take an interest in developing the technology further, so gave up on it. Other filmmakers had also caught on to the idea though, and in 2018 an Australian cinematographer, Greig Fraser, used the technology to film the Star Wars franchise spin-off series, The Mandalorian (released 2019). Instead of using the green screen during the filming stage, the team combined post-production with the production stage of the series. They installed huge LED walls linked to powerful computers which ran Unreal Engine gaming software (used for Fortnite, among others). They called this soundstage "the volume", a term already used to refer to a stage where visual effects techniques take place.[1]

Since its inventive use in The Mandalorian, which used ILM's StageCraft, the technology has become increasingly popular. Miles Perkins, industry manager of film and TV for Epic Games and maker of the Unreal Engine, estimated that there were around 300 stages by October 2022, increased from only three in 2019.[2] Most of these were built during or after the COVID-19 pandemic, when lockdowns meant that production studios had to find ways to produce films without travelling to other locations.[1]

In March 2023, the world's largest virtual production stage was opened at the Docklands Studios Melbourne, in the city of Melbourne, Australia.[1]

## Terminology

On-set virtual production (OSVP) is also known as virtual production (VP), In-Camera Visual Effects (ICVFX),[3] or The Volume.[citation needed]

## Technology

With careful adjustment and calibration, an OSVP set can be made to closely approximate the appearance of a real set or outdoor location.[4] OSVP can be viewed as an application of extended reality. OSVP contrasts with virtual studio technology, in which a green screen backdrop surrounds the set, and the virtual surroundings are composited into the green screen plate downstream from the camera, in that in OSVP the virtual world surrounding the set is visible to the camera, actors, and crew, and objects on set are illuminated by light from the LED screen, creating realistic interactive lighting effects, and that the virtual background and foreground are captured directly in camera, complete with natural subtle cues like lens distortion, depth of field effects, bokeh and lens flare. This makes it a far more natural experience that more closely approximates location shooting, making the film-making process faster and more intuitive than can be achieved on a virtual set.[citation needed]

To render parallax depth cues correctly from the viewpoint of a moving camera, the system requires the use of match moving of the background imagery based on data from low-latency real-time motion capture technology to track the camera.[citation needed]

Industry organizations including SMPTE, the Academy of Motion Picture Arts and Sciences and the American Society of Cinematographers have started initiatives to support the development of OSVP.[4][5][6][7]

## Examples

Stages that use OSVP include the various StageCraft stages, Pixomondo's Toronto-based LED stage, which has a longterm lease from CBS, ZeroSpace's Volume in New York City,[8] or Lux Machina various stages.[2] In Japan, the LED wall and virtual production were used by Toei Company for its Super Sentai shows Avataro Sentai Donbrothers[9] and Ohsama Sentai King-Ohger, with the latter also being produced in collaboration with Sony PCL Inc.[10]

(As of March 2023) the largest OSVP is at Docklands Studios Melbourne in Australia.[1]

## Productions using the technologies

### Television series

* The Mandalorian (2019–present)[11]
* The Book of Boba Fett (2021)[12]
* Star Trek: Discovery (2021–2024)[13]
* Star Trek: Strange New Worlds (2022–present)
* How I Met Your Father (2022–2023)[14]
* Our Flag Means Death (2022–present)[15]
* Obi-Wan Kenobi (2022)[16]
* The Old Man (2022)[17]
* Andor (2022)[18]
* House of the Dragon (2022–present)[19]
* 1899 (2022)[20]
* Avataro Sentai Donbrothers (2022–2023)[9]
* Ohsama Sentai King-Ohger (2023–2024)[10]
* Silo (2023)[13]
* Ahsoka (2023)[13]
* Percy Jackson and the Olympians (2023)[21]
* Masters of the Air (2024)[22]
* The Acolyte (TBA)[23]

### Feature films

* Oblivion (2013)[24]
* The Jungle Book (2016)[25]
* Rogue One: A Star Wars Story (2016)[17]
* Ghost in the Shell (2017)[26]
* Solo: A Star Wars Story (2018)[17]
* First Man (2018)[27]
* The Lion King (2019)[28]
* The Irishman (2019)[29]
* Star Wars: The Rise of Skywalker (2019)[30]
* The Midnight Sky (2020)[31]
* In vacanza su Marte (2020)[32]
* Red Notice (2021)[33]
* The Batman (2022)[34]
* Thor: Love and Thunder (2022)[35]
* Top Gun: Maverick (2022)[36]
* Bullet Train (2022)[13]
* Black Adam (2022)[37]
* The Fabelmans (2022)[38]
* Divinity (2023)[39]
* Ant-Man and the Wasp: Quantumania (2023)[40]
* Shazam! Fury of the Gods (2023)[13]
* Barbie (2023)[13]
* Daddio (2024)[41]
* Megalopolis (TBA)[42]

### Short films

* The Vandal (2021)[43]

## References

1. ↑ 1.0 1.1 1.2 1.3 Purtill, James (22 March 2023). "Virtual production studios are replacing green screens as world's largest opens in Melbourne". Australian Broadcasting Corporation. https://www.abc.net.au/news/science/2023-03-22/virtual-production-studios-cinema-mandalorian-dune-technology/102105942.
2. ↑ 2.0 2.1 Giardina, Carolyn (2022-10-19). "Too Much Volume? The Tech Behind 'Mandalorian' and 'House of the Dragon' Faces Growing Pains" (in en-US). https://www.hollywoodreporter.com/business/digital/volume-house-of-the-dragon-stage-mandalorian-1235244158/.
3. ↑ "In-Camera Visual Effects (ICVFX)". https://www.nepgroup.com/tech-blog/in-camera-visual-effects-icvfx.
4. ↑ 4.0 4.1 "2021: The On-Set Virtual Production Initiative" (in en). https://www.smpte.org/rapid-industry-solutions/on-set-virtual-production.
5. ↑ "Partnering to Develop Advisory Group of the On-Set Virtual Production (OSVP) Initiative." (in en). 2022-01-18. https://www.avnetwork.com/news/partnering-to-develop-advisory-group-of-the-on-set-virtual-production-osvp-initiative.
6. ↑ "SMPTE Hollywood To Explore Lighting and Color Science For Virtual Production" (in en). https://www.shootonline.com/spw/smpte-hollywood-explore-lighting-and-color-science-virtual-production.
7. ↑ "AMPAS Science & Technology Council Presents Virtual Production Primer - The American Society of Cinematographers". https://theasc.com/news/ampas-virtual-production-primer.
8. ↑ "Cleo Abram We Tested Sci-Fi Movie Tech (MKBHD)". https://www.youtube.com/watch?v=u7pu1cQBqtQ&t=22s.
9. ↑ 9.0 9.1 VFXアナトミー インカメラVFXによる特撮をTVシリーズで実現！ 『暴太郎戦隊ドンブラザーズ』
10. ↑ 10.0 10.1 新番組「王様戦隊キングオージャー」の背景CGがハイクオリティーと話題に グリーンバックを使わない最新撮影技術とは？
11. ↑ Gartenberg, Chaim (February 20, 2020). "How The Mandalorian teamed up with Fortnite creator Epic Games to create its digital sets". https://www.theverge.com/2020/2/20/21145671/mandalorian-sets-stagecraft-epic-games-ilm-fortnite-baby-yoda-digital.
12. ↑ Lane, Carly (June 9, 2021). "Exclusive: Here's When 'The Mandalorian' Season 3 Is Filming; New Details on 'Book of Boba Fett' Connection". https://collider.com/the-mandalorian-season-3-the-book-of-boba-fett-production-filming/.
13. ↑ 13.0 13.1 13.2 13.3 13.4 13.5 Giardina, Carolyn (2022-10-19). "Too Much Volume? The Tech Behind 'Mandalorian' and 'House of the Dragon' Faces Growing Pains" (in en-US). https://www.hollywoodreporter.com/business/digital/volume-house-of-the-dragon-stage-mandalorian-1235244158/.
14. ↑ "ILM STAGECRAFT HONORED WITH ENGINEERING, SCIENCE & TECHNOLOGY EMMY® AWARD". July 29, 2022. https://www.ilm.com/television-academy-announces-recipients-of74th-engineering-science-technologyemmy-awards/.
15. ↑ Peterson, Karen M. (April 29, 2022). "How 'Our Flag Means Death' Designers Mixed History With Broad Theatricality". Variety. https://variety.com/2022/artisans/news/our-flag-means-death-production-design-ra-vincent-1235254416/.
16. ↑ Bui, Hoai-Tran (June 17, 2020). "The Obi-Wan Kenobi Disney+ Series Will Use the Same Technology as 'The Mandalorian'". https://www.slashfilm.com/obi-wan-kenobi-tv-series-technology-the-mandalorian/.
17. ↑ 17.0 17.1 17.2 "StageCraft | Industrial Light & Magic". https://www.ilm.com/stagecraft/.
18. ↑ Dominguez, Noah (November 14, 2022). "Andor Actually Used StageCraft Technology Despite Early Reports". ComicBook.com. https://www.cbr.com/andor-stagecraft-led-screen-despite-early-reports-star-wars/.
19. ↑ "HOUSE OF THE DRAGON MAY LOOK TOTALLY DIFFERENT IN SEASON 2 FOR ONE CONTROVERSIAL REASON". 19 October 2022. https://www.inverse.com/entertainment/house-of-the-dragon-season-2-vfx.
20. ↑ "Inside Dark Bay, the spinning LED volume at the heart of Netflix's upcoming '1899'". 10 February 2022. https://techcrunch.com/2022/02/10/dark-bay-1899-virtual-production/.
21. ↑ Giardina, Carolyn (June 28, 2022). "'Percy Jackson and the Olympians' Takes ILM Virtual Production Route (Exclusive)". https://www.hollywoodreporter.com/tv/tv-news/percy-jackson-and-the-olympians-ilm-virtual-production-1235171943/.
22. ↑ "MoTA = @cary_fukunaga" (in en). https://www.instagram.com/stories/highlights/17917895687035563/.
23. ↑ Fisher, Jacob (June 24, 2021). "'The Acolyte' Aiming To Begin Filming In February In London". https://discussingfilm.net/2021/06/24/the-acolyte-aiming-to-begin-filming-in-february-in-london-exclusive/.
24. ↑ "Oblivion" (in en-US). https://www.luxmc.com/oblivion.
25. ↑ Bardini, Julio (2022-08-14). "Industrial Light & Magic's Digital StageCraft Technology: What We Know About the Volume" (in en). https://collider.com/industrial-light-magic-what-is-the-volume-stagecraft-technology/.
26. ↑ "Lux Machina Consulting on Instagram: "We had a great time working with Director of Photography Jess Hall on Ghost in the Shell to provide LED green screen replacement for some of the driving scenes in the movie. Contact us for help with any film or television projects! We specialize in capturing realistic environments in camera as part of the visual effects workflow! Pictured: Pilou Asbaek and Scarlett Johansson in Ghost in the Shell from Paramount Pictures and Dreamworks Pictures."" (in en). https://www.instagram.com/p/BZMuO0InZ9y/.
27. ↑ Desowitz, Bill (October 15, 2018). "Beyond Christopher Nolan: 'First Man' Redefines In-Camera VFX" (in en-US). IndieWire. https://www.indiewire.com/2018/10/first-man-vfx-christopher-nolan-in-camera-1202012086/.
28. ↑ "How Cutting-Edge ILM Technology Brought 'The Mandalorian' to Life". February 5, 2020. https://nofilmschool.com/ilm-stagecraft-mandalorian.
29. ↑ "The Irishman" (in en-US). https://www.luxmc.com/the-irishman.
30. ↑ "Lux Machina Consulting on Instagram: "Congrats to @starwarsriseofskywalker and @theirishmanfilm for the many Oscar nominations this week, including for Visual Effects! We are so grateful to have been a part of both productions. Looking forward to the big night, Feb 5 live at 8e | 5p! . . . . . . #virtualproduction #incameravfx #vfx #film #technology #videoproduction #led #projection #videoscreens #luxmc #luxmachina #consultants #engineers #programmers"" (in en). https://www.instagram.com/p/B7XdP-LpqOR/.
31. ↑ Mike Seymour (January 21, 2021). "The Midnight Sky's Virtual Production using StageCraft". https://www.fxguide.com/fxfeatured/the-midnight-skys-virtual-production-using-stagecraft/.
32. ↑ Filippo Magnifico (December 23, 2020). "In Vacanza su Marte: il film con Boldi e De Sica realizzato con una tecnologia simile a quella di The Mandalorian". https://blog.screenweek.it/2020/12/in-vacanza-su-marte-film-boldi-de-sica-mandalorian-762136.php//.
33. ↑ "The Next Generation of Extended Reality Sounds Stages" (in en-US). https://www.luxmc.com/red-notice.
34. ↑ Giardina, Carolyn (October 19, 2020). "'The Batman' Using 'Mandalorian' Virtual Production Techniques". https://www.hollywoodreporter.com/behind-screen/the-batman-using-mandalorian-virtual-production-techniques.
35. ↑ Carolyn Giardina (September 10, 2020). "Industrial Light & Magic Expands Virtual Production Services, Supports 'Thor 4'". https://www.hollywoodreporter.com/movies/movie-news/industrial-light-magic-expands-virtual-production-services-supports-thor-4-4057886/.
36. ↑ "Lux Machina Consulting on Instagram: "Congratulations to @topgunmovie "Maverick" on its six @theacademy nominations, including Best Visual Effects and Best Picture! It was an honor for @luxmc_ to be part of such an iconic project by providing in-camera virtual backgrounds and working closely with "Maverick's" overall VFX supervisor Ryan Tudhope and Director of Photography @claudiomirandadop We are thrilled at how much the fans, the critics, and now the Academy of Motion Picture Arts and Sciences have responded to the film. The Oscars will be handed out Sunday, March 12, 2023. Get your popcorn ready! (@luxmc_ is part of @nep_group) #vfx #visualeffects #luxmc #awards #oscars #nominations #academy #topgun #NEP #NEPVS #topgunmaverick #sequel #ICFVX #film #photography #directorofphotography"" (in en). https://www.instagram.com/p/Cn0raM7LbmL/.
37. ↑ Villaverde, Noah (June 8, 2022). "'Black Adam' Director Talks Using Lucasfilm's Volume & Other New Tech". Heroic Hollywood. https://heroichollywood.com/black-adam-director-talks-using-lucasfilms-volume-other-new-tech/.
38. ↑ Kadner, Noah (2023-04-24). "Virtual Production—When The Fabelmans Met The Mandalorian" (in en-US). https://blog.frame.io/2023/04/24/virtual-production-when-the-fabelmans-met-the-mandalorian/.
39. ↑ "Lux Machina Consulting on Instagram: "Wishing Eddie Alcazar the best of luck as he heads to #Sundance to premier his newest film "Divinity." A portion of the film was shot using the #LuxMC LED volume, and Junaid Baig, VP of Research & Development for #NEPVS, used AI to provide slow motion footage of already animated stop motion sequences on the film."" (in en). https://www.instagram.com/tv/CnnAEahrREi/.
40. ↑ Erao, Math (May 17, 2021). "Ant-Man and the Wasp: Quantumania Director Shares Photo From The Mandalorian's Virtual Set". https://www.cbr.com/ant-man-wasp-quantumania-mandalorian-volume-photo/.
41. ↑ "Christy Hall Shot Daddio Entirely Inside a Taxi Cab". https://www.moviemaker.com/christy-hall-daddio-immersive-led-technology/.
42. ↑ Couch, Kim Masters, Scott Feinberg, Aaron; Masters, Kim; Feinberg, Scott; Couch, Aaron (2023-01-09). "Francis Ford Coppola's 'Megalopolis' in Peril Amid Ballooning Budget, Crew Exodus (Exclusive)" (in en-US). https://www.hollywoodreporter.com/movies/movie-features/francis-ford-coppolas-megalopolis-in-peril-1235284875/.
43. ↑ "Lux Machina Consulting on Instagram: "What does it take to make an award winning film? We're glad you asked! Watch the new behind the scenes video and find out how the creative artists were able to seamlessly blend live action video with stop-motion for @eddiealcazar 's #VimeoBestOfTheYear award-winning short film "The Vandal." In "The Making of The Vandal," you'll see how we used our #LUXMC LED stage and interactive lighting sources for the miniature photography. Watch "The Making of The Vandal" via the link in our bio."" (in en). https://www.instagram.com/p/CnkMrTVtcfd/.

* v
* t
* e

Mixed and virtual reality

Concepts|

* Applications of virtual reality
* Artificial reality
* Augmented reality
* Augmented virtuality
* Degrees of freedom
* Immersion
* Projection augmented model
* Real life
* Reality–virtuality continuum
* Room-scale
* Telepresence
* Virtual reality sickness
* Virtual world
* persistent

Technology|

* Eye tracking
* Finger tracking
* Haptic suit
* Omnidirectional treadmill
* Positional tracking
* Wearable computer

| Display|

* Head-mounted display
* optical
* Head-up display
* Virtual retinal display
* Virtual reality headset (comparison)

|

Software|

* Asynchronous reprojection
* Foveated rendering
* Image-based modeling and rendering

Photography|

* Free viewpoint television
* 360-degree video
* VR photography
* Omnidirectional camera

Peripherals|

* Cyberith Virtualizer
* Oculus Touch
* Leap Motion
* PlayStation Move
* Razer Hydra
* Virtuix Omni
* Wired glove

Immersive
devices|

| Current|

* Google Cardboard
* Google Daydream
* HTC Vive
* Nintendo Labo VR Kit
* Oculus Go
* Oculus Quest
* Oculus Rift
* CV1
* S
* OSVR
* Magic Leap
* Samsung Gear VR
* PlayStation VR
* Pimax
* Valve Index
* Windows Mixed Reality
* Microsoft HoloLens
* 2
* Google Glass

|

Legacy|

* Famicom 3D System
* Liquid Image
* Sensorama
* Sega VR
* Sword of Damocles
* Virtual Boy
* Virtuality

Applications|

* ARToolKit
* ARCore
* Interactive art
* Virtual graffiti
* OpenVR
* OpenXR
* Pervasive game
* WebVR

| Games|

* List of HTC Vive games
* List of Oculus Quest games
* List of Oculus Rift games
* List of PlayStation VR games

|

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/On-set virtual production. Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=On-set_virtual_production&oldid=3382768"
*[v]: View this template
*[t]: Discuss this template
*[e]: Edit this template