Visual media has come a long way since the first proto-human cave dwellers used the flickering of torch light to bring the hand-drawn art on their walls to life. Today, the pixel — despite its humble, low-resolution origins — sits as the current pinnacle of digital display technology. In his new book, Biography of the Pixel, Pixar co-founder Alvy Ray Smith examines the fascinating history and development of picture elements (hence "pix"-"el") from their often-contested start in the labs of pioneering computer researchers like Alan Turing to their ubiquitous presence in modern life. In the excerpt below, Smith takes a look at the bad old days before digital displays to explain the science behind our brain's' ability to perceive motion through the rapid flashing of static images. 

MIT Press

Excerpted from ‘A Biography of the Pixel’ by Alvy Ray Smith (MIT Press, 2021)

How Movies Were Really Done

What did the inventors of cinema do (or not) to make the system they gave us so non-ideal? First, they didn’t give us instantaneous samples as required by sampling. Film frames are fat. They have duration. The camera shutter is open for a short exposure time. A moving object moves during that short interval, and so smears slightly across the frame during the film exposure time. It’s like what happens when you try to take a long-exposure still photo of your child throwing a ball and his arm is just a blur. This turns out to be a saving grace of cinema as it was actually practiced.

Second, they made it so each frame is projected twice (at least) by the projector. Ouch! That’s not sampling at all. Why did the inventors do that? Simple economics demanded it: 24 frames per second costs half as much film as 48 frames per second. But the eye needs to be refreshed about 50 times per second, or the retinal image fades between frames. Actually, 48 is close enough to 50 to work in a dark theater. How do you get 48 from 24? You show each frame twice! If you show just 24 frames per second, the screen appears to flicker. Hence the “flicks” from the early days of cinema before higher frame rates were adopted.

The third thing the original inventors did was to shut off the light between projected frames. This meant that 48 times per second, nothing (blackness) was projected into the eye — inside the pupil, onto the retina. It’s convenient for movie machines — both the camera and projector part — to “shutter” to blackness like this between frames. It allows time for the mechanical advancement of the next film frame into position. In a camera, it keeps the film from recording the real world during the physical advancement of the film. In a projector, it keeps the moving film out of eyesight as it’s physically advanced.

When you ask how a movie projector works, some people say something like this: There’s a top reel of film which is the source of film, and a bottom take-up reel. The film moves from reel to reel and passes between the light source of the projector and its lens, which magnifies the frame-size image up to screen size. In other words, the film moves continuously past the light source. But that doesn’t work. The eye sees exactly what’s there, and with this scheme the eye would see one frame sliding away as the next frame slides in from the opposite side. It would see the sliding. And that won’t work.

What a projector actually does is exactly this: It brings each frame into fixed position with the light source blocked. That’s the function of the shutter. Then the shutter opens and the illuminated frame projects onto the screen. Then the shutter closes. Then it opens again, and the illuminated frame is projected a second time onto the screen. Then the shutter closes and the next frame slides into position, and so forth.

We’ve just described the discrete, or intermittent, movement of film through a projector, as opposed to unworkable continuous movement. The same idea holds for a camera. The physical device that implements this action is called, in fact, an intermittent movement. This is the key notion in cinema history that is comparable to the conditional branch instruction in computer history. The mad rush to the movie machine turned on who first got a projector to work correctly, and that hinged on who got an intermittent movement working properly. It’s a defining notion.

To recap: An actual film-based movie projector doesn’t reconstruct a continuous visual flow from the frame samples and present this to the eye. Instead, it sends “fat samples” — thick with time duration and smeared motion — directly to the eye’s retina. It sends each frame twice, and it sends blackness between. It’s up to the brain to reconstruct motion from these inputs. How does that work?

Somehow the eye-brain system “reconstructs the visual flow” that’s represented by the fat visual samples it receives. Of course, it really does no such thing. Light intensities come in through the pupil as input. But the output from the eye to the brain, through the optic nerve, is an electrochemical pulse train. Neuronal pulse trains aren’t visual flows. It could be that the retina actually does reconstruct a visual flow and then converts it to pulse trains for brain consumption. The responses of some of the neurons in the eye certainly suggest the spreader function, complete with a high positive hump and negative lobes. But brain activity is beyond the scope of this book. Let’s concentrate instead on the customary explanations of the perception of motion from sequences of still snapshots.

Perception of Motion

The classic explanation is hoary old persistence of vision. It’s a real characteristic of human vision: once an image stimulus to the retina ceases, we continue to perceive the image there for a short while. But persistence of vision explains only why you don’t see the blackness between frames in the case of film-based movies. If an actor or an animated character moves to a new position between frames then — by persistence of vision — you should see him in both positions: two Humphrey Bogarts, two Buzz Lightyears. In fact, your retinas do see both, one fading out as the other comes in—each frame is projected long enough to ensure this. That’s persistence of vision. But it doesn’t explain why you perceive one moving object, not two objects at different positions. What your brain does with the information from the retinas determines whether you perceive two Bogarts in two different positions or one Bogart moving between them.

Psychophysicists have performed experiments to determine the characteristics of another real brain phenomenon, called apparent motion. The experiments don’t explain how the brain perceives motion, but they do describe the limitations of the phenomenon. A small white dot on a black background is presented to a subject’s retina. Then that dot is removed, and another dot is presented in a different position. The experimenters can vary two things, the spatial separation of the two dots and the time delay between position change. The brain perceives one dot here and another dot there, but only if the distance and delays are long enough. If the distance and delays are short, the brain perceives that the dot moves from one position to the other. It’s apparent motion because no actual motion is presented to the eye. The brain perceives what it doesn’t see.

Motion Blur

Persistence of vision is such that we still perceive the first image when the second one arrives. That sounds a lot like frame spreading. A frame of short duration spreads out in time and adds to the next frame also spread out in time. It’s as if the retina does the image spreading and the adding of successive spread frames. Something like this must be going on because we perceive a continuous visual field although the film projector doesn’t present one. You can think of the shape of the persistence function of the eye as the shape of the frame spreader that’s built into us human perceivers. Another reason we can assume that the eye-brain system must be doing a reconstruction, one that implicitly uses the Sampling Theorem, is because we perceive exactly the errors we would expect if that were the actual mechanism—such as wagon wheels spinning backward.

Classic cel animation — of the old ink-on-celluloid variety — relies on the apparent-motion phenomenon. The old animators knew intuitively how to keep the successive frames of a movement inside its “not too far, not too slow” boundaries. If they needed to exceed those limits, they had tricks to help us perceive the motion. They drew actual speed lines, which showed the brain the direction of motion and implied that it was fast, like a blur. Or they provided a POOF of dust to mark the rapid descent of Wile E. Coyote as he stepped unexpectedly off a mesa in hot pursuit of that truly wily Road Runner. They provided a visual language that the brain could interpret.

Exceed the apparent motion limits — without those animators’ tricks — and the results are ugly. You may have seen old school stop-motion animations — such as Ray Harryhausen’s classic sword-fighting skeletons in Jason and the Argonauts (1963) — that are plagued by an unpleasant jerking motion of the characters. You’re seeing double, at least — several edges of a skeleton at the same time — and correctly interpret it as motion, but painfully so. The edges stutter, or “judder,” or “strobe” across the screen. Those words reflect the pain inflicted by staccato motion.

Live-action movies are sequences of discrete frames just like animations. Why don’t these movies stutter? (Imagine directing Uma Thurman to stay within “not too far, not too slow” limits.) There’s a general explanation that works. It’s called motion blur, and it’s simple and pretty. A frame that’s recorded by a real movie camera is fat with duration. It’s not a sample at a single instant like a Road Runner or a Harryhausen frame. Motion blur is what you see in a still photograph when the subject moves and the shutter isn’t fast enough to stop the motion. In still photographs, it’s often an unintended result, but it turns out to be a feature in movies. Without the blur all movies would look as jerky as Harryhausen’s skeletons—unless Uma miraculously stayed within limits. The motion blur of moving objects in a fat frame gives clues to the brain about what is moving and what is not. The direction of a blur gives the direction of motion, and its length indicates the speed. Somehow, mysteriously, the brain converts that spatial information — the blurs — into temporal information and then perceives motion with the help of the apparent motion phenomenon.

Two decades after the release of Bigger, Longer and Uncut, the first and only South Park movie to date, series creators Trey Parker and Matt Stone will produce 14 direct-to-streaming films for Paramount+ as part of a new deal the two signed with ViacomCBS to keep the show on Comedy Central through 2027. According to Bloomberg, the agreement is worth more than $900 million over six years, easily eclipsing the approximately $500 million AT&T subsidiary WarnerMedia spent in 2019 to secure exclusive streaming rights to the show for HBO Max.

Parker and Stone haven't made a movie together since 2004's Team America: World Police. The first two films included in their ViacomCBS deal will debut later this year. Stone told Bloomberg he and Parker plan to expand the world of South Park by using the movies as an opportunity to introduce new characters and concepts.

What the deal doesn't include is streaming rights to the South Park TV series. As mentioned above, Viacom licensed those to WarnerMedia in 2019, and that agreement is still in place. However, the $900 million investment in the brand does suggest the company will attempt to bring the series to Paramount+ eventually. 

ViacomCBS executive Chris McCarthy hinted as much in the press release announcing today's news. "Matt and Trey are world-class creatives who brilliantly use their outrageous humor to skewer the absurdities of our culture and we are excited to expand and deepen our long relationship with them to help fuel Paramount+ and Comedy Central," he said. "Franchising marquee content like South Park and developing new IP with tremendous talent like Matt and Trey, is at the heart of our strategy to continue growing Paramount+."

Since the onset of the COVID-19 pandemic, many film festivals have shifted to online-only or hybrid formats. Later this month, another movie will premiere as a paid online event. This time around, you'll be able to watch it on Facebook.

Users in any country where Facebook's paid online events are available (which now number more than 100) can watch the premiere of The Outsider, a behind-the-scenes documentary about the National 9/11 Memorial and Museum in New York City. A virtual ticket costs $4, and the film will debut on August 19th at 8PM ET. It'll be available for 12 hours.

As Axios reports, a Facebook Live panel discussion will follow the premiere of The Outsider. The doc will hit select theaters and other streaming platforms in September.

Facebook will run some promos for the event, which is being run by distributor Abramorama, but it won't take a cut of ticket sales. The company is waiving commissions on creators' revenue through 2022.

The film has already caused controversy. Officials at the museum asked the filmmakers to cut 18 "defamatory" scenes from The Outsider, but directors Pamela Yoder and Steven Rosenbaum said they wouldn't back down. Michael Shulan, the museum's former creative director and a central figure in the film, reportedly claims in the movie that the museum represents the “Disneyfication” of the September 11th, 2001 terrorist attacks.

Film distribution is a tough nut to crack for indie studios and filmmakers, especially when they try to release movies in a number of markets. Much like others have had success in hosting online classes or livestreaming gameplay on Facebook, they could harness the platform's enormous reach to find an audience.

It remains to be seen whether other filmmakers and distributors premiere their movies on Facebook. Still, with the company having its fingers in an ever-increasing number of pies, it's not hard to imagine Facebook being interested in hosting similar events in the future.

When Inspiration4, SpaceX’s first all-civilian flight, takes off next month, Netflix will chronicle the historic mission with a documentary series. Countdown: Inspiration4 Mission to Space will stream in five parts, with the first two episodes debuting on September 6th. In that way, it will be the first docuseries from the streaming giant to chronicle an event in near real-time.

Early episodes will detail, among other things, the astronaut training Inspiration4 commander Jared Isaacman and his crewmates had to undertake ahead of the flight. Meanwhile, the final feature-length episode will recount the mission’s flight to space and eventual return to Earth and include footage from inside the Crew Dragon spacecraft. The finale doesn’t have a release date yet, but Netflix expects to start streaming it sometime in late September.

The production involves parts of the team behind ESPN’s The Last Dance Michael Jordan documentary, including director Jason Hehir. Alongside the series, Netflix will release A StoryBots Space Adventure, a hybrid live-action and animation special that will feature the Inspiration4 crew answering questions from kids about their flight. The special will debut on September 14th, one day before the Inspiration4 mission is scheduled to lift off.

Adult Swim and Crunchyroll has released the first trailer for Blade Runner: Black Lotus, the anime series they're co-producing, at San Diego Comic-Con this year. The show is set in Los Angeles in the year 2032, putting its events in between the original Harrison Ford movie set in 2019 and the sequel film starring Ryan Gosling set in 2049. It features a new replicant named Elle known as the "Black Lotus," who was created with special powers. She seems to have escaped from her creators, and is currently being hunted down by authorities.

In the action-packed trailer, you'll see Elle take down foe after foe — she goes from not knowing how she's able to knock a handful of men completely out cold to wielding a katana — in a backdrop of smoke, fog and neon lights. Elle is voiced by Jessica Henwick (Iron Fist) in the English version and Arisa Shida in the Japanese version. The show will run for 13 episodes, which will be directed by Shinji Aramaki (Ultraman, Ghost in the Shell: SAC 2045) and Kenji Kamiyama (Ghost in the Shell: Stand Alone Complex, SAC_2045). It's produced by Alcon Entertainment and animation studio Sola Digital Arts, with Shinichiro Watanabe (Cowboy Bebop) serving as a creative producer.

When Blade Runner: Black Lotus debuts this fall, you can watch it in English on Adult Swim and in Japanese on Crunchyroll.

District 9 director Neill Blomkamp never did get to make his Halo movie. However, Blomkamp still has a foothold in the video game world after directing an Anthem prequel short film. He's now helping a studio called Gunzilla Games with its first game, a multiplayer shooter.

Blomkamp — who also directed Elysium, Chappie and the upcoming Demonic — has joined Gunzilla as chief visionary officer, as IGN reports. He'll help guide the aesthetic of the game and provide input on aspects such as design, audio and the narrative from a film director's perspective. He admitted that he hasn't worked in game development before, so collaborating closely with other key creatives on the project will be crucial.

The studio was formed last year and employs developers who have experience at the likes of Crytek, Ubisoft and EA. The shooter, which is still largely under wraps, might not be a one-and-done deal for Oscar nominee Blomkamp.

"Games will [...] become what films were in the 20th century," he told IGN. "They'll just be the thing that is the dominant form of cultural entertainment and [I want] to be in that. Mixing my history in visual effects and interest in 3D graphics means I want to have a home base in the creation of games for a really long time. So if the game is a success and everything works out, hopefully I'm staying at Gunzilla for a long time."

Blomkamp is joining a long line of notable filmmakers and creatives from other mediums who've moved into games. Guillermo del Toro was set to direct the canceled Silent Hills alongside Hideo Kojima, while George R. R. Martin helped craft the world of Elden Ring. Steven Spielberg has credits on several games too. Along with his work on the Medal of Honor series, he was creative director on an EA puzzle game called Boom Blox.

On July 16th, Roadrunner: A Film About Anthony Bourdain will open in US theatres. Like many documentaries, the film pieces together archival footage, including interviews and show outtakes, to attempt to tell the story of its subject in their own words. It also includes words Bourdain never spoke to a camera before his death by suicide in 2018, and yet you’ll hear his voice saying them.

In an interview with The New Yorker, the film’s director, Morgan Neville, said there were three quotes he wanted Bourdain to narrate where there were no recordings, and so he recreated them with software instead. “I created an AI model of his voice,” he told the magazine.

It appears that was no easy feat either. In a separate interview with GQ, Neville said he contacted four different companies about the project before deciding on the best one. That company fed about a dozen hours of audio to an AI model. A lot of the work involved deciding the exact tone of Bourdain’s voice Neville wanted the software to replicate since the way the author and travel host narrated his writing changed so much over the years he was on TV.

Compared to some of the other ways we’ve seen AI and deepfakes used to trick people, this isn't the worst example, but the ethics of it are still questionable. The film, as far as we’re aware, doesn’t include a disclosure that AI was used to replicate Bourdain’s voice. “If you watch the film, other than that line you mentioned, you probably don’t know what the other lines are that were spoken by the AI, and you’re not going to know,” Neville told The New Yorker. “We can have a documentary-ethics panel about it later.” In his interview with GQ, he said Bourdain’s family told him “Tony would have been cool with that,” adding, “I was just trying to make [the quotes] come alive,”

In case you didn’t hear earlier, Lokiwill return for season two, and thank goodness: the finale didn’t resolve a whole lot, if it resolved anything. Well, we did find out who was pulling the strings behind the Time Variance Authority and why, but it really served as an introduction to a villain who’s scheduled to make his next appearance in Ant-Man and the Wasp: Quantumania.

However, Loki never felt like an extended prequel to well… anything. It’s been a show that’s stood largely on its own, one which forged a unique identity apart from everything else in the Marvel Cinematic Universe, including the Thor movies. After all, this isn’t the Loki we spent eight years watching on screen. This one was created in 2019 in the middle of Avengers: Endgame, a variant that so far is walking a path free of Asgard and all that pesky Avengers business.

Chuck Zlotnick / Marvel Studios

And so Loki the show paved its own road as well, not really relying too much on knowledge of the films past the first and last Avengers movies. You could walk into the series knowing as much about Loki’s fate as the variant Loki and walk away after the first season knowing just as little. The finale was interesting because for the first time, I have no idea how a Marvel show fits into the greater scheme of things. And like He Who Remains, I find that a bit exciting.

One thing that has been true of all the Marvel shows has that we’ve always generally known where they’re supposed to slot into the bigger universe. Agents of SHIELD was originally intended to be a way for the side stories of the Marvel Cinematic Universe to thrive but for a while it was treated as a place to dump movie leftovers. Daredevil, Luke Cage, Jessica Jones and Iron Fist were originally aimed toward setting up The Defenders limited series. WandaVision was a prequel for Doctor Strange and the Multiverse of Madness, while The Falcon and the Winter Soldier did character work for the next Captain America film.

Marvel Studios

But Loki, aside from wrapping up a loose thread from Endgame, doesn’t actually tie into anything, especially any upcoming projects. They don’t even name the man behind the curtain, he’s just “He Who Remains.” And he’s killed by the end of the episode, which means any time we see him from this point forward, it’s technically a different person; another cosmic iteration of the same jerk. Fans of the comics know that he’s meant to be Kang the Conqueror, but that big reveal is yet to come — because the man we met is not a conqueror. He was a cosmic bureaucrat.

The conqueror, though hinted at in the Loki finale, will make his first full appearance in early 2023. He won’t remember the events shown in the Disney+ show since he wasn’t actually there, which means either the film will completely ignore all we learned here, or explain it to the audience anew. Prior knowledge of Loki shouldn’t and most likely won’t be necessary.

Chuck Zlotnick / Marvel Studios

Which leaves season two of Loki free to do well, almost anything it wants. The man who seems to now be in charge of the TVA is unlikely to be even close to the same man that Scott Lang and Hope Pym will have to tangle with, since we are dealing with a multiverse of possibilities. It does throw the next Doctor Strange movie into a bit of uncertainty, since in our original non-COVID timeline that was supposed to have premiered back in May. Were we supposed to see the debut of the multiverse before we saw its origin? Or will there be some other cosmic wrench to mess things up further for the MCU? A lot of things that seemed inevitable when all these projects were originally announced have now been thrown into uncertainty.

But for now, the key takeaway from the Loki finale is that the series is not beholden to anything else in the Marvel Cinematic Universe. It didn’t have to load up a ton of character development or make sure everything is back in place plot wise before the next film. It’s free to take its small cast of characters and fully explore their possibilities, with no worries about how it will affect the other Marvel properties. What will happen to Loki, Sylvie, Mobius, Ravonna and the rest? We can focus on their individual stories instead of fussing over what can and can’t happen.

We can no longer make predictions based on future projects in the pipeline, or contractual obligations of actors. Theories can proliferate and almost nothing is off-limits. Every step Loki and the rest take just means another multiverse to explore in shows like What If?. The Sacred Timeline is dead, and so are the shackles of movie continuity.

A few days ago, Universal signed a deal with Amazon to give Prime Video exclusive streaming rights for its live-action releases. This time, Universal has struck a deal with Netflix, specifically for its animated films. The companies have signed a multi-year licensing agreement that gives the streaming service exclusive access to animated movies from Universal-owned Illumination, such as Minions: The Rise of Gru, which comes out in 2022. They've also expanded their partnership to include films from DreamWorks Animation, including the upcoming The Bad Guys and Puss In Boots: The Last Wish.

Similar to Universal's pact with Amazon, though, the Netflix deal is a bit complex. Yes, Netflix will have exclusive rights to animated films from Illumination and Dreamworks in the US, but only for a chunk of the 18-month period after the movies' theatrical run. Within the first four months, all Universal films will only be available for streaming on Peacock, the streaming service also owned by its parent company Comcast. Netflix will then have exclusive streaming rights for 10 months after that before the movies go back to Peacock for another four months. 

Once that 18-month period is done, Universal's animated films will also be available on Amazon Prime Video. Meanwhile, under their new deal, Netflix will license Universal's full animated and live-action slate four years after their release and will also secure streaming rights to select titles from the studio's library. While the deal can be a bit confusing, it all boils down to the fact that Netflix will continue having exclusive access to some of what could become the most-watched films on the platform. As Variety noted, Illumination's The Secret Life of Pets 2 is the most-watch Netflix movie in 2021 so far, according to a Forbes analysis. Last year, that distinction belonged to another film by Illumination: Despicable Me.

Disney+ has been shy about divulging sales for Premier Access movies like Mulan, but it's now eager to brag. As AV Clubreports, Disney has revealed that Marvel blockbuster Black Widow raked in more than $60 million in estimated revenue solely from customers paying $30 each for Premier Access. That's more than a quarter of the roughly $215 million Disney expected to make from all sources, including US and international box office sales.

The company also bragged that Black Widow was the largest US box office opening during the COVID-19 pandemic, just besting F9. That wasn't difficult, though, given that movie theaters have routinely struggled to attract viewers when they weren't dealing with widespread closures.

We wouldn't count on Disney providing Premier Access figures on a regular basis, especially as the pandemic continues to (hopefully) subside and more people are comfortable venturing to theaters. To some extent, the media giant needed the $60 million number to make Black Widow's debut sound more impressive. We'd add that it's difficult to compare this against data for rivals like Amazon and Netflix. They don't have equivalents to Premier Access, so they measure success for original movies largely in terms of viewership and new subscriptions.

Nonetheless, this represents an important milestone for Disney as it shifts toward streaming. It's now comfortable mentioning Premier Access numbers after months of staying quiet. Even if Premier Access fades into the background, it's now a significant part of Disney's movie release strategy.