The rolling shutter and the propeller

Simulating the rolling shutter effect in R

Introduction

I recently flew on a Bombardier Dash 8 aeroplane whilst travelling to see my parents for the Christmas holidays. During that flight, I was reminded of an interesting phenomenon that occurs when you take a photo of an object moving at fast speeds (relative to the shutter speed of the camera). More specifically, when you take a photo of an object moving at fast speeds with a camera that employs a rolling shutter.

Because the shutter ‘rolls’ from one side of the picture to the other, objects that significantly change position whilst the shutter is ‘rolling’, tend to appear distorted in the final photograph. This site explains it brilliantly. And below is a nice example photo taken from here

So I’ve decided to try and simulate this in R. As with most things - this took a bit of time, with several failed attempts, and what you see here is the final ‘sanitised’ version of where I got to. There is a clear bug in this implementation that we will see below (and I’m not sure I know how to fix it)…something I will need to think about another time!

Code

I originally made this implementation in base R but I think it’s much more elegant using the tidyverse!

library(tidyverse)

This function returns a dataframe that I can then plot. I’ve commented the code to try and make sense of it, but basically it

  • Creates an initial tibble with res rows that has
    • shutter_position varying from 1 to -1 in res steps
    • shutter_time which varies from 0 to 1 in res steps (so that the shutter speed of the camera is always 1 second).
    • prop_freq The propeller angular frequency in revolutions per second.
      • As the shutter speed is always 1 second, a prop_freq of 1 means the propeller will make one whole rotation during the photo. A prop_freq of 2 means the propeller will make two whole rotations during the photo.
  • I then add the position of each of the three propeller blades at each time (each row) and compute where the propeller blade crosses the shutter (this is where the bug comes in…as if the situation arises where the propeller blade and the shutter are exactly parallel (only possible at shutter_position = 0.5) then the two lines cross at infinity and I will not have any crossing points to plot)
  • The rest it just uniting the columns for each propeller into its own column beofre gathering into long format (for easier plotting). I then separate the propeller columns out and a grouping variable for the animation (later)
prop <- function(res = 100, prop_freq = 1, shutter_time = 1){
  
  tibble(shutter_position = seq(1, -1, l=res),
         shutter_time = seq(0, 1, l=res),
         prop_freq = prop_freq) %>% 
  mutate(prop1_theta = ((2*pi*prop_freq) * shutter_time) + 0,
         prop1_x = cos(prop1_theta),
         prop1_y = sin(prop1_theta),
         prop1_crossing = shutter_position/(prop1_y/prop1_x),
         
         prop2_theta = ((2*pi*prop_freq) * shutter_time) + (2*pi/3),
         prop2_x = cos(prop2_theta),
         prop2_y = sin(prop2_theta),
         prop2_crossing = shutter_position/(prop2_y/prop2_x),
         
         prop3_theta = ((2*pi*prop_freq) * shutter_time) + (4*pi/3),
         prop3_x = cos(prop3_theta),
         prop3_y = sin(prop3_theta),
         prop3_crossing = shutter_position/(prop3_y/prop3_x),
         ) %>% 
  unite(prop1, prop1_theta, prop1_x, prop1_y, prop1_crossing) %>% 
  unite(prop2, prop2_theta, prop2_x, prop2_y, prop2_crossing) %>% 
  unite(prop3, prop3_theta, prop3_x, prop3_y, prop3_crossing) %>% 
  gather(k, v, -shutter_position, -shutter_time, -prop_freq) %>% 
  separate(v, into=c("angle", "x", "y", "crossing"), sep="_", convert = T) %>% 
  mutate(col = case_when((crossing^2 + shutter_position^2) > 1 ~ "outside",
                         TRUE ~ k)) %>% 
  group_by(shutter_position*-1) %>% 
  mutate(ind = group_indices()) %>% 
  ungroup()
}

I can now map the prop function over a range of propeller frequencies and plot the ‘photos’ that will be produced.

map_dfr(seq(0.2, by=0.2, l=15), ~prop(res=500, prop_freq = .x)) %>% 
  ggplot(aes(crossing, shutter_position))+
  geom_point(aes(col=col), size=1)+
  coord_fixed(xlim=c(-1, 1), ylim=c(-1,1))+
  scale_colour_manual(values=setNames(c(1, 1, 1, NA), 
                                      c("prop1", "prop2", "prop3", "outside")))+
  geom_polygon(data = tibble(x=cos(seq(0, 2*pi, l=100)), y=sin(seq(0, 2*pi, l=100))),
               aes(x, y), fill = NA, col = 1, lty = 2)+
  geom_polygon(data = tibble(x=0.2*cos(seq(0, 2*pi, l=100)), y=0.2*sin(seq(0, 2*pi, l=100))),
               aes(x, y), fill = 1)+
  facet_wrap(~prop_freq, ncol=5)+
  theme_minimal()+
  theme(legend.position = "")+
  labs(title = "Rolling shutter and propeller",
       subtitle = "Shutter speed is 1 second - facetted by propeller rotation frequency (revolutions per second)",
       x = "",
       y = "Shutter position")

Animation

Here I loop through a large set of images and produced an animated GIF with imageJ. I did try to get the animation to build from the R markdown chunk but I couldnt get it to build with a purple hue over the whole picture.

NOTE how no points are plotted when the shutter line is at y=0 for the second and third animated plots (propeller frequency of 1 and 2) as the shutter line and the propeller blade are parallel to eachother. This is the bug I mentioned earlier!

res <- 400
all <- map_dfr(c(0.5, 1, 2), ~prop(res=res, prop_freq = .x))

all %>% filter(prop_freq  == 2, between(shutter_position, 0.49, 0.51))

for(i in 1:res){
  
  p <- 
    all %>% 
    filter(ind %in% 1:i) %>%
    ggplot(aes(crossing, shutter_position))+
    geom_point(aes(col=col))+
    geom_hline(data = all %>% filter(ind == i), aes(yintercept = shutter_position))+
    coord_fixed(xlim=c(-1, 1), ylim=c(-1,1))+
    geom_segment(data = all %>% filter(ind == i), aes(x=cos(angle), y=sin(angle), xend=-cos(angle), yend=-sin(angle), col=k))+
    scale_colour_manual(values=setNames(c(viridis::plasma(3, end=0.8), NA), 
                                        c("prop1", "prop2", "prop3", "outside")))+
    geom_polygon(data = tibble(x=cos(seq(0, 2*pi, l=100)), y=sin(seq(0, 2*pi, l=100))),
                 aes(x, y), fill = NA, col = 1, lty = 2)+
    geom_polygon(data = tibble(x=0.2*cos(seq(0, 2*pi, l=100)), y=0.2*sin(seq(0, 2*pi, l=100))),
                 aes(x, y), fill = 1)+
    theme_minimal()+
    theme(legend.position = "")+
    facet_wrap(~prop_freq)+
    labs(x = "",
         y = "Shutter position",
         title = "Rolling shutter and propeller",
         subtitle = "Three different propeller angular frequencies [0.5, 1 and 2 revolutions per second]")
  
  ggsave(filename = glue::glue("animation_figures/{i}.png"), plot = p, 
         units = "in", width = 10, height = 6)
}

Example rolling shutter and propeller image

Avatar
Chris Holmes
Senior Data Scientist

PhD physicist making his way in the world of data science!

comments powered by Disqus