Using shot-scraper to automate screenshots for my writing

Author

Wasim Lorgat

Published

May 22, 2023

I’m trying out Simon Willison’s shot-scraper to include a screenshot of the website for our JupyterCon 2023 tutorial in my blog post.

First install the shot-scraper command line tool:

!pip3 install shot-scraper

Collecting shot-scraper
  Downloading shot_scraper-1.2-py3-none-any.whl (15 kB)
Collecting click
  Downloading click-8.1.3-py3-none-any.whl (96 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 96.6/96.6 kB 3.7 MB/s eta 0:00:00
Requirement already satisfied: PyYAML in /opt/homebrew/lib/python3.11/site-packages (from shot-scraper) (6.0)
Collecting playwright
  Downloading playwright-1.33.0-py3-none-macosx_11_0_arm64.whl (31.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 31.2/31.2 MB 5.5 MB/s eta 0:00:0000:0100:01
Collecting click-default-group
  Downloading click-default-group-1.2.2.tar.gz (3.3 kB)
  Preparing metadata (setup.py) ... done
Collecting greenlet==2.0.1
  Downloading greenlet-2.0.1-cp311-cp311-macosx_10_9_universal2.whl (259 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 259.4/259.4 kB 5.3 MB/s eta 0:00:00a 0:00:01
Collecting pyee==9.0.4
  Downloading pyee-9.0.4-py2.py3-none-any.whl (14 kB)
Requirement already satisfied: typing-extensions in /opt/homebrew/lib/python3.11/site-packages (from pyee==9.0.4->playwright->shot-scraper) (4.5.0)
Building wheels for collected packages: click-default-group
  Building wheel for click-default-group (setup.py) ... done
  Created wheel for click-default-group: filename=click_default_group-1.2.2-py3-none-any.whl size=3383 sha256=816707e9974b43b6f04f2e646cf851f2e041f0c7cc25fcb67612fb4d72973d4f
  Stored in directory: /Users/seem/Library/Caches/pip/wheels/69/9a/ed/1979767796ee1379d161a35fea9745a788476be12fb2ac664a
Successfully built click-default-group
Installing collected packages: pyee, greenlet, click, playwright, click-default-group, shot-scraper
Successfully installed click-8.1.3 click-default-group-1.2.2 greenlet-2.0.1 playwright-1.33.0 pyee-9.0.4 shot-scraper-1.2

[notice] A new release of pip is available: 23.0.1 -> 23.1.2
[notice] To update, run: python3.11 -m pip install --upgrade pip

Then install the underlying browser used by shot-scraper:

!shot-scraper install

We can now take a screenshot of any website!

!shot-scraper https://fastai.github.io/jupytercon-2023/ --output ../posts/images/jupytercon-2023-tutorial.png --width 1600 --height 900

Here’s the image:

I want a resolution of 1600x900 but the font size is too small here. Halving the width and height and doubling the device scale factor with the --retina flag should fix this:

!shot-scraper https://fastai.github.io/jupytercon-2023/ --output ../posts/images/jupytercon-2023-tutorial.png --width 800 --height 450 --retina

Finally, I want to exclude the nav bar. We can use a CSS selector for that:

!shot-scraper https://fastai.github.io/jupytercon-2023/ --output ../posts/images/jupytercon-2023-tutorial.png --selector '#quarto-content' --width 800 --height 450 --retina

Ah, it doesn’t seem like --selector works together with --width and --height. Another way to do this might be by hiding the navbar by executing --javascript code:

!shot-scraper https://fastai.github.io/jupytercon-2023/ --output ../posts/images/jupytercon-2023-tutorial.png --width 800 --height 450 --retina --javascript "document.querySelector('#quarto-header').style.display = 'none'"

Perfect! I’m chuffed with how easy this was. I typed this out exactly as I used it, and everything worked first time with zero errors. I think it would’ve been neat if I could use --width and --height to crop the image after a --selector is applied, but in the end I could get what I wanted using a single line of --javascript.