Stacks Explorer 2

This page details how the site was built. If you're looking the user guide, click here.

GitHub Repo: https://github.com/DevinSmithWork/prelinger-stacks-explorer-2

Table of Contents

1.  How This Thing Was Put Together: Overview

1. PHOTOS:
   • Shooting: This time around, I shot in raw format, resulting in better-quality photos at the expense of some extra conversion steps. I also rigged up a camera mount that attached to the library's rolling ladders.
   • Editing: You'll notice the individual photos are now 1:1 with the shelves' dimensions. I wrote a Processing program to facilitate this editing in a streamlined way.
   • Deep Zoom Image [.dzi] format: I converted the full-stack composites to .dzi for use with OpenSeadragon (see below). If you're looking to replicate this workflow, there's some edge cases to be aware of.
2. DATA ENTRY: During the library's refresh, Megan produced an excellent guide for in-person visitors. I used this guide as a starting point for the zones/subzones organization of the site. The Internet Archive's command-line tool was helpful for creating a starting point for entering data about the scanned items.
3. UX: OpenSeadragon [OSD] is an open-source JavaScript [JS] library for displaying .dzi images. OSD is robust, full-featured, and has an active community with a fantastic lead developer. The JS code that translates back-and-forth between library-space and OSD-space is fairly complex; See the section "4 Website Structure" below for details.

2.  Photography & Image Processing

2.1 General Overview

At a high level, what's needed is a photo of every shelf and a way to stitch them together -- but along the way, there's some procedural decisions that can influence both the shooting efficiency and the overall aesthetic feel of the website. My process involved (1) rigging up a camera mount for shooting, (2) automating raw-to-tiff conversions with Darktable, (3) writing a Processing program for adjusting the photos' perspective/skew, (4) using Processing to create the shelf composites, and (5) converting the composites to .dzi format for OpenSeadragon with a quick python script. Of the three main parts of this documentation (Photos, Data, and UX), this section includes the most potential for automation and general workflow improvements.

2.2 Photography

I used a camera similar setup as the first version, with a fixed aperture, variable exposure time, and fixed white balance. The fixed aperture provides a consistent depth of field, and the fixed white balance provides a consistent color spectrum when the photos are composited (This helps the the user visually locate themselves in a physical environment).

Once I figured out a feasible way to handle the skew/perspective editing (see below), I opted to shoot in 4:3, the camera's native sensor dimensions. The shooting went quicker this time around because the framing didn't need to be precise, and I rigged some stand clamps from Sammy's onto the library's rolling ladders, which helped with shooting higher shelves.

Another request for v2 was shooting in raw, a lossless digital format. (I shot v1 in HQ .jpg to reduce filesizes and expedite the Processing workflow.) Raw files require an application to combine the camera's sensor data with metadata to produce a viewable image. Adobe software is the industry standard, but Darktable is the free, open-source solution, and it has a command line interface for automating common tasks. (Like most open-source software, it's a little clunky and there's a learning curve.) Here's the batching workflow I used:

1. Toss a couple raw files into the lighttable
2. Open one up in the darkroom, and make your desired adjustments with the modules.
3. Save your editing history stack as a style.
4. Switch back to the lighttable, and apply this style to the other images to test how it looks. Adjust as needed.
5. After you've got a style that you like, you apply it to batches using the command-line tool:

2.3 Adjusting skew/perspective and compositing with Processing

Once the raw files were processed and converted to .tif, the next step was skew/perspective adjustments and resizing, which I wrote a Processing program for. The program asks for a folder of images, which are displayed in order; The user clicks on the four corners of the shelf, after which the clicked-corners are mapped to the shelf's IRL height:width ratio and the resized image is exported. The library has some slight variation in shelf heights and widths, so taking measurements and changing the IRL height & width variables was sometimes needed.

Processing's default texture-mapping renderer is kinda lazy: it draws an "X" connecting the four edge points, and stretches the four resulting triangles... which looks lousy. After some experimentation, I found excellent results from creating a large number of stepped points (i.e., lerping) between both the clicked-corner and export-image-corner points, and drawing them as vertices inside a loop. From a rendering standpoint, this creates so many tiny "X"s that the triangles appear contiguous and the resizing looks smooth.

One unexpected benefit of this strategy is that placing edge points in odd places or out-of-order creates some wild artwork!

Creating large composites from rectangular photos is fairly straightforward, requiring only a few nested loops. But for composites of this size, using an offscreen PGraphics object is advisable to avoid possible pixel limits in size() and redraw() delays.

2.4 Creating DeepZoom images

Before your very-large stack composite can be used with OpenSeadragon, it needs to be converted to one a few different image tiling formats: .dzi, .iiif, .tms, etc -- See here for a full list. I used deepzoom.py to generate .dzi files; Here's the basic "hello world" they provide:

import os
import deepzoom

SOURCE = "example.jpg"

creator = deepzoom.ImageCreator(
    tile_size=128,
    tile_overlap=2,
    tile_format="jpg",
    image_quality=0.8,
    resize_filter="bicubic",
)

creator.create(SOURCE, "./example.dzi")

3.  Data Entry & Prep for website

3.1 Physical library

The Prelinger Library's idiosyncratic organization creates both opportunities and drawbacks for getting data into a website like this. Detached from a standardized catalog, I instead worked from Megan's guide as a starting point, going shelf-by-shelf and noting the zone/subzone, periodicals, and special collections.

Some areas of the library (oversize, ephemera, and Therkleson in particular) include material from a variety of different zones/subzones on a single shelf. In these cases, the first zone/subzone is displayed first in the sidebar "click view," with the additional zones/subzones under "Material also on this shelf." In the sidebar "list view," every shelf containing a zone/subzone will be highlighted; Hence the user sees every place in the library with material on that topic.

3.2 Scanned Items (Internet Archive collection)

Stacks Explorer v1 used a roundabout method to link scanned items to the physical collection: We entered keywords for each library shelf, and if those keywords returned a non-zero result when searching the Prelinger's IA collection, we included that search link in the interface.

This time around, we instead stepped through every item in the IA collection, and assigned to one-or-more of the physical library's zones/subzones -- That is, we "shelved" the scanned items into the library's physical organization. This means (1) every scanned item is present in the interface, and (2) some scanned items (most, really!) appear in multiple locations.

ia search "collection:prelinger_library" -i > item_identifiers.txt;

ia download --itemlist="item_identifiers.txt" --destdir=cli-meta-downloads --glob=*_meta.xml --no-directories;

This multiple-location shelving strategy led to some fascinating consequences, my favorite of which is the unsioling of children's material. Typically, material for youngsters is segregated into a physical location regardless of its topic; But with this new strategy, a kids book about cars sits alongside a technical document on highway design.

3.3 Data Prep

Python's streamlined looping made it the easiest tool for getting our various CSVs into a more digestible .json format for the website. The main data file uses a 3D array structure for storing each shelf's data: That is, library_nested[4][11][2] contains the data for bank 4, shelf 11, row 2. The overall relationship between the files is quasi-2NF: Periodicals, Special collections, and IA items are referred to by index keys in the main file, with the details stored in additional files.

Once again, this data structure responds to our minimal back-end setup and is designed for quick load times. Compared to the data entry CSV files, we dropped from 673kb to 404kb, a 40% compression ratio. (It ain't much, but it's honest work.)

4.  Website Structure

4.1 General Overview

• Not including the <head>, the index.html is only 100 lines long; Nearly all the content is created dynamically with JS. (Side note: This is terrible for accessibility -- we're brainstorming alternate interfaces for screen readers.)
• Script.js begins with a chonky object named layout, containing all the variables needed for activities like:
  • placing the stacks dzi onto the canvas
  • figuring out which shelf the user clicked on
  • highlighting shelves when a user selects something in the sidebar's "list view."
• Because several stacks share similar sizing characteristics with only slight variations, the layout.standard object contains a few more variables to accommodate these slight variations.
• The OpenSeadragon [OSD] canvas is loaded (unsurprisingly) into <div id=osd-container></div> in the .html.
• All the .json files are loaded and parsed (also unsurprisingly) in the JS function loadLibrary­Data­WithCallbacks(). After all the .json files have loaded, (1) the sidebars are populated,(2) the event handler for processing clicks on the OSD canvas (which reference the .json data) is added, and (3) the loading window cross-fades to the menu.
• From there on out, the JS code is primarily concerned with (1) translating back-and-forth between the visual space of the OSD canvas and the library data (see "Idea: Skip the compositing..." above for an alternate strategy); And (2) handling the UI functions in the sidebars.

The interchange between library data ↔ OSD canvas is the hairiest part of the code, so I'll describe how it works in these next two sections. (The non-minimized script.js is fairly well-commented if you want more fine-grain details.)

4.2 OSD canvas click → library data

The JS function getLocationFromViewportPoint() handles this process:

1. OSD provides the stack number from the clicked image; Retrieve tileSources[].layout from the clicked image.
2. Using tileSources[].layout, loop layout.bank_widths or uniform_bank_width to find the bank number.
3. In tileSources[].layout, select the apropriate shelf_heights array, and loop to find the row number.
4. During the steps to find the stack, bank, and row, if the click's viewport point is found to be outside the image's boundaries, false is returned.
5. The resulting values are used to get the data in stacks_data[stack][bank][row].

4.3 Library data → OSD canvas/viewer

When the user interacts with an element in the sidebar lists -- e.g. carroting open the zone/subzones <details>, or clicking on periodicals or special collections -- the element calls the JS function hl(this):

1. If the element is already highlighted, it's classList will include "text-hl1" or "text-hl2." If so, the element's highlights are removed along with its subelements' highlights. Otherwise:
2. this.id contains an alphanumeric string indicating the type of element (e.g. zone, subzone, periodical, etc), and an index in the relevant data source. For example:
   • z-2 = zones_data[2]
   • s-4-5 = zones_data[4].sub[5]
   • p-139 = periodicals_data[139]
3. This alphanumeric string is parsed in process­Element­For­Highlighting(), and the resulting loc array (each item is a stack location, stored in [stack][bank][row] format) is passed to drawHighlights()

DrawHighlights() loops through the loc array, calculating the stack location's pixel position on the OSD viewer with calcViewerHighlight(), and creates the highlights (which are OSD Overlay objects).

CalcViewerHighlight() finds the stack location's pixel values in a 2-step process (x & width and y & height), and references the chonky layout object.

Each highlight (aka OSD Overlay) requires an HTML element so it can be appended to the DOM and styled as needed. The letter in the alphanumeric string determines the CSS styling applied to the Overlay, via classList.add().