Skip to content

Instantly share code, notes, and snippets.

@charlesdaniel

charlesdaniel/streamforge-docs.md

Created February 26, 2025 18:31
Show Gist options
  • Save charlesdaniel/8daa16c7ca8fd59b36be1a78e71d81b3 to your computer and use it in GitHub Desktop.
Save charlesdaniel/8daa16c7ca8fd59b36be1a78e71d81b3 to your computer and use it in GitHub Desktop.
Download ZIP
Streamforge Docs
Raw
streamforge-docs.md

StreamForge

Introduction

StreamForge is an Asset Retrieval and Transformation Service originally built for DataCraft but can also be used directly standalone. It offers several features:

  • Common HTTP interface to various asset storage backends (vedi://, https://, conductor:// urls)
  • Authentication based on Notary token provided via header or cookie
  • Authorizes using the backend but also provides extra layer of prefix level security policies based on complex Apple Group unions/intersections
  • Provides for on-the-fly transformations for various Image/Video/Archive/Header manipulations on the asset using a simple Domain Specific Language encoded in the URL itself
  • Built-in caching of transformation results

Be sure to try the Playground to load up some assets and try some transformations.

Use Cases

StreamForge was originally created for Web interfaces to quickly and easily retrieve and display various assets from several storage locations. StreamForge provides several on-the-fly transformations including for images and videos so it's ideal for generating thumbnails for assets, extracting individual files from zip files, manipulating HTTP headers, etc.

While nothing stops you from using StreamForge for other use cases; for times where you just want the raw asset and you have the credentials to get to it directly, going through StreamForge will likely be slower than going directly to the asset (since StreamForge itself is going to the source to get the asset anyway).

Server Locations:

Authentication:

StreamForge requires either a Notary token (preferred) with all the required audiences or a DAW token in order to find the user’s groups and pass it to the upstream service for authentication. StreamForge looks for the user’s authentication in the following places in order:

  1. HTTP Authorization as Notary token header - Authorization: Bearer <NOTARY_TOKEN>
    1. If available it is treated as a Notary token and sent to Notary for validation, if the validation is successful then the user is logged in.
  2. A Notary Cookie by the name of notary
    1. If a cookie called notary is present it used as the notary token and is sent to Notary for validation, if the validation is successful the user logged in.
  3. HTTP Authorization as DAW token header - Authorization: Bearer <DAW_TOKEN>
    1. If available the authorization header is tried as a DAW token and is used in creating a notary_token with the proper audiences on behalf of the user. If successful the notary token will then be used as the logged in user. The notary cookie is then set by StreamForge as well so we don’t have to redo this process.
  4. A DAW Cookie by the name of acack (if set as enabled in the StreamForge configurations)
    1. If a cookie called acack is present then it is used as the DAW token. A notary_token is created for the user with the proper audiences using that DAW token. If successful the notary token will then be uses as the logged in user. The notary cookie is then set by StreamForge as well so we don’t have to redo this process.
  5. A DAW Cookie by the name of acack-uat (if set as enabled in the StreamForge configurations)
    1. If a cookie called acack-uat is present then it is used as the DAW token. A notary_token is created for the user with the proper audiences using that DAW token. If successful the notary token will then be uses as the logged in user. The notary cookie is then set by StreamForge as well so we don’t have to redo this process.

Notary

The following audiences are required to be present in the notary token generated for the user:

aprn:apple:turi::notary:application:conductor
aprn:apple:turi::notary:application:turi-directory
aprn:apple:turi::notary:application-group:polymer
aprn:apple:turi::notary:application-group:turi-platform

You can easily generate it via the NotaryCLI like so

export NTOKEN=$(notarycli issue \
    --audience="aprn:apple:turi::notary:application:conductor" \
    --audience="aprn:apple:turi::notary:application:turi-directory" \
    --audience="aprn:apple:turi::notary:application-group:polymer" \
    --audience="aprn:apple:turi::notary:application-group:turi-platform")

Side Note: Refrain from using the environment variable NOTARY_TOKEN as it’s already used internally by the notarycli for caching the token and it could just result in chaos and confusion as it may not refresh correctly.

Usage

From the Browser

  1. Get the user Authenticated with proper cookies (do one of the following)
    1. If your site is Notary based: Authenticate the user as normal and set a cookie called notary using the proper audiences (see Notary section above)
    2. If your site is AppleConnect based: Go through the AppleConnect flow so the acack (or in uat the acack-uat) cookie is set
  2. Now use StreamForge urls as needed

From the CLI

  1. Use the Notary CLI to create a Notary token with the appropriate audiences (see Notary section above)
  2. Call curl or some other Python script, send an Authorization header with a Bearer Token with the value of that Notary token you generated
> export NTOKEN=$(notarycli issue \
    --audience="aprn:apple:turi::notary:application:conductor" \
    --audience="aprn:apple:turi::notary:application:turi-directory" \
    --audience="aprn:apple:turi::notary:application-group:polymer" \
    --audience="aprn:apple:turi::notary:application-group:turi-platform")

> curl -v -H "Authorization: Bearer ${NTOKEN}" 'https://streamforge-dev.videoeng.apple.com/raw/vedi%3A//DEMO/...' -o /tmp/somefile

StreamForge URL Formats

HEAD Requests

https://streamforge-dev.videoeng.apple.com/<TARGET_URL>

NOTE: HEAD requests only work on the TARGET_URL with no transformations.

GET Requests

https://streamforge-dev.videoeng.apple.com/<TRANSFORMATIONS>/<TARGET_URL>

<TARGET_URL>

The TARGET_URL MUST be URL-escaped reference to the actual backend asset you want. A TARGET_URL’s url scheme determines the backend/upstream that is used. The following backend schemes are currently supported:

VEDI URLs

These URLs are those that are generated from Volstagg/Falstaff SDKs/APIs and accepted by the VEDI S3-asset service. The person_id is extracted from the notary token and is used for authorization to the backend.

vedi://<SILO>/<path_to_asset_with_query_string>

Conductor URLs

These URLs represent the conductor controlled S3-based locations. The Notary token is passed on to the Conductor backend to determine access.

conductor://<BUCKET>/<path_to_asset>

Generic HTTP/HTTPS URLs

These are any http or https based URLs to any website that’s accessible to StreamForge. This includes presigned URLs for S3 and Conductor as well.

https://...

http://...

Note: If there are URLs that are useful or require special permission/credientials please work with the StreamForge team and we can create dedicated url schemas that can simplify/enable access to those websites.

Blobby URLs (untested)

These URLs represent assets stored in Blobby. Note that Blobby uses DAW tokens for access and thus requires that you pass in the DAW access method. This URL scheme hasn’t been tested due to not having access to any current blobby urls and also no new buckets are allowed to be created in Blobby as it’s being deprecated in favor of Conductor.

blobby://<BUCKET>/<path_to_asset>

Kennel k9: URLs

We are currently awaiting support in Kennel for Notary tokens but when it happens we should be supporting k9:// urls. Stay Tuned.

<TRANSFORMATIONS>

Transformations are ways for converting the original asset into something that the consumer would find more preferable. Transformations are comma separated function calls where the order matters. Transformations are URL-escaped comma separated functions (to be exact it uses Rust Object Notation which is very similar to function call syntax). The currently available transformations are as follows:

No Transformations

Stability Content-Type Transformation Description Example
*/* raw Preserves the asset in it's raw form, unchanged. This is essentially a 'no-op' and is a final transformation. Do not use it with any other transformation as 'raw' is implied as the initial transformation. Note depending on the target you may get back a HTTP redirect to the asset rather than the asset in the body. raw

Image/Video Transformations

Stability Content-Type Transformation Description Example
image/*
video/*		 thumbnail(width: u32, height: u32) A Thumbnail quality version of the image to a width and a height. Pass in 0 for either height or width and the other dimension will be auto calculated preserving the aspect ratio. If both are > 0 the aspect ratio will still be preserved while filling the maximum space of the bounding area.

Video: When thumbnail() is used against a video it is converted to an animated WebP file that loops with a default fps of 1 to keep the file size small.		 thumbnail(width: 100, height: 100)
image/*
video/*		 flip_horizontal Flips the image on the Horizontal axis flip_horizontal
image/*
video/*		 flip_vertical Flips the image on the Vertical axis flip_vertical
image/*
video/*		 resize(width: u32, height: u32) Resizes an image to the width and height requested. Pass in a 0 for either the height or the width and it will preserve the aspect ratio. If both are passed in with values > 0 then they are used explicitly (no aspect ratio preservation) resize(width:200, height:200)
resize(width: 0, height: 200)
resize(width: 200, height: 0)
image/*
video/*		 crop(x: u32, y: u32, width: u32, height: u32) Crops an image from the (x,y) location to a width of W and a height of H crop(x: 10, y: 20, width: 100, height: 200)
image/*
video/*		 rotate(RotationEnum) Rotate the image in 90degree increments.
Valid values are only deg0, deg90, deg180, deg270		 rotate(deg90)i
image/*
video/*		 grayscale Grayscales the image grayscale
image/*
video/*		 invert Inverts the colors in the image invert
image/*
video/*		 contrast(f32) Changes the contrast of the image (value is a 32 bit float) contrast(2.0)
image/*
video/*		 brighten(f32) Changes the brightness of the image (value is a 32 bit float) brighten(4.0)
image/* cvt_color(ColorCvtEnum) Shifts the color channel bytes of an image based on the ColorCvtEnum parameter passed in.
Valid values are rgb_to_bgr, bgr_to_rgb, custom3(r_index, g_index, b_index)		 cvt_color(bgr_to_rgb)
image/* webp Converts the image to the image/webp format which may or may not be better compression size than the original. webp

Video Only Transformations

Stability Content-Type Transformation Description Example
BETA video/* frames(seek: String, num: u32) Seeks to a given timestamp (ffmpeg format equivalent to the -ss flag) and extracts num frames from it. The result is output as a webp (image/webp) animation. This can then be post-processed as an image. frames(seek: "00:00:30", num: 10)
BETA video/* fps(f32) Converts the video's frames per second to the value specified as a decimal. Negative values are also allowed. This is equivalent to the --fps flag in ffmpeg. fps(0.5) Equivalent to --fps 1/2
fps(30) 30 frames per second

Header Transformations

Stability Content-Type Transformation Description Example
*/* content_type(String) Sets the Content-Type header of the output. Depending on the original content-type, SF may conver the content into the newer content-type. Images will auto-convert when going from one image/* to another. For image/x-exr as the source it will also convert to rgba or rgb depending on the output content-type specified.

NOTE: You can use this multiple times and setting it as the first thing in the transformations list will allow you to set the content-type of the original which can be useful if the backend is missing the content-type or it's wrong. Then subsequent transformations would use the content-type you set.		 content_type("image%2Fjpeg")
content_type("application%2Foctet-stream")
*/* content_disposition(filename: String) Adds a Content-Disposition: attachment; filename=X header to trigger browsers to download the file. content_disposition(filename:"blah.jpg")
*/* content_encoding(encoding: String) Adds a Content-Encoding: X header content_encoding(encoding: "gzip")

Archive Transformations

Stability Content-Type Transformation Description Example
application/zip unzip(list) Lists the entries from a zip file target in a line delimied JSON format (1 line per entry) unzip(list)
application/zip unzip(list_glob(String)) Lists the entries that match the glob pattern within a zip file target in a line delimited JSON format (1 line per entry) unzip(list_glob("*.jpg"))
application/zip unzip(index(u32)) Extract the file at a certain index (look at the list to find the index) from a zip file. The body will be just that file not the original zip target. unzip(index(2))
application/zip unzip(file(String)) Extract the file exact matching the String from the zip file target. The body will be just that file not the original zip target. unzip(file("foo/bar/zab.txt"))
application/zip unzip(from_path) Looks at the full target url, finds the spot where ".zip" is in that string then takes everything after as the file path to extract from the zip file. The body will be just that file not the original zip target. .../unzip(from_path)/conductor://somebucket/foo/bar.zip/some/file/inside.txt
This would extract "some/file/inside.txt" from the zip file "bar.zip" in the conductor bucket.

Examples of using a StreamForge

This pulls the raw asset from the vedi://DEMO/volstagg/image/011/000/000/000/001/asset/1474.aes location (NOTE: The asset is encrypted so it’s decrypted on the fly by StreamForge prior to all transformations).

https://streamforge-dev.videoeng.apple.com/raw/vedi%3A//DEMO/volstagg/image/011/000/000/000/001/asset/1474.aes%21ZT0xLGY9ZixrPU0scz0zNTYyNSx0PXM%3D%3Fname%3DIMG_0301_0002.jpg%26md5%3D31a8e2dfa43649cb4cfb245d99d5a9b6

This pulls the same asset as above but preforms some transformations on it: sets/converts it to an image/png, flips horizontally, flips vertically, resizes to 210x190, rotates 90 degrees, grayscales and finally inverts it

https://streamforge-dev.videoeng.apple.com/content_type("image%2Fpng"),flip_horizontal,flip_vertical,resize(width:210,%20height:190),rotate(deg90),grayscale,invert/vedi%3A//DEMO/volstagg/image/011/000/000/000/001/asset/1474.aes%21ZT0xLGY9ZixrPU0scz0zNTYyNSx0PXM%3D%3Fname%3DIMG_0301_0002.jpg%26md5%3D31a8e2dfa43649cb4cfb245d99d5a9b6

StreamForge Cache API

Get the current User’s Cached Session

curl -v -H "Authorization: Bearer <NOTARY_TOKEN>" \
           'https://streamforge-dev.videoeng.apple.com/_cache/user'

Delete the current User’s Cached Information (including apple group association)

curl -v -XDELETE -H "Authorization: Bearer <NOTARY_TOKEN>" \
           'https://streamforge-dev.videoeng.apple.com/_cache/user'

Delete the Cached Headers and Content Body of a <TARGET_URL>

curl -v -XDELETE -H "Authorization: Bearer <NOTARY_TOKEN>" \
           'https://streamforge-dev.videoeng.apple.com/_cache/obj/<TARGET_URL>'

NOTE: The User is verified to have access prior to being allowed to delete the cached object.

NOTE: The <TARGET_URL> is the base url that you would pass in to a HEAD call (no transformations). When deleting an object, the original target asset and all transformed derivatives of that target asset including all cached headers will be removed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment