Art & Design

Graphic Design

Web Design

See: Web Design

GIMP

GIMP (GNU Image Manipulation Program) is an extensible Open Source raster image editor (BMP, PNG, JPEG, GIF, ICO, ..., GIMP .xcf).

GIF

GIF is a file format for sequences of images (“animations”).

GIFs appear to load quickly because there is no media player or codec initialization time, but are actually less eficient than compressed video (e.g. H.264) because GIFs are not compressed and contain every frame of the animation.

Video codecs compress the changes between frames, in order to avoid basically GIF.

ICO

ICO is a computer graphics file format for icons.

  • ICO files store multiple resolutions in one file.
  • GIMP can work with ICO files

JPEG

File Extension: .jpg, .jpeg

PNG

File Extension: .png, .png

SVG

File Extension: .svg

3D Modeling

Blender

Blender is an Open Source 3D computer graphics software for movies, apps, and games written in C, C++, and Python.

  • Blender has a scene graph over time t
  • Blender has multi-camera tracking
  • Blender has a Python API

NURBS

NURBS (“Non-rational Uniform B-Splines”) are scale-invariant n-dimensional models.

  • NURBS curves are sort of like SVG for 3D Modeling: they scale losslessly from the very small to the very large.

OpenGL

OpenGL is an API for 2D and 3D graphics on any platform which is implemented by all modern GPUs.

Rhinocerous 3D

Rhinocerous 3D is a commercial NURBS 3D Modeling tool for Windows and OSX.

Mediated Reality

Video Production

See: Visual Composition, Screen Captures

Cinelerra

CinePaint

CinePaint is an Open Source tool for hand-painting video frames.

FFmpeg

FFmpeg is an audio/video transcoding library with support for very many codecs and a great command line interface (ffmpeg).

  • FFmpeg can be used to rescale audio/video to a different resolution
  • FFmpeg can be used to (re-)encode audio/video

FLV

FLV (“Flash Video”) is a media file container format for sharing media; especially over HTTP.

  • FLV can be downloaded in segments (is “seekable”)
  • FLV contains MPEG-4 encoded content
  • FLV supports H.264 MPEG-4 video
  • Most Tube Sites (natively) support FLV

HandBrake

HandBrake is an offline batch GUI audio/video transcoding tool.

  • HandBrake can use FFmpeg for transcoding
  • HandBrake helps with rescaling a video for e.g. mobile devices with limited bandwidth ahead-of-time

H.264

H.264 (or MPEG-4 AVC) is a widely implemented video compression codec.

  • Blu-ray discs are encoded with H.264
  • HDTV, digital cable, and digital satellite are encoded with H.264
  • Hulu, iTunes, Vimeo, VEVO, and YouTube all encode video with H.264

Note

Some mobile devices include hardware support for H.264 decoding, which can save CPU time and battery, resulting in smoother playback.

LiVES

LiVES is an Open Source video editing and realtime VJing software tool written in C, Perl, and Python.

  • LiVES has full undo and redo (“nondestructive editing”)
  • LiVES has strong native support for networked streaming
  • LiVES has support for LADSPA, JACK, MIDI

Screenwriting

See: Small Business > Concept

Fountain.io

Fountain.io is a Lightweight Markup Language similar to Markdown with extensions for writing formatted screenplays with e.g. Scene Headings, Characters, Action (scene description; cues), Dialogue.

Scorewriting

MuseScore

MuseScore is an Open Source software tool for writing and engraving musical score compositions with support for MIDI, MusicXML written in C++ and Qt.

Rosegarden

Rosegarden is an Open Source digital audio workstation for score composition.

Musical Composition

ChucK

Chuck is an Open Source realtime, concurrent Musical Composition Audio Synthesis programming language.

  • ChucK supports “On the Fly Programming”: code and loops can be changed while the ChucK program is running
  • ChucK supports JACK
  • ChucK works with STK Synthesis Toolkit

Csound

Csound is an Open Source Musical Composition Audio Synthesis programming library API.

  • Csound works on Linux, OSX, Windows, Browsers (emscripten (Javascript)), Chrome Native Client (PNaCl), iOS, and Android
  • Csound supports MIDI
  • Csound ships with the OLPC XO laptops

Csound is simultaneously both ‘old school’ and ‘new school’.

http://booki.flossmanuals.net/csound/introduction/

Hydrogen

Hydrogen is an Open Source drum machine software with support for JACK, ALSA, MIDI, and drumkits.

VST

VST is a reusable audio transform API for Audio Synthesis and Sound Production tools.

  • A VST plugin is a packaged module of instruments, effects, and/or MIDI code
  • A VST host is a program which can run VST plugins
    • Ardour
    • Audacity
    • ACID, Adobe Premiere, Cakewalk, FL Studio, Sound Forge, Vegas

Sound Production

https://en.wikipedia.org/wiki/Comparison_of_free_software_for_audio

https://en.wikipedia.org/wiki/List_of_Linux_audio_software

https://en.wikipedia.org/wiki/Comparison_of_digital_audio_editors

https://en.wikipedia.org/wiki/Recording_studio

AAC

AAC is an audio compression standard designed to replace MP3.

AAC is a component of MPEG-4 audio (“MP4”).

  • AAC is a lossy audio compression algorithm
  • iTunes files are in AAC format (MPEG-4 Audio, MP4, M4a)

Ardour

Ardour is an Open Source Digital Audio Workstation (DAW) for multi-track recording, editing, and mixing.

  • Ardour supports JACK, MIDI sequencing (drum machines), AudioUnit, LADSPA, LV2, Mackie control protocol, OSC
  • Ardour can sync one video in the timeline.
  • Ardour integrates with external control surfaces (soundboard, mixing console, audio production surface).
  • Ardour is included in Ubuntu Studio.

FLAC

FLAC is an Open Source lossless audio codec.

  • FLAC files are, on average, like 10x (TODO ?) the file size of e.g. MP3, because they encode the full spectrum.

JACK

JACK is a low-latency (“realtime”) sound server for POSIX operating sytems (Linux, OSX, Windows) written in C and C++.

LADSPA

LADSPA (“Linux Audio Developer’s Simple Plugin API”) is an Open Source standard for audio filters and effects written in C.

Mixxx

Mixxx is an Open Source sound recording, mixing, editing, beatmatching, and production software tool with MIDI and HID DJ controller support (for use with keyboards, analog turntables, CD players) written in C++, Qt, XML, and Javascript.

  • Mixx has dual and quad decks with scratchable waveforms and beatmatching indicators (“cue points”), time stretching, beat looping, EQ, crossfading
  • Mixx supports timecoded vinyl and CDs

MP3

MP3 refers to one of two MPEG audio codecs.

  • MP3 can be around a tenth the size of a raw WAV file
  • MP3 can be CBR (“Constant Bitrate”) or VBR (“Variable Bitrate”) depending on the encoding parameters
  • MP3 is a lossy audio compression algorithm
  • Amazon Music (Amazon MP3) files are in MP3 format

See also: AAC, FLAC, Ogg, SHN

PulseAudio

PulseAudio is an Open Source realtime sound server.

  • How can I use PulseAudio to stream music from my main PC to my LAN with multiple PCs with speakers?
  • How can I use PulseAudio to share a single LINE-IN/MIC jack on the entire LAN?
  • How can I use PulseAudio as an RTP based N:N multicast conferencing solution for the LAN?
  • PulseAudio runs on FreeBSD, NetBSD, OpenBSD, Linux, Solaris, OSX, and Windows
  • There are many PulseAudio modules for the PulseAudio ModuleAPI http://www.freedesktop.org/wiki/Software/PulseAudio/Documentation/User/Modules/
    • module-native-protocol-unix, module-native-protocol-tcp – PulseAudio native protocol
    • module-rtp-send, module-rtp-recv – RTP
    • module-raop-discover, module-raop-sink – AirPlay
    • module-jack-sink, module-jack-source, module-jackdbus-detect – JACK
    • module-loopback, module-echo-cancel, module-equalizer-sink, module-ladspa-sink LADSPA, module-switch-on-connect, module-rescue-streams, module-device-restore, module-stream-restore, module-card-restore – sink, source, filters
    • module-pipe-sink, module-pipe-source, module-always-sink, module-null-sink
    • module-zeroconf-publish, module-zeroconf-discover – ZeroConf / Bonjour peer discovery (local multicast LAN/WLAN segment)
    • module-bluetooth-discover, module-bluetooth-policy, module-bluetooth-proximity – Bluetooth A2DP audio peer discovery
    • module-lirc – Infrared remote volume control
    • module-cli – pactl commandline interface
  • There are a number of volume control and mixer apps for PulseAudio.
  • pactl – PulseAudio control CLI utility
  • pacmd – PulseAudio runtime reconfiguration utility
  • pacatcat for PulseAudio servers (play (un)encoded audio)
  • pavucontrol – PulseAudio taskbar widget (Gnome)
  • paprefs – PulseAudio preferences config GUI (Gnome)
  • pax11publish – PulseAudio / X11 GUI synchronization
man pulse-cli-syntax
man pactl
pactl info
pactl list
pactl load-module module-native-protocol-tcp
pacmd load-module module-native-protocol-tcp

A Bash script to supports pushing all system audio from one machine to another (e.g. speakers connected to a PC or a Pi on the other side of the room):

WAV

WAV (“Waveform Audio Format”) is a file format for storing an uncompressed audio bitstream.

  • Like WAV, Audio CDs are encoded with PCM; but unlike WAV, Audio CDs are encoded with PCM and Red Book audio

Audio/Visual Production

Advanced Visualization Studio

Advanced Visualization Studio is a GUI tool for developing music visualization presets for the Winamp media player.

Geiss

Geiss is a Winamp plugin for music visualization.

Windows 95/98/ME/2000/XP/Vista, DirectX 3.0 or later, and a 200 MHz or faster processor. You can also run Geiss on Linux, using Wine.

To use the plugin, you must have Winamp... if you don’t already use Winamp, just download the screensaver version of Geiss.

ProjectM

Source: git git://git.code.sf.net/p/projectm/code

ProjectM is an Open Source implementation of MilkDrop with OpenGL.

  • MilkDrop Presets work with ProjectM
  • ProjectM is available for Linux, Unix
  • ProjectM is available for iOS and Android

Resolume

Resolume Arena and Avenue are realtime DVJ audiovisual production tools for live visuals.

  • Resolume Arena is designed for multi-projector setups (“projection mapping”, “projection blending”)

vvvv

vvvv (“v4”) is a realtime audiovisual production tool for Windows written in Delphi and C# .NET.

Web Production

M3U

M3U is an audio playlist format.

  • An M3U file includes unqualified file URLs (e.g. C:/music/file.mp3 or /srv/music/file.mp3)
  • An M3U file incldues HTTP URLs (e.g. ``
  • VLC can playback (read) M3U playlists.

PLS

PLS is an audio playlist format.

  • ShoutCAST / IceCAST streams are shared in PLS format.
  • VLC can playback (read) PLS playlists.

Popcorn Maker

Popcorn Maker is an Open Source tool for creating multitrack web media presentations.

Stream Annotation

Media codecs do not solve for commenting, other tools support commenting on points or ranges of timecodes (e.g. “this part is great!” or “who is that?”)

OpenAnnotation

OpenAnnotation is an Open Source RDF standard for threaded commenting on things with URLs and URIs.

OpenAnnotation annotations (comments with metadata) can be stored on (synced with) a central server, generally under a key, URN, or URI:

<script async defer src="//hypothes.is/embed.js"></script>
<!-- show comments by default -->
<script>window.hypothesisConfig=function(){return{showHighlights:true};</script>

owl:sameAs: Knowledge Engineering > Web Standards > OA

XSPF

XSPF (XML Shareable Playlist Format) is an Open Source media playlist format for files with URIs, URNs, and URLs.

  • XSPF understands file://-schema-prefixed local paths.
  • XSPF understands http://-prefixed remote paths.
  • XSPF is designed to be utilized with a content resolver; which CAN or COULD lookup alternate representations of resources from a given set of resource collections.
  • “What a playlist is not” http://xspf.org/xspf-v1.html#rfc.section.3.2
  • XSPF solves for all of the use cases of M3U and PLS.
  • VLC can playback (read) XSPF playlists.

What cannot be done with XSPF?

  • Cue points, crossfading, gap times
  • HTML content schema.org/description _HTML
  • schema.org/CreativeWork

JSPF

JSPF (JSON Shareable Playlist Format) is an Open Source media playlist format for files with URIs, URNs, and URLs.

Web Distribution

CDN

A CDN (“Content Delivery Network”) is a service and infrastructure designed for serving Web Content efficiently and geodistributedly.

CDN Services:

CDN APIs

BitTorrent

BitTorrent is a file distribution protocol built on top of HTTP.

  • A BitTorrent client downloads pieces of files over HTTP from peers discovered through a tracker or another peer discovery protocol (such as a DHT).

  • BitTorrent magnet URIs open in an installed BitTorrent downloading program (e.g. BitTorrent, uTorrent, Transmission (Gnome), Deluge (Gnome))

  • There are programs to retrieve the latest .torrent files from RSS and Atom feeds (e.g. a released [episode] of a series or topic)

  • BitTorrent can be run headlessly (as a server daemon) e.g. with an HTTP API, SMTP pattern, web interface, or SSH for enqueuing torrents to download to a central location.

  • BitTorrent does not preserve file permissions or extended attributes, but archives can.

  • For a time, BitTorrent accounted for the largest slice of web traffic; but “now accounts for only 6.3% of total traffic in North America”.

    https://www.sandvine.com/trends/global-internet-phenomena/

BitTorrent Torrent File

A BitTorrent .torrent file includes:

  • SHA-1 checksum hashes of all the pieces of all the files
  • (optional) a DHT key (for trackerless peer discovery)
  • (optional) a list of HTTP BitTorrent Tracker URLs
  • (optional) a list of HTTP Web Seeds.

https://en.wikipedia.org/wiki/Torrent_file

BitTorrent Tracker
A BitTorrent Tracker maintains a list of recently connected peers from which matching pieces can be downloaded.
BitTorrent Client
A BitTorrent Client opens a BitTorrent Torrent File, registers with the given BitTorrent Trackers, looks for peers in a DHT, downloads the pieces of the files listed by choosing faster (more efficient, less costly, ~more local) peers to download from, and, according to the configuration, uploads pieces to other peers.
BitTorrent Seed
A BitTorrent Seed is a complete, checksummed copy of all of the pieces of all the files in a BitTorrent Torrent File which a peer is sharing through a BitTorrent tracker or a DHT.
Web Seeding

Web Seeding is when one or more HTTP servers serve default standby seeds (thus ensuring Availability when no-one is online).

https://en.wikipedia.org/wiki/BitTorrent#Web_seeding

Use Cases:

Note

“Scrambled” DRM content (which may or may not just work when I just copy the file to my other device) can be shared over BitTorrent all the same.

debtorrent

Source: svn svn://svn.debian.org/debtorrent/debtorrent/trunk

debtorrent downloads APT DEB packages with BitTorrent for Linux distribtions like Ubuntu and/or Debian.

  • debtorrent can efficiently transfer packages to many local or remote machines (instead of downloading the same file from the other side of the network multiple times)
sudo apt-get install debtorrent
echo 'deb debtorrent://localhost:9988/foo distro main' | \
    sudo tee -a /etc/apt/sources.list

webtorrent

webtorrent is a BitTorrent client written in Javascript that can run in browsers with WebRTC support or node.js.

  • With nodejs, webtorrent can stream to AirPlay, Chromecast, VLC
  • webtorrent can work with HTML5 <video> tags

Steam

Steam is a game and game content distribution system.

  • Steam is part of SteamOS
  • Steam registers DRM licenses with Steam for downloading files
  • Steam does not yet download files from more local peers (e.g. everyone at the LAN party purchases the game)

Kerbal Space Program

Kerbel Space Program is a space flight simulator game.

KERBAL SPACE PROGRAM IS A MULTI-GENRE GAME WHERE THE PLAYERS CREATE THEIR OWN SPACE PROGRAM.

In KSP, you must build a space-worthy craft, capable of flying its crew out into space, without killing them. At your disposal is a collection of parts, which must be assembled to create a functional ship. Each part has its own function and will affect the way a ship flies (or doesn’t).

Tube Sites

Fueled by the dramatic growth of YouTube, the term “Tube Site” refers to any of a number of sites serving (seekable) video; initially with Flash Video FLV and now with HTML5 and HTTP Live Streaming and MPEG-DASH.

Features

  • GIF thumbnails
  • seekable thumbnails
  • download links
  • download for my device links
  • threaded comments, reviews, upvotes, downvotes
  • Stream Annotation
  • playlists this item is contained in
  • featured playlists
  • related content

HTTP Live Streaming

HTTP Live Streaming (HLS) is a draft IETF standard for streaming playlists of audio and video at various resolutions over HTTP.

MPEG-DASH

MPEG-DASH is an ISO standard for streaming audio and video content with adaptive bitrates over HTTP.

  • MPEG-DASH downloads and buffers file segments.
  • MPEG-DASH selects from a number of possible resolutions according to available bandwidth and processing capacity.
  • MPEG-DASH specifies optional HTML5 Encrypted Media Extensions DRM
  • MPEG-DASH segments may contain MPEG-2 files in an MP4 container or other combinations of codecs and containers
  • It currently costs $265 USD to download the MPEG-DASH standard, which is not necessary to enjoy MPEG-DASH adaptability.
  • nginx-rtmp-module can host MPEG-DASH streams.
  • VLC can play MPEG-DASH streams.
  • Chromecast, YouTube, NetFlix support HTML5 MPEG-DASH

nginx-rtmp-module

nginx-rtmp-module is an Open Source module for the nginx web server which implements RTMP, HTTP Live Streaming, and MPEG-DASH for hosting streaming audio and video written in C.

  • nginx-rtmp-module uses FFmpeg for online audio/video transcoding
  • nginx-rtmp-module can record streams to seekable Flash Video FLV files
  • nginx-rtmp-module works with H.264, AAC, MPEG-DASH audio / video compression / decompression codecs

Video Chat

https://en.wikipedia.org/wiki/Videoconferencing

https://en.wikipedia.org/wiki/Videotelephony

Video chat: what might’ve been an exercise in Collaboration Engineering:

Models for interactive video chat:

  • Video Relay Server (RTMP, nginx-rtmp-module)
    • Clients capture, encode, and push video to server
    • Server [re]encodes stream and pushes video to other client(s)
  • Browser-to-Browser (WebRTC)
    • Clients connect in a mesh and relay data streams
    • Clients can be browsers or hosted applications (e.g. SaaS)
    • WebRTC is also useful for centralized Web Distribution

Media Tools

http://distrowatch.com/search.php?category=Multimedia

DreamStudio

Celeum DreamStudio is a distribution of Ubuntu Linux with lots of great packages and a kernel tuned for media production.

  • DreamStudio includes software with a diverse set of software licenses.

Ubuntu Studio

Ubuntu Studio is a distribution of Ubuntu Linux with lots of great packages and a kernel tuned for media production.

  • Ubuntu Studio includes software with a diverse set of all Free and Open Source software licenses.

Home A/V

Standards

DIAL

DIAL (Discovery and Launch) is a standard for discovering media devices and launching media resources built upon SSDP, UPnP, and HTTP.

  • A DIAL Server (a “1st screen device”; e.g. a TV or streaming device) receives DIAL requests from a DIAL client.
  • A DIAL Client (a “2nd screen device”; e.g. a phone or a PC) sends DIAL requests to a DIAL server.
  • DIAL Clients discover DIAL devices with DIAL Service Discovery.
  • DIAL Clients interact with DIAL devices over a DIAL REST Service API.

DLNA

DLNA: Digital Living Network Alliance
DLNA is a standards group and a standard for networked devices; particularly audio/video media devices like standalone media players and networked stereos.

Google Cast SDK

Google Cast SDK powers Chromecast devices.

  • A Cast sender application pushes a URI to a Cast receiver application.
  • The Cast receiver application then retrieves the resource/app identified by the URI; saving the battery of the Cast sender application.
    • Bluetooth Audio (A2DP, ) and Chromecast full device audio/video cast require audio [recompression and] transmission, which drains a mobile device battery.

SSDP

SSDP (Simple Service Discovery Protocol) is a networking protocol for discovering services over HTTPU (HTTP over UDP) unicast and multicast; without DNS or DHCP.

Zeroconf

Zeroconf/Bonjour are standards for zero-configuration networking; wherein networked devices assume local IP addresses if necessary, and broadcast their capabilities over multicast DNS.

  • Devices must be on the same LAN/WLAN segment (!?)
    • [ ] Otherwise forwarding / ebtables ?
      • [ ] pulseaudio-dlna

10-foot interfaces

Gaming Consoles

[...]

  • Steam Machine (SteamOS)
  • NVIDIA Shield (AndroidTV)
  • XBox
    • XBMC (now Kodi) started out as a an application for modded XBoxen way back in 2002.
  • XBox 360 (joysticks tied to visualizations)
  • XBox One
  • PS3
  • PS4

Standalone Media Devices

Pros:

  • Upgradeable
  • Interchangeable

Cons:

  • Extra HDMI, Power
  • Remote Control Integration
Android TV
  • Android TV is originally based on Android 5.0 (“Lollipop”)
  • Android TV supports the Chromecast API
  • Android TV supersedes Google TV (which was also Android-based)
Chromecast
  • 1: 2013, 2: 2015
  • In: Power: MicroUSB
  • Configure WiFI w/ Android/iOS device and Chromecast app
  • Cast content from Android, computer (Chrome, ChromeOS), iOS to Chromecast
  • Out: HDMI
Chromecast Audio

Chromecast Audio turns just about any stereo with an AUX input into a web-connected audio player.

  • In: Power: MicroUSB
  • Configure WiFI w/ Android/iOS device and Chromecast app
  • Cast content from Android, computer (Chrome, ChromeOS), iOS to Chromecast Audio
  • Out: 3.5mm Headphone Jack
  • Out: Optical Audio Out
Apple TV
  • 1: 2007, 2: 2010, 3: 2012/2013, 4: 2015
  • Extremely simple remote
  • Supports Bluetooth Keyboards, iOS devices
Kodi

Kodi (formerly XBMC) is a “10 foot” media playback application designed for use with a remote.

Kodi addons are written in Python and XML.

  • Kodi runs on Windows
  • Kodi runs on OSX
  • Kodi runs on Linux
  • Kodi runs on Android
  • Kodi runs on iOS
  • Kodi runs on Apple TV
  • Kodi runs on FreeBSD
  • Kodi runs on ARM, MIPS, PowerPC, x86, x86-64
  • Kodi supports MilkDrop audio visualizations.
MythTV

MythTV (2002) is a networked DVR server and protocol standard with a number of client implementations.

  • MythTV supports [remote] tuner cards
  • MythTV supports TV listings
  • MythTV supports IR remotes, timeshifting, Linux, Windows
Plex
Plex Media Server
Plex Media Server is a free app which runs on a computer or a NAS device and serves media to Plex client apps.
Plex Home Theater
Plex Home Theater is a free 10-foot interface client app for a computer attached to a TV (e.g. with VGA, DVI, or HDMI).
Plex Pass

Plex Pass is an optional premium subscription service upgrade.

https://plex.tv/subscription/about

  • Plex is/was originally a fork of Kodi
  • Plex works with Windows
  • Plex works with Windows Phone
  • Plex works with OSX
  • Plex works with Linux
  • Plex works with Android
  • Plex works with Chromecast
  • Plex works with FreeBSD
  • Plex works with iOS
  • Plex works with Apple TV
  • Plex works with Xbox 360
  • Plex works with Xbox One
  • Plex works with PS3
  • Plex works with PS4
  • Plex works with Roku
Roku
  • Through the Roku Channel Store, Roku supports Google Play, Netflix, PBS, HBO Go, HBO Now, Epix, Showtime, and a number of other channels:
  • Roku supports Amazon Instant VOD
  • Roku apps are written in a Roku-specific language called “BrightScript”
  • Roku OS is derived from Linux
  • The Roku remotes are all very simple and easy.
  • The Roku 3 remote has a headphone jack for private listening (Wi-Fi Direct)