General Operation

Saving and Loading: File->Save and File->Load.

Hooking up a display to the projector: First, make sure the desktop is
set up so that projector counts as a second monitor onto which the 
desktop is extended. Next, go to Links->Projector Output and select
the display you wish to project.  This should create a window which
is a copy of the display; drag this window to the projector's area 
of the desktop and then maximize the window.


Delay Loops

The only type of loop that can be made without crashing the program is
a delay loop.  A delay loop won't crash the program because the delay
does not fetch its input when its output is requested, so no infinite
recursion occurs.

Delay loops have some special considerations, however:

1) For correct behavior, the 'output' of the loop must come from the
delay itself.

2) Since the loop's input is a function of its own output, the dimensions
of the loop's image are initially undefined, and so the loop must be 
seeded with an image of the correct size.

Creating a delay loop (tutorial)

Suppose we want to create a loop which continually blurs an image, compositing
the thresholded camera image into the loop.

First, open Base.ser which will already have camera calibration, noise/background
subtraction, and thresholding.

Now, add a delay filter, a blur filter, a composite filter, and a three-way switch.

First we make the loop body: connect the output of the composite to the blur, and
the output of the blur to the delay.

Now, we connect the camera input: connect the threshold filter output to the composite
filter inputA, and then also connect the same threshold output to the composite filter
mask input.

All that remains to do is actually connect the loop, but for convenience we will connect
the loop in such a way that it can be quickly seeded via a three-way switch.

Connect the new three-way switch's output to the composite filter's B input. Connect the
threshold's output to the new three-way switch's 0 input, and connect the delay's output
to the new three-way switch's 1 input.  Finally, connect the loop to the display by
connecting the delay output to three-way switch connected to the display.

Flip the display switch so it is showing the output of the delay: since the loop is not
connected, it should just display the threshold output. Set the delay to 1 frame of delay
(far left), and let at least 16 frames pass to make sure all the frames in the delay's buffer
are the correct size.  Finally, switch your new three-way switch's position to position 1,
which should connect the loop.


Troubleshooting

**The console keeps on printing "Null tt"

Null tt means that some transform-based filter has not created its Transform Table:
just move the sliders or hit the "make transform" buttons on all your transform
filters to force them to create their tables.  The combined calibration filter will
always give this error when it is first loaded: the safest (in terms of not disturbing
settings) way to get rid of it is to click to the lens distortion tab and then "make
transform".

**The camera input says "unable to create player" and the console keeps on printing
"player is null"

Do you have more than one camera input? The system only supports one camera input open
at a time.

Did you load a filter set on top of an already open filter set? If you do this without
first closing the old filter set's camera input, the camera input won't be garbage collected
in time to free the camera input slot for the new input filter. Just restart the filter desktop
and reload the filter set. Alternately, when loading filter sets on top of old ones, make
sure to close the camera input first.

**The output has frozen even though I'm still able to interact with the interface and the
camera is working properly.

The image processing thread has probably crashed, most likely due to an accidentally created
loop.  Make sure there aren't any non-delay loops in the graph, and then save the graph. Restart
the filter desktop and then load your graph back; if you sucessfully eliminated all the loops,
it shouldn't crash again.  If you are absolutely sure that you eliminated all the loops and 
it's still crashing, you have probably found a bug in one of the filters; the console will
usually indicate which filter caused the crash.

**The thresholded image is calibrated correctly, but when I switch over to the output of
a delay loop, it is seriously out of alignment.

The delay loop has probably been seeded to a smaller size than the thresholded image:
reseed the loop (usually by just switching the loop's three-way switch to the zero position
for a second and then switching it back).

**My delay loop is running extremely slowly

The delay might be set to more than one frame: for instance, if it is set to two frames,
then the loop will only appear to update half as often as the main update rate.  Otherwise,
you might just have too many expensive filters running at once.

**My delay loop is flickering/acting strange

Make sure that the ONLY OUTPUT ANYWHERE from the delay loop is from the delay itself: even
something as innocuous as hooking up a display to a different filter in the loop can result
in undesirable behavior.

**I'm getting weird graphical glitches when moving filters over the camera input filter

The camera input filter does not play well with other filters, so avoid having it on top
of or below other filters.  Move all the other filters out from underneath/on top of it
and then minimize and restore the filter desktop window to clear the artifacts.

**I'm getting artifacts with the arrows

That happens.  Just move a window to redraw all the arrows, or just minimize and restore the
filter desktop window.

**Java ran out of memory

Try giving it more with the command line argument -Xmx256M (to give it up to 256 megabytes)
or -Xmx512M (to give it up to 512MB). Otherwise, try reducing the resolution of the image
stream: each filter has to keep an internal buffer, so going from (for instance) 640x480 
to 320x240 will cut the memory usage significantly.


Filters

Input

CameraInput (CameraInputFilter.java):
Purpose: Provides a video stream from a webcam
Inputs: None
Outputs: ImageOut
Settings: Resolution button allows camera capture resolution to be changed. Capture button does nothing.
Persistence: Camera resolution settings are not saved.
Warnings: Only one CameraInput can work at a time; if additional CameraInputs are created, they will not work. Attempting
to load a saved file without closing the existing camera input will result in the loaded camera input not functioning.


ImageLoader (ImageLoaderFilter.java):
Purpose: Provides a video stream of a static image
Inputs: None
Outputs: ImageOut
Settings: The LoadImage button is used to select the image to be loaded.
Persistence: The loaded image is saved with the filter.
Warnings: Linking an image loader without a loaded image occasionally causes the system to crash; to be safe, always load in
some image after creating an image loader.

MovieLoader (MovieInputFilter.java):
Purpose: Provides an image stream from a movie
Inputs: None
Outputs: ImageOut
Settings: Load is used to select the movie to be loaded (.avi; must have appropriate codecs installed), capture does nothing.
Upon loading, standard movie player controls can be used to control movie playback.
Persistence: Loaded movie is not saved with filter; must be reloaded each time.
Warnings: Movies do not automatically loop.

MultiImageLoader (MultiImageLoaderFilter.java):
Purpose: Provides a video stream from a series of still frames.
Inputs: None
Outputs: ImageOut
Settings: Start and End text boxes are used to enter the starting and ending frame numbers.
Prefix is the front part of the frame filenames (eg: C:\anim\frame), suffix is the ending
(eg:.png).
Warnings: Don't use this for real time applications, as it is very slow. Make your frames
into an .avi instead, and load the movie with the movie loader. If you need loopings, just
make a longer .avi.

Output

Display (DisplayFilter.java):
Purpose: Displays a video stream (pulls)
Inputs: ImageIn
Outputs: None
Settings: Video is stretched to fit window
Warnings: None

Effect

BaseFilter (EffectFilter.java):
Purpose: None; base class for all filters

Three-Way Switch (ThreeWaySwitchFilter.java):
Purpose: GUI element for switching between three different inputs; slider controls
which of the three inputs is routed to the one output.
Inputs: 3 ImageIns
Outputs: ImageOut
Settings: Slider position determines routing: To far left, output is input 0, in middle,
output is input 1, and at right, output is input 2.
Persistence: Slider position is saved.
Warnings: Be careful to avoid accidentally creating non-delayed loops.

ThresholdA (ThresholdAFilter.java):
Purpose: Thresholds an image; that is, all pixels where the sum of the r, g, and b values
is larger than the threshold level are set to pure white (1.0,1.0,1.0), and all other pixels
are set to pure black (-1.0,-1.0,-1.0).
Inputs: ImageIn
Outputs: ImageOut
Settings: Threshold level slider; far left is a threshold of -1.0, far right is a threshold
of 1.0.
Persistence: Threshold level is saved.
Warnings: None

Distortion (DistortionFilter.java)
Purpose: Perturbs pixels in the input image according to values in two distortion maps. Call the
distortionXIn image dx[x,y] and the distortionYIn dy[x,y] and the input image i[x,y], and the
distortion level D, the for each pixel x,y in the output image o[x,y],
o[x,y] = i[x + dx[x,y]*D, y + dy[x,y]*D].
Inputs: ImageIn, DistortionXIn, DistortionYIn
Outputs: ImageOut
Settings: Distortion level
Persistence: Distortion level is saved
Warnings: None

Skylight (SkylightFilter.java):
Purpose: Used to mimic the effect of light coming from above and casting shadows.  In particular,
skylight takes in a thresholded image, and starting with the top pixel of each column, it scans down
until it finds a white pixel in the input; in the output, all pixels above the found white pixel become
white, and all pixels below it become black.
Inputs: ImageIn (should be thresholded)
Outputs: ImageOut (a thresholded image in which black is shadows and white is light)
Settings: None
Persistence: N/A
Warnings: None

RandomColor (RandomColorFilter.java):
Purpose: Its output is a solid color image in the dimensions of the sizeIn input, where the output
color is the color of a randomly chosen pixel in the colorsIn input.
Inputs: Colors in, size in
Outputs: ImageOut
Settings: None
Persistence: N/A
Warnings: None

Chains (Chainer.java):
Purpose: Simulates chains hanging from the ceiling which are attracted to peoples' outlines.
Inputs: ImageIn- thresholded image of objects for the chains to be attracted to
Outputs: ImageOut
Settings: None
Persistence: N/A
Warnings: None

Outline Grow (OutlineGrowFilter.java):
Purpose: Creates concentric outlines around objects
Inputs: ImageIn- thresholded image of objects to outline
Outputs: ImageOut
Settings: None
Persistence: N/A
Warnings: None

Lens Correction (LensCorrectionFilter.java):
Purpose: Corrects radial (barrel or pincushion) distortion in an image
Inputs: ImageIn
Outputs: ImageOut
Settings: The parameters a,b,c, and d correspond to coefficients in a polynomial
that specifies the radial perturbation at a given radius. Changes in the text
boxes will not take effect until the 'make transform' button is pressed.
Persistence: Parameters are saved.
Warnings: CombinedCalibration is usually a better choice than this filter, since
it includes this filter's functionality.

Shrinker (ShrinkFilter.java):
Purpose: Shrinks the input by a factor of two to produce the output (eg: if the
input is 320x240, then the output is 160x120).
Inputs: ImageIn
Outputs: ImageOut
Settings: None
Persistence: N/A
Warnings: None

Inversion (InvertFilter.java):
Purpose: Inverts an image stream 
(that is, for all x,y output[x,y] = -1.0 * input[x,y]).
Inputs: ImageIn
Outputs: ImageOut
Settings: None
Persistence: N/A
Warnings: None

Upscale (UpscaleFilter.java):
Purpose: Doubles the dimensions of an image (eg: an input of 320x240 results
in an output of 640x480).
Inputs: ImageIn
Outputs: ImageOut
Settings: None
Persistence: N/A
Warnings: Doubling the dimensions increases the number of pixels by a factor
of four; this could result in a serious speed decrease.

Erosion (ErosionFilter.java):
Purpose: 'Erodes' a thresholded image by 1 pixel. Technically, every pixel for which
any of its eight neighbors are black becomes black, and only pixels that are already
white and which have eight white neighbors stay white.
Inputs: ImageIn - a thresholded image
Outputs: ImageOut
Settings: None
Persistence: N/A
Warnings: None

ConnectedComponents (ConnectedComponentsFilter.java): 
Purpose: Finds connected components in a 
thresholded image and randomly colors them.
Inputs: ImageIn - thresholded
Outputs: ImageOut
Settings: None
Persistence: N/A
Warnings: Not very useful.

RemoveSmallComponents (RemoveSmallComponentsFilter.java):
Purpose: Fills in components in BWImageIn below a threshold size with the (BW) data
from CorrectionIn.
Inputs: BWImageIn- image from which to remove small components, CorrectionIn- image data
to fill removed components with.
Outputs: ImageOut
Settings: Minimum Component Size- determines the threshold size below which components are
removed.
Persistence: Threshold size is saved.
Warnings: None.

BodySegmentation (BodySegmentationFilter.java):
Purpose: Attempts to identify body parts of humans in the image. Found heads are colored
pure green, torsos pure red, and non-head extremities (legs, arms, hands, feet) are colored
pure blue.
Inputs: ImageIn - a thresholded image
Outputs: ImageOut
Settings: Salience threshold- The salience of a body part is the ratio of its perimiter along
the outline of the silhouette to the length of the cut that made the body part; increasing
the salience threshold will therefore mean that less protruding parts of the silhouette will
not be identified as parts. Max cut length- places an upper limit on how long cuts can be; can
be used to discourage extremely long cuts. Critical point curvature threshold- determines how
sensitive the algorithm is when attempting to find cut start position (lower is more sensitive).
Persistence: All setting are saved.
Warnings: None

Weighted Average [Average] (AvgFilter.java):
Purpose: Blends two input images together.  Let the average weighting be called p (0.0 <= p <= 1.0),
then for all x, and all y, output[x,y] = p * input0[x,y] + (1.0 - p) * input1[x,y].
Inputs: InputA, InputB
Outputs: ImageOut
Settings: Slider controls average weighting- at left, p = 0.0, at right, p = 1.0.
Persistence: Everything is saved.
Warnings: None.

Composite (CompositeFilter.java):
Purpose: Composites two images together based on a black and white mask image.  Where the mask
is white, the output is the same as inputA, and where the mask is black, the output is the same
as inputB; values inbetween black and white blend the two inputs together.
Inputs: InputA, InputB, maskInput (BW)
Outputs: ImageOut
Settings: None
Persistence: N/A
Warnings: None

Center of Mass (CMFilter.java):
Purpose: Calculates the center of mass of a thresholded image (the center of mass
of the white pixels)
Inputs: ImageIn (BW)
Outputs: DataOut (CM coordinates)
Settings: Continually displays the coordinates of the calculated CM.
Persistence: N/A
Warnings: The data out is data stored in an image, and so won't appear correctly if displayed.

ImageMover [ImagePositioning] (ImagePositioningFilter.java):
Purpose: Positions the image from ImageIn at the coordinates supplied by CoordinatesIn on top of
an all black image with the dimensions of SizeIn.
Inputs: ImageIn, CoordinatesIn, SizeIn
Outputs: ImageOut
Settings: None
Persistence: N/A
Warnings: Don't link a non-coordinate data stream to the data in unless you want strange behavior.

BWMin (BWMinFilter.java):
Purpose: Converts a color image to BW, where each pixel's BW value is simply the smallest of the
r, g, and b channels' values.
Inputs: ImageIn
Outputs: ImageOut
Settings: None
Persistence: N/A
Warnings: None

SimpleFire (SimpleFireFilter.java):
Purpose: Produces a decent fire effect. The cooling image determines how fast the fire cools (darker is
slower cooling and produces longer flames). The colors image is a mapping from fire temperature (left to right
is hot to cold) to color (image color at (temperature, 0)). The HotSpotIn determines where the fire originates
from (white pixels in the HotSpotIn image).
Inputs: CoolingIn, ColorsIn, HotSpotsIn
Outputs: ImageOut
Settings: None (load cooling and load colors must be pressed to actually load in those maps).
Persistence: Color map and cooling map are not saved, so the buttons must be pressed every time
the filter is loaded.
Warnings: None.

Cycler (SimpleCyclerFilter.java):
Purpose: Pixels are set to 'new' according to the hotspots in, and they age at either a fixed
rate (default), or a rate set by the cycling map, if one is loaded from cyclingIn.  The age of
each pixel is mapped to a color according to colorsIn.
Inputs: HotSpotsIn, ColorsIn, CyclingIn
Outputs: ImageOut
Settings: None
Persistence: The 'load colors' and 'load cycling' buttons must be pressed each time the filter
is loaded.
Warnings: None.

Delay (DelayFilter.java):
Purpose: The delay waits a specified number of frames before it returns the current input as its output.
More specifically, if the input to the delay 
is given as a function of the frame number as I[f], then 
the output of the delay O[f] = I[f-D], where D is the number of frames to delay. 
Inputs: ImageIn
Outputs: ImageOut
Settings: Delay amount- far left is 1 frame, far right is 16.
Persistence: Delay amount is saved, as are all the buffered frames, so that the delay does not
have to be reseeded every time it is loaded.
Warnings: If used outside of a loop, none. If used within a loop, see the section on delay loops.

Noise Reduction Filter (NoiseReductionFilter.java):
Purpose: Noise reduction, background subtraction. Note that the filter only finds pixels where
the current pixel is lighter than the same pixel in the noise image.
Inputs: ImageIn
Outputs: ImageOut
Settings: The 'Capture Noise' button is used to capture the noise/background image.
Persistence: The noise frame is saved.
Warnings: None.

Combined Calibration (CombinedCalibrationFilter.java):
Purpose: Combines translation, scaling, rotation, and lens correction of the input. Used to
align the projected image with the image captured from the camera.
Inputs: ImageIn
Outputs: ImageOut
Settings: Too many to list.
Persistence: All settings are saved, but the actual transform table is not, so after loading the
filter the transform table will have to be remade by either 1) moving a slider, 2) clicking the
'make transform' button in the lens correction tab.
Warnings: None.

Projector Transform, Manual Calibration, and Manual Rotation: Made obsolete by 
Combined Calibration.

Sine (SineFilter.java):
Purpose: Each output pixel is simply the sine of the corresponding input pixel.
Inputs: ImageIn
Outputs: ImageOut
Settings: Wavelength- determines the distance between wave crests; frequency- determines
how many wave crests will pass a point in a period of time.
Persistence: Frequency and wavelength are saved.
Warnings: Sine is very slow- it is perhaps the slowest filter.

Blur (BlurFilter.java):
Purpose: Blurs an image.
Inputs: ImageIn
Outputs: ImageOut
Settings: Blur level- the radius of blurring (from 1 to 6).
Persistence: The blur level is saved.
Warnings: The border, with a radius of the blur level, is not blurred.

Multiply (MultiplyFilter.java):
Purpose: Multiplies two images together. Note that the value of (0.0,0.0,0.0)
corresponds to gray rather than black, multiplying any image by a RGB (0.5,0.5,0.5)
gray image will result in the same gray image as output, and since black is (-1.0,-1.0,-1.0),
any image multiplied by a black image will be inverted.
Inputs: InputA, InputB
Outputs: ImageOut
Settings: None
Persistence: N/A
Warnings: None

ZMultiply (ZMultiplyFilter.java):
Purpose: Multiplies two images together as if black were (0.0,0.0,0.0) and white
were (1.0,1.0,1.0). This means that multipying any image by a black image results
in a black image as output, and multipying any image by a white image simply returns
that image. Note that this can be used to mask images: multipying an image by a 
black and white mask will result in the areas corresponding to white in the mask
remaining unchanged, while the areas corresponding to black in the mask become
black. For finer control, use the composite filter.
Inputs: InputA, InputB
Outputs: ImageOut
Settings: None
Persistence: N/A
Warnings: None.