Background Subtraction in the Projected Space

USB Connection

USBs must be connected in a very specific manner:

back: 1 keyb
23

front: 5
4

ImAcq & Detection

-Code:

• 5camdisplay.m - displays from 5 cameras
• MCrectangle.m - Multiple camera, bgnd subtraction, projection, draws rectangle around detected object.

Notes

"Starfield" Problem: Cameras and transform matrices dont always coincide.

Threshold value T is very important to compare to individual camera results. (for MCBSTransform)

The order of doing Background subtraction and Transform matters. Do Transform first, then BS.

The order the background is subtracted also matters. image-backgnd or backgnd-image.

Transform & Background Subtraction

Which should be done first? Background subtraction or Projective Transform?

-Code:
MCBSTransform.m- 5 cameras, background subtract then transform DOESNT WORK!
MCTransformBS.m- 5 cameras, transform then background subtract
transformbackgndsub.m - still image transform then background subtract
backgndsubtransform.m - still image background subtract, then transform

GLCM Gray Level CoOccurrence Matrix

-Code:

• blockimage.m - displays image by blocks, good for block processing
• blockGLCM.m - calculates GLCM features for each block in an image
• GLCMbackground.m - calculates GLCM features locally and compares them to the background features

Projective Transformations!!

Projective Transformations

-Code:

• transformimage5.m - takes still images from 4 cameras and applies a projective transformation. Returns an image on the ground plane, comparable to the top view camera.
  
  Note: calibration.m must be run beforehand in order to generate the transform.mat which contains the transform matrices.

GLCM

Gray Level CoOccurrance Matrix

-Code:

• graylevelcomatrix.m -plots Contrast, Correlation, Energy and Homogeneity for still images. Done for Camouflage and non-camouflage.
• RealTimeGLCM.m -plots Contrast, Correlation, Energy and Homogeneity for a single camera input.
• RealTimeGLCM5.m -plots Contrast, Correlation, Energy and Homogeneity for 5 cameras.

USB Bandwidth Issues Strike Again!

USB bandwidth issues were experienced using the desktop PC.

Solution:
Make sure the Logitech Cameras are connected to the two front USB ports.
Three other cameras can be connected in the back USB ports.

Centroids & Transformations

Code:

• MCDemo5.m -5 Camera image subtraction demo
• MCDemo5Calibration.m -5 camera image subtraction n transform using T matrix from calibration code
• MCDemoCentroid.m - 5 Camera image subtraction and centroid display
• Calibration.m - Code to select points and calibrate cameras. Use a checkerboard or any object and use the resulting transform.mat

Note: Image5 must be from the topview camera. All other images will be transformed to this plane of view.

• MultiCamDemo.m - 5 Camera image subtraction demo

ImAcq and Detection: 4 Cameras

Happy Bday!! :D

-Code:

• 4avidiffimproved.m - displays result of frame differencing on 4 RGB videos at once
• 4aviplay.m - displays 4 RGB videos on a single figure
• 4avidiffgray.m - displays result of frame differencing on 4 grayscale videos at once
• 4aviplaygray.m - displays 4 grayscale videos on a single figure
• 4avidiffbw.m -displays result of frame differencing and thresholding in B&W

ImAcq & Frame differencing

Image data was successfully acquired from 4 cameras with the scenario setup. Camouflage makes detection less obvious, but movement breaks the camouflage.

-Code:

• 4avidiff.m - does frame differencing on 4 avi files.
• 4camdisplay2avi.m - displays and stores data from 4 cameras to avi files.

Projective Transformation #2: Select Points

-Code:

• selectptstransform.m - calculates and applies a transform matrix by selecting correpsondence points from an image and a transformed version of it.

Projective Transformations

This will be necessary when changing from one camera view to another.

Code:

• transforms.m - translation, scale, shearing and rotation examples in matlab

ImAcq #5: Same Cameras

Using the same camera models allows the same frame rate and better synchronization.

-Code:

• 3camdisplay.m - streams video from 3 cameras to matlab figure
• 4camdisplay.m - streams video from 4 cameras to matlab figure
• 4camdisplay2mov.m - streams and saves video from 4 cameras to matlab movie struct
• 4camdisplay2avi.m - streams and saves video from 4 cams to avi

Kalman Filter

-Code:

• kalman.m - AVi detection, and tracking using a kalman filter

Detection

-Code:

• ball.avi - just a movie with a ball on a string
• detect.m - loads video, calls extractball and displays results
• extractball.m - frame differencing, bubble sort and finds centroids

Kalman Example

-Code:

• kalmanf.m - the function that executes the kalman filter update equations
• kalmanfilterexample1.m - example of measured observations and kalman filtering

Meeting Minutes

1. Email Dr. Rosiles regarding:
   USB clip camera & USB extension cables
   
2. Attempt Acquisition one camera per computer?
3. Meetings with Rosiles will be Tuesdays at 4pm.
4. Vision Networks discussions will be Fridays 1:30pm.

ImAcq #4: Synchronized

-Code:

• 3camtoAviver3.m - corrected synchronization problems with frame rates using 3 cameras added playback capability.
• playvideo3.m - plays back video from 3 avi files.

1. Logitech QuickCam 4000 have 15 fps, since Low Light boost and other features slow it down.
2. Logitech ClipCam has 30fps.

ImAcq #3: 4 Cameras

Video from 4 cameras was achieved by replacing the Intel CS110 with the Logitech clip-on Cam.
The bandwidth required by the Logitech Clip-On Cam is less than the Intel CS110.

-Code:

• 4caminput.m - displays input from 4 different cameras
• 4camtoAvi.m - saves input from 4 cameras to 4 Avi Files

Image Acquisition #2: Saving to an AVI file

- Code:

• 1camtoavi.m - records from 1 cam to an avi file
• 2camtoavi.m - records from 2 cam to an avi file
• 3camtoavi.m - records from 3 cam to an avi file
• 3camtoaviVer2.m - records from 3 cam to an avi file IMPROVED! Synched Cams
  
• frameratecalc.m - calculates frame rate for each camera
• timelapse.m - creates time lapse video

- Problem:

1. After extending the 1camtoavi to several cameras, it was found that the cameras were not synchronized. There were delays due to the code that started and stopped them.
2. The IBM PC Camera does not have the same frame rate as the others. So it would display faster moving video than the other cameras.

- Solution:

1. Vectorizing the start and stop functions helped to keep their Initial Trigger time to 1 second of difference.
2. By calculating the frame rate in each camera first and then doing the video capture based on the total video time, it was possible to keep the cameras at the same speed.

Image Acquisition

-Installed Image Acquisition Toolbox (IAT) for Matlab in the SPC Lab.
-Installed cameras:

• Logitech Quickcam 4000
• Logitech Quickcam 4000 #2
• IBM PC Camera
• Intel CS 110

-Code:

• camerainput.m - displays 500 frames for 1 cam
• 2camerainput.m - displays 500 frames for 2 cam
• 3camerainput.m - displays 500 frames for 3 cam
• 4camerainput.m - displays 500 frames for 4 cam NOT WORKING

-Limitations:

• 2 Cameras work on laptop. Adding a 3rd camera will exceed bandwidth limits.
• 3 Cameras work on Desktop. Adding a 4th camera will exceed bandwidth limits.

-Problem: 4 cameras exceed USB bandwidth

-Possible Solutions:

• Disconnect keyb and mouse before connecting cameras.
• Try different USB slots for cameras.
• Try another camera instead of Intel CS110
• Have 3 cameras on each PC