Red Hen is working with a computer science team at the University of Basel in Switzerland on a sketch-based visual search engine called iMotion. You can contribute to creating the training data for this project with the iMotion annotator, which is a simple java application that allows you to draw a rectangle on an image to indicate the location of an event or entity, such as a gesture or a person.
Here's the full 10-step recipe to get you going with the iMotion annotator. It assumes you have access to a mac or linux computer.
- Open Terminal and check that you have at least Java 7 -- if not, update your Java:
- Create the directory ~/Documents/redhen/images with this command:
mkdir -p ~/Documents/redhen/images
- Browse to http://spello.sscnet.ucla.edu/pdf/vrNewsScape/annotator.jar
and download the file to ~/Documents/redhen
This is our tagging utility,
used to create bounding boxes on images. The syntax for this utility is:
java -jar annotator.jar <input folder>
- To test that this works, place some random images (png, jpg) in the ~/Documents/redhen/images folder.
Back in Terminal, change directory to redhen, verify the program is there, and run the annotator:
ls -l (you should see annotator.jar)
java -jar annotator.jar images test.iMotion
You should see the first image. Use your
mouse to mark a bounding box (in red) on the image, and press the
Spacebar. That should call up the next image.
- Browse to this page and install MacPorts:
https://www.macports.org/install.php (linux users can skip this step).
Select your version of OS X. Note that
you also need XCode and the command-line developer tools, see the
It's a bit complicated, but it works!
- MacPorts lets you install various open-source
programs Red Hen routinely uses. Give these commands:
sudo port selfupdate
sudo port install ffmpeg-devel +gpl2 +gpl3 +nonfree
When you type the sudo command, you'll be asked for a password
-- that's the regular password you use to log into your own laptop.
If that goes well (it will take a while), continue with
sudo port install coreutils gsed bash findutils wget mp4v2 ossp-uuid dos2unix alpine moreutils autossh
(Linux users instead install these packages: ffmpeg wget mp4v2-utils uuid dos2unix alpine moreutils autossh)
- Protect the machine from attacks
The macports package fail2ban is useful for blocking attackers; see Configure fail2ban.
- Create a folder to hold the videos and download the first video (assuming you are a turnerstudent):
- Next, let's get the Red Hen script that extracts frames from the
video at one-second intervals and place it in your local script folder:
email@example.com:/usr/local/bin/extract-frames-ffmpeg.sh /tmp -av
sudo mkdir -p /usr/local/bin
sudo mv /tmp/extract-frames-ffmpeg.sh /usr/local/bin
- Verify you have the script in place
If you get a response, you can cd to the folder where you have the
mp4 file (if you're not there already) and issue the frame-extraction command:
This should extract frames at one-second intervals into a folder
- If that works, move that folder into images and try the annotator tool on it; see if you
can call up one image after the other by using the spacebar:
mv 2015-07-03_2300_US_WKYC_Channel_3_News_at_7.hq ../images/
java -jar annotator.jar images/2015-07-03_2300_US_WKYC_Channel_3_News_at_7.hq 2015-07-03_2300_US_WKYC_Channel_3_News_at_7.iMotion
Congratulations! Let us know you have the iMotion annotator working!