Final Project Documentation

Post Reply
Posts: 21
Joined: Tue Oct 14, 2014 10:57 am

Final Project Documentation

Post by rluo » Thu Apr 12, 2018 9:01 am

Please upload the final project documentation as a PDF file.
Also attach the source codes as a zip file to this forum.

Due by 11:59:59pm, Friday, June 8th

Sample Documentation

A Curious Robot
Jieliang Luo

Describe the concept of the project
-- some images --

Implementation Process
Describe how you implemented the project
-- some images --

Final Result
Discuss the final result and possible future improvements
-- some images from the show --
A link to the video [optional but highly encouraged]

Distribution of Labor
Discuss who was taking charge of which part of the project.

List the projects and codes that inspired you.

Posts: 7
Joined: Fri Jan 19, 2018 11:15 am

Re: Final Project Documentation

Post by christinalast » Fri Jun 08, 2018 2:53 pm

Hide-and-Seek Robot
by Christina Last and Ari Gillmore

Many similarities of the hide-and-seek game can be found in the interactions between humans, or robots and humans, in urban spaces, similar to looking for a person in a crowded urban environment (Goldhoorn et al. 2014). It is suggested that the game of hide-and-seek is an ideal domain for studying cognitive functions in robots and it is a basic mechanism for human robot interaction in mobile robotics, because hide-and-seek requires the robot to navigate, search, interact on, and predict actions of the opponent (Johansson and Balkenius 2005).

In this project we demonstrate an ability to program on Arduino & Raspberry Pi and communicate between the two systems, program in Python, focusing on OpenCV, and implement deep learning models on the robots to identify faces from purely visual inputs.
Our Robot (105) also known as Robo-Cop
DSC03258.JPG (64.74 KiB) Viewed 34108 times
The concept is as follows: the project uses machine learning classification techniques to successfully find a hider by identifying the hider’s face, and move towards the human (the robot’s method of seeking). The visual information captured by the Pi camera is used to train the robot to find a hider. Through the artistic novelty of human-robot playing we explore the real-world implications in supporting human disaster response teams. For example, one of the many practical applications of this experiment is to find persons; as hide-and-seek could be seen as a simplification of search-and-rescue.
Image showing original idea for hide-and-seek game to be played in a grid
Sketch_1.jpeg (28.43 KiB) Viewed 34111 times
Image showing original idea that Images would be stitched together.
Sketch_2.jpeg (27.41 KiB) Viewed 34111 times
Implementation Process
There are two players, the hider (a person) and the seeker (the robot) and they play the game in free space. The game of hide and seek runs for a maximum number of timesteps. Within the number of time steps, the hider will attempt to move past the robot and collect a reward (perhaps enter a certain location in the room, or collect an item the robot is guarding). At each time step, the seeker will scan the room and capture images during the 360 degrees rotation. It will run a machine learning classification algorithm to identify faces from the captured images. Each image is serialised and is described by the angle at which the image was captured. When a face is identified, the robot chassis moves to the correct angle of orientation and forward in attempt to catch the hider. The process is repeated until the hider is successfully caught, or the hider wins.
Screen Shot 2018-06-05 at 10.55.35 AM.png
Part of Ardunio code where one can see how we implemented the angles that the robot turned at and the signals it sent to the Raspberry Pi.
Screen Shot 2018-06-05 at 10.55.35 AM.png (14.42 KiB) Viewed 34111 times
Screen Shot 2018-06-06 at 10.35.59 PM.png
Part 1 of Python Code:
Reading in the serial number from the Arduino, so the Raspberry Pi knows when to take an image
Screen Shot 2018-06-06 at 10.35.59 PM.png (17.67 KiB) Viewed 34111 times
Screen Shot 2018-06-06 at 10.36.09 PM.png
Python 2 Code Part:
Checks each image saved in the images folder to see if there is a face. If there is a face send that number to the Arduino, if not send a random number. Wait for Arduino to send next serial number to delete the old images and restart the process.
Screen Shot 2018-06-06 at 10.36.09 PM.png (24.45 KiB) Viewed 34111 times
Processing Code: The gave the viewer the opportunity to learn about the project, see the images the robot was taking, and show the faces it found.
Processing_1.PNG (22.72 KiB) Viewed 34111 times
Steps of implementation:
1. Capture an image at each angle during a single rotation
  • a. Determine how many photos are needed to capture all detail in a 360 degree rotation.
    b. Using Arduino, accelerate chassis at correct speed for correct time to pause at each 30 degree angle in 360 rotation.
    c. Using Arduino, send serial number to an ‘Image capture’ python program to take an image at each 30 degree angle.
    d.Use facial recognition machine learning program to attempt to detect a face in each image.
    e. Name image files ‘num.jpeg’ and send all image files to a folder called ‘Images’.
2. Detect the face of the hider
  • a. Use the machine learning face classification programme to identify potential faces in each image.
    b. Name each image containing a face ‘numFace.jpeg’ and send all face images to a folder called ‘FaceImages’.
3. Move robot towards hider
  • a. Use LCD screen on chassis to determine angle that image is captured from any given start angle (ranges from 0-(+180), 0-(-180)).
    b. Use this angle as signal to send serial number from python, (identifying the angle of the selected image for the arduino).
    c. Function in arduino to move forward a fixed amount (one step) at this given angle when serial is read.
3. Develop programme to display images to public
  • a. Create title and instruction slide in ‘star wars screen title theme’.
    b. Draw live images into processing from ‘Images’ and ‘FaceImages’ folder and display them when specific keys are pressed.
    c. Reset programme after one round of ‘seeking’ is complete.
Final Result
We anticipated that this experiment would give us important insights into the functionality of automates seeker methods used by a robot in real-world playing (interacting and predicting) against humans. We were able to successfully implement machine learning techniques by classifying faces from purely visual inputs. However, we were unable to implement model-based reinforcement learning to enable to robot to make a strategic movement. Instead, we extracted the angle of image, gave it a serial number, and sent this to the arduino to move to the specific angle, then move forward for a given distance.

In the future, we would like to find a better way of implementing the Processing visual with the automated robotic ‘seeking’. Because the images took up large amount of working memory on the Pi, the 360 scan and the face identification processing time increased significantly after the first round of seeking. We were also planning to ‘stitch’ the panorama of images the robot took, and visually display this to the audience. However, the multiple image stitching programmes we found were difficult to implement with the Pi and did not work well with over 4 images in the dataset. In addition, we would like to produce our own training data which identifies a ‘hider’ and a ‘base’. Using this training data, we can reward the ‘seeker’ when it completes a move that:
  • - Minimises the distance of the seeker from the hider (dsh)
    - Maximses the hider from the base (dhb)
    - The distance of the seeker from the base is less than the distance of the hider from the base (dsb < dhb).
Because of the time limitation on the project we were unable to generate enough training data and implement the reward function we created.

Overall, the success of this project can be assessed from the reactions at the MAT EoYS. We believe the audience felt engaged, inspired and even amused at the outcome of our project and the novelty of real-world play with a robot. Our project involved our audience visually and physically, and engaged them with questions such as “what situations can these techniques be adopted?”, and “what situations can these techniques be abused?”. Although the novelty of human-robot playing may evoke purely artistic expression, this experiment successfully explored the first steps toward real world search-and-rescue simulation.
Ari presenting our final project at MAT EoYS
DSC03256.JPG (75.72 KiB) Viewed 34108 times
Robo-Cop performing a search
IMG_4834.jpeg (14.57 KiB) Viewed 34108 times
Screenshot of how the processing code was presented on the screen.
Processing_2.PNG (23.57 KiB) Viewed 34111 times
Distribution of Labor
The distribution of labour is as follows:
Proposal Writing
  • Content - Christina
    Layout - Christina
Capture an image at each angle during a single rotation
  • Arduino code - Ari and Christina
    Python code - Ari and Christina
Detect the face of the hider
  • Python code - Ari and Christina
Move robot towards hider
  • Arduino code - Ari
    Python code - Ari
Develop programme to display images to public
  • Processing code - Christina
End of Year Show
  • Delivery - Ari and Christina
Final Documentation
  • Content - Christina
    Layout - Ari
    Film - Christina
  • Delivery - Ari
M.A. Armada et al. (2013) ROBOT2013: First Iberian Robotics Conference, 505 Advances in Intelligent Systems and Computing, 253.
Johansson, E., Balkenius, C. (2005) It’s a child’s game: Investigating cognitive development with playing robots. In: International Conference on Development and Learning, vol. 164.
Luo, R. (2017) Collecting boxes in the Unreal. (available at:
Professor Craven. (2016) Example code for Polulu's Arduino-based Zumo 32U4 robot, (available at
RodgerLuo (2018) UCSB Intelligent Machine Vision Course, (available at ... ion-course).
Final Documentation 2.pdf
PDF Version of our final documentation
(1.83 MiB) Downloaded 3571 times

Posts: 7
Joined: Wed Jan 11, 2017 10:40 am

Re: Final Project Documentation

Post by ariellalgilmore » Fri Jun 08, 2018 3:43 pm

Attached is our source code zip file and a link to our video.
source code
(1.61 MiB) Downloaded 1023 times ... sp=sharing

Posts: 3
Joined: Mon Apr 16, 2018 10:28 am

Re: Final Project Documentation

Post by robert.gnzalez » Fri Jun 08, 2018 5:30 pm

ART185 Optical Digital Culture - Intelligent Machine Vision

Dream Catcher
An Autonomous Robot that Dreams

Roberto Gonzalez
Adam Jhanke
Chris Hoang
Source codes .zip
(12.12 KiB) Downloaded 859 times
DC Final Report.pdf
(1.83 MiB) Downloaded 2208 times

Posts: 6
Joined: Wed Apr 12, 2017 5:48 pm

Re: Final Project Documentation

Post by mfargas53 » Fri Jun 08, 2018 9:59 pm

Scavenger Hunt

Attached are documentation and codes for our project.

Thanks for an amazing quarter and all the guidance. Good luck in your future endeavors, Rodger!

-Kara Newman and Margie Fargas
Art 185 Final
(295.41 KiB) Downloaded 859 times
Final Project Documentation.pdf
(720.56 KiB) Downloaded 807 times

Posts: 2
Joined: Mon Apr 16, 2018 10:28 am

Re: Final Project Documentation

Post by stevenhoward » Fri Jun 08, 2018 10:21 pm

Attached is a .pdf and .zip file documenting the process for our project "AMP"

- Chloe Hopen
- Janet Diaz
- Steven Howard
(83.95 KiB) Downloaded 889 times
(2.47 KiB) Downloaded 866 times

Posts: 14
Joined: Mon Apr 16, 2018 10:23 am

Re: Final Project Documentation

Post by yichenli » Fri Jun 08, 2018 10:31 pm

Please find attached the pdf file for groups 2 and 6.
The zip file is linked here: ... sp=sharing

Emily Beckius
Lawrence Chen
Adrian Esqueda
Yichen Li
Wen Ying Liu
(1.06 MiB) Downloaded 2233 times

Posts: 3
Joined: Mon Apr 16, 2018 10:20 am

Re: Final Project Documentation

Post by yurim_lee » Fri Jun 08, 2018 11:17 pm ... _TyUjeiaPd

I couldn't upload our final documentation as the file sizes were too large but here's a link!

Posts: 6
Joined: Fri Jan 19, 2018 11:03 am

Re: Final Project Documentation

Post by annikatan » Fri Jun 08, 2018 11:42 pm

PDF of final project report.

- Chantel Chan
- Wilson Mui
- Annika Tan
(689.83 KiB) Downloaded 823 times

Post Reply