Final Project Documentation

[rluo]

Please upload the final project documentation as a PDF file.
Also attach the source codes as a zip file to this forum.

Due by 11:59:59pm, Friday, June 8th

Sample Documentation
-----------------------------------

A Curious Robot
Jieliang Luo

Concept
Describe the concept of the project
-- some images --

Implementation Process
Describe how you implemented the project
-- some images --

Final Result
Discuss the final result and possible future improvements
-- some images from the show --
A link to the video [optional but highly encouraged]

Distribution of Labor
Discuss who was taking charge of which part of the project.

References
List the projects and codes that inspired you.

[christinalast]

Hide-and-Seek Robot
by Christina Last and Ari Gillmore

Overview
Many similarities of the hide-and-seek game can be found in the interactions between humans, or robots and humans, in urban spaces, similar to looking for a person in a crowded urban environment (Goldhoorn et al. 2014). It is suggested that the game of hide-and-seek is an ideal domain for studying cognitive functions in robots and it is a basic mechanism for human robot interaction in mobile robotics, because hide-and-seek requires the robot to navigate, search, interact on, and predict actions of the opponent (Johansson and Balkenius 2005).

In this project we demonstrate an ability to program on Arduino & Raspberry Pi and communicate between the two systems, program in Python, focusing on OpenCV, and implement deep learning models on the robots to identify faces from purely visual inputs.

Our Robot (105) also known as Robo-Cop

DSC03258.JPG (64.74 KiB) Viewed 40231 times

Concept
The concept is as follows: the project uses machine learning classification techniques to successfully find a hider by identifying the hider’s face, and move towards the human (the robot’s method of seeking). The visual information captured by the Pi camera is used to train the robot to find a hider. Through the artistic novelty of human-robot playing we explore the real-world implications in supporting human disaster response teams. For example, one of the many practical applications of this experiment is to find persons; as hide-and-seek could be seen as a simplification of search-and-rescue.

Image showing original idea for hide-and-seek game to be played in a grid

Sketch_1.jpeg (28.43 KiB) Viewed 40234 times

Image showing original idea that Images would be stitched together.

Sketch_2.jpeg (27.41 KiB) Viewed 40234 times

Implementation Process
There are two players, the hider (a person) and the seeker (the robot) and they play the game in free space. The game of hide and seek runs for a maximum number of timesteps. Within the number of time steps, the hider will attempt to move past the robot and collect a reward (perhaps enter a certain location in the room, or collect an item the robot is guarding). At each time step, the seeker will scan the room and capture images during the 360 degrees rotation. It will run a machine learning classification algorithm to identify faces from the captured images. Each image is serialised and is described by the angle at which the image was captured. When a face is identified, the robot chassis moves to the correct angle of orientation and forward in attempt to catch the hider. The process is repeated until the hider is successfully caught, or the hider wins.

Part of Ardunio code where one can see how we implemented the angles that the robot turned at and the signals it sent to the Raspberry Pi.

Screen Shot 2018-06-05 at 10.55.35 AM.png (14.42 KiB) Viewed 40234 times

Part 1 of Python Code:
Reading in the serial number from the Arduino, so the Raspberry Pi knows when to take an image

Screen Shot 2018-06-06 at 10.35.59 PM.png (17.67 KiB) Viewed 40234 times

Python 2 Code Part:
Checks each image saved in the images folder to see if there is a face. If there is a face send that number to the Arduino, if not send a random number. Wait for Arduino to send next serial number to delete the old images and restart the process.

Screen Shot 2018-06-06 at 10.36.09 PM.png (24.45 KiB) Viewed 40234 times

Processing Code: The gave the viewer the opportunity to learn about the project, see the images the robot was taking, and show the faces it found.

Processing_1.PNG (22.72 KiB) Viewed 40234 times

Steps of implementation:
1. Capture an image at each angle during a single rotation

• a. Determine how many photos are needed to capture all detail in a 360 degree rotation.
  b. Using Arduino, accelerate chassis at correct speed for correct time to pause at each 30 degree angle in 360 rotation.
  c. Using Arduino, send serial number to an ‘Image capture’ python program to take an image at each 30 degree angle.
  d.Use facial recognition machine learning program to attempt to detect a face in each image.
  e. Name image files ‘num.jpeg’ and send all image files to a folder called ‘Images’.

2. Detect the face of the hider

• a. Use the machine learning face classification programme to identify potential faces in each image.
  b. Name each image containing a face ‘numFace.jpeg’ and send all face images to a folder called ‘FaceImages’.

3. Move robot towards hider

• a. Use LCD screen on chassis to determine angle that image is captured from any given start angle (ranges from 0-(+180), 0-(-180)).
  b. Use this angle as signal to send serial number from python, (identifying the angle of the selected image for the arduino).
  c. Function in arduino to move forward a fixed amount (one step) at this given angle when serial is read.

3. Develop programme to display images to public

• a. Create title and instruction slide in ‘star wars screen title theme’.
  b. Draw live images into processing from ‘Images’ and ‘FaceImages’ folder and display them when specific keys are pressed.
  c. Reset programme after one round of ‘seeking’ is complete.

Final Result
We anticipated that this experiment would give us important insights into the functionality of automates seeker methods used by a robot in real-world playing (interacting and predicting) against humans. We were able to successfully implement machine learning techniques by classifying faces from purely visual inputs. However, we were unable to implement model-based reinforcement learning to enable to robot to make a strategic movement. Instead, we extracted the angle of image, gave it a serial number, and sent this to the arduino to move to the specific angle, then move forward for a given distance.

In the future, we would like to find a better way of implementing the Processing visual with the automated robotic ‘seeking’. Because the images took up large amount of working memory on the Pi, the 360 scan and the face identification processing time increased significantly after the first round of seeking. We were also planning to ‘stitch’ the panorama of images the robot took, and visually display this to the audience. However, the multiple image stitching programmes we found were difficult to implement with the Pi and did not work well with over 4 images in the dataset. In addition, we would like to produce our own training data which identifies a ‘hider’ and a ‘base’. Using this training data, we can reward the ‘seeker’ when it completes a move that:

• - Minimises the distance of the seeker from the hider (dsh)
  - Maximses the hider from the base (dhb)
  - The distance of the seeker from the base is less than the distance of the hider from the base (dsb < dhb).

Because of the time limitation on the project we were unable to generate enough training data and implement the reward function we created.

Overall, the success of this project can be assessed from the reactions at the MAT EoYS. We believe the audience felt engaged, inspired and even amused at the outcome of our project and the novelty of real-world play with a robot. Our project involved our audience visually and physically, and engaged them with questions such as “what situations can these techniques be adopted?”, and “what situations can these techniques be abused?”. Although the novelty of human-robot playing may evoke purely artistic expression, this experiment successfully explored the first steps toward real world search-and-rescue simulation.

Ari presenting our final project at MAT EoYS

DSC03256.JPG (75.72 KiB) Viewed 40231 times

Robo-Cop performing a search

IMG_4834.jpeg (14.57 KiB) Viewed 40231 times

Screenshot of how the processing code was presented on the screen.

Processing_2.PNG (23.57 KiB) Viewed 40234 times

Distribution of Labor
The distribution of labour is as follows:
Proposal Writing

• Content - Christina
  Layout - Christina

Capture an image at each angle during a single rotation

• Arduino code - Ari and Christina
  Python code - Ari and Christina

Detect the face of the hider

• Python code - Ari and Christina

Move robot towards hider

• Arduino code - Ari
  Python code - Ari

Develop programme to display images to public

• Processing code - Christina

End of Year Show

• Delivery - Ari and Christina

Final Documentation

• Content - Christina
  Layout - Ari
  Film - Christina

Presentation

• Delivery - Ari

References
M.A. Armada et al. (2013) ROBOT2013: First Iberian Robotics Conference, 505 Advances in Intelligent Systems and Computing, 253.
Johansson, E., Balkenius, C. (2005) It’s a child’s game: Investigating cognitive development with playing robots. In: International Conference on Development and Learning, vol. 164.
Luo, R. (2017) Collecting boxes in the Unreal. (available at: http://www.rodgerluo.com/projects/rl_drones.html).
Professor Craven. (2016) Example code for Polulu's Arduino-based Zumo 32U4 robot, (available at https://github.com/pvcraven/zumo_32u4_examples).
RodgerLuo (2018) UCSB Intelligent Machine Vision Course, (available at https://github.com/RodgerLuo/robotic-vi ... ion-course).

[ariellalgilmore]

Attached is our source code zip file and a link to our video.

Source_code.zip

source code

(1.61 MiB) Downloaded 1457 times

https://drive.google.com/file/d/1zotG3N ... sp=sharing

[robert.gnzalez]

ART185 Optical Digital Culture - Intelligent Machine Vision

Dream Catcher
An Autonomous Robot that Dreams

Roberto Gonzalez
Adam Jhanke
Chris Hoang

[mfargas53]

Scavenger Hunt


Attached are documentation and codes for our project.

Thanks for an amazing quarter and all the guidance. Good luck in your future endeavors, Rodger!

-Kara Newman and Margie Fargas

[stevenhoward]

Attached is a .pdf and .zip file documenting the process for our project "AMP"

By:
- Chloe Hopen
- Janet Diaz
- Steven Howard

[yichenli]

Please find attached the pdf file for groups 2 and 6.
The zip file is linked here: https://drive.google.com/file/d/1c1PPpw ... sp=sharing

Emily Beckius
Lawrence Chen
Adrian Esqueda
Yichen Li
Wen Ying Liu

[yurim_lee]

https://drive.google.com/open?id=1AJAky ... _TyUjeiaPd

I couldn't upload our final documentation as the file sizes were too large but here's a link!

[annikatan]

PDF of final project report.

- Chantel Chan
- Wilson Mui
- Annika Tan