Proj 4: 3D Volumetric, Spacial Visualization

glegrady
Posts: 203
Joined: Wed Sep 22, 2010 12:26 pm

Proj 4: 3D Volumetric, Spacial Visualization

Post by glegrady » Mon Jan 05, 2015 3:45 pm

3D Volumetric, Spacial Visualization
In this assignment, 2D mapping is extended into 3D space through the introduction of a 4th data set so that data is positioned in x,y,z location using color and/or variable data scale to represent value. 3D volumetric representation is an ideal method to express “change over time”.

3D spatial navigation and interactivity is introduced using the PeasyCam library: http://mrfeinberg.com/peasycam/

ControlP5 is a GUI and controller library that includes functions like Sliders, Buttons, Toggles, Knobs, Textfields, RadioButtons, etc. h[url]ttp://www.sojamo.de/libraries/controlP5/[/url]

OpenGL (Open Graphics Library), a cross-platform graphics interface is used in this demo. Translate, pushmatrix, popmatrix functions are introduced. Information about push, pop and translation can be found at:
https://www.processing.org/tutorials/transform2d/
http://processing.org/tutorials/p3d

Demo is at: http://www.mat.ucsb.edu/~g.legrady/acad ... _MH_v5.zip

Assignment Criteria
Use the full capabilities of 3D where each x,y,z location represents data. You may map data directly into 3D space based on the 4 datasets, or explore clustering algorithms as a means of organizing the data in 3D space.

This assignments requires granular definition in 3D space (increased data to achieve visual detail), so mapping the 10 main Dewey categories will not be sufficient, nor fulfill the expectation of an interesting query.

1st draft assignment is due: February 19, 2015
George Legrady
legrady@mat.ucsb.edu

donghaoren
Posts: 5
Joined: Sat Jan 10, 2015 11:33 am

Re: Proj 4: 3D Volumetric, Spacial Visualization

Post by donghaoren » Wed Feb 18, 2015 10:57 am

Spatial-temporal Checkouts
screenshot.png
In this assignment, I built a visualization to show checkout trends of different kinds of books. The design is a volumetric visualization, using the X-Y plane to layout the books, and the Z axis to show the checkout trend.

Book layout: The books are layout by their keywords. Similar books (with similar keywords) are placed closely in the layout. The algorithm was a 2-layer RBM model. There are 807,493 books in the collection, each has a X, Y coordinate given by the algorithm.

Rendering: Since volume rendering is not possible in Processing, I rendered the volume as 108 slices, corresponding to the 108 months from Jan. 2006 to Dec. 2014. The colors was chosen from a given gradient using shaders. So what you see in the picture is actually 108 transparent images stacked together, giving you a volumetric feeling.

The image of the 108 slices and the colormap (bottom):
flatten.png
Volume rendering (ray casting) of the data:
volume-rendering.jpg
Assignment_4.zip
(1.49 MiB) Downloaded 295 times

a.lazareva
Posts: 5
Joined: Sat Jan 10, 2015 11:29 am

Re: Proj 4: 3D Volumetric, Spacial Visualization

Post by a.lazareva » Wed Feb 18, 2015 8:22 pm

3D Spider Diagram

This vis is a 3D version of my previous project. The vis allows you to see the diagram in 3D and control the spacing between the different layers as well as the position of the grid.
screenshot.jpg
mockup2.jpg
Updated
screenshot.jpg
final
https://www.dropbox.com/s/0010vm9if0f8c ... 4.zip?dl=0
Last edited by a.lazareva on Tue Feb 24, 2015 2:20 pm, edited 3 times in total.

kurtoon
Posts: 5
Joined: Sat Jan 10, 2015 11:28 am

Re: Proj 4: 3D Volumetric, Spacial Visualization

Post by kurtoon » Thu Feb 19, 2015 12:14 am

Into The Wild: 3D Comparison of Seattle's Interests in Fiction and Non-fiction

For this project, I was curious what the most popular titles in the Seattle Public Library were, and how the check out behavior of those titles changed over time. In performing the MySQL queries, it became apparent that the most checked out titles were not within Dewey categories, so I decided to perform two queries: what were the top 20 titles overall and the top 20 titles with a Dewey category number? Then, I used those bib numbers to search for their activity over time. I found that many of the most popular titles were not acquired until after 2008, so I began there. The query ends at 2013.

In Processing, I used a Kiviat Tube (described here and here) to display the information. Each pillar contains the check out data of the top 20 most popular items either across the entire library, or constrained to within Dewey. One vertical polygon equals one week and its distance from the center is how often it was checked out that week. The data is normalized appropriately based on what is being displayed. Circles are added to indicate the year, and at each year the names of each title fade in as the camera comes close. Color is chosen based on the rank of each title, and polygons become transparent when there is no activity. A shaded effect was achieved by assigning darker values to polygons at the base of each strip.
pic1.jpg
pic2.jpg
pic3.jpg
pic4.jpg
This visualization clearly shows some large scale trends in the library: the popularity of non-Dewey categorized titles far outweighs that of the Dewey categorized titles, and most of the popular items were acquired in 2008 or 2009, suggesting the library is experiencing an overall decline in activity. Of note, "Uncataloged folder or Bag BAL" came up as the 12th most checked out title in the system. Its activity is clearly much lower than its competitors but is also more consistent, eventually rising above them by 2013 as their popularity wanes. What lies in this mysterious "Bag?"

MySQL Queries
Most checked out titles in the database:

Code: Select all

select * from (
select deweyClass.deweyClass, deweyClass.bibNumber, title.title, x_checkOutCountBib.checkOutCount
from deweyClass, x_checkOutCountBib, title
where deweyClass.bibNumber = x_checkOutCountBib.bibNumber and deweyClass.bibNumber = title.bibNumber
order by x_checkOutCountBib.checkOutCount desc) as t 
group by deweyClass
order by checkOutCount desc
Most checked out items with a dewey number:

Code: Select all

select * from (
select x_checkOutCountBib.bibNumber, title.title, x_checkOutCountBib.checkOutCount
from x_checkOutCountBib, title
where x_checkOutCountBib.bibNumber = title.bibNumber
order by x_checkOutCountBib.checkOutCount desc) as t 
limit 50
I used the bib numbers obtained from these two queries to look at their check out activity across time. I grouped the data into weekly increments, so each row returned is the sum of one week of activity.

Most checked out titles frequency:

Code: Select all

select YEAR(checkOut), MONTH(checkOut),
sum(case when bibNumber = 2469502 then 1 else 0 end) as "Into the wild",
sum(case when bibNumber = 2542732 then 1 else 0 end) as "Burn after reading",
sum(case when bibNumber = 2474843 then 1 else 0 end) as "Michael Clayton",
sum(case when bibNumber = 2482761 then 1 else 0 end) as "Atonement",
sum(case when bibNumber = 2560429 then 1 else 0 end) as "reader",
sum(case when bibNumber = 2560415 then 1 else 0 end) as "Doubt",
sum(case when bibNumber = 2472916 then 1 else 0 end) as "Darjeeling Limited",
sum(case when bibNumber = 2532278 then 1 else 0 end) as "Vicky Cristina Barcelona",
sum(case when bibNumber = 2474845 then 1 else 0 end) as "No country for old men",
sum(case when bibNumber = 2569682 then 1 else 0 end) as "curious case of Benjamin Button",
sum(case when bibNumber = 2485732 then 1 else 0 end) as "Juno",
sum(case when bibNumber = 2569703 then 1 else 0 end) as "Slumdog millionaire",
sum(case when bibNumber = 1205054 then 1 else 0 end) as "Uncataloged Folder or Bag BAL",
sum(case when bibNumber = 2560425 then 1 else 0 end) as "Milk",
sum(case when bibNumber = 2472920 then 1 else 0 end) as "Gone baby gone",
sum(case when bibNumber = 2461823 then 1 else 0 end) as "Once",
sum(case when bibNumber = 2560428 then 1 else 0 end) as "Rachel getting married",
sum(case when bibNumber = 2485720 then 1 else 0 end) as "Charlie Wilsons war",
sum(case when bibNumber = 2469492 then 1 else 0 end) as "Becoming Jane",
sum(case when bibNumber = 2480593 then 1 else 0 end) as "golden compass"
from transactions 
where YEAR(checkOut) > 2007 and YEAR (checkOut) < 2014
GROUP BY YEAR(checkOut), MONTH(checkOut), mod(DAY(checkOut), 4) 
ORDER BY YEAR(checkOut), MONTH(checkOut), DAY(checkOut)
-- ran in 289 sec
Most checked out dewey-categorized titles frequency:

Code: Select all

select YEAR(checkOut), MONTH(checkOut),
sum(case when bibNumber = 2450933 then 1 else 0 end) as "Sicko",
sum(case when bibNumber = 2549448 then 1 else 0 end) as "Religulous",
sum(case when bibNumber = 2317605 then 1 else 0 end) as "March of the penguins",
sum(case when bibNumber = 2532708 then 1 else 0 end) as "Man on wire",
sum(case when bibNumber = 2337880 then 1 else 0 end) as "omnivores dilemma a natural history of four meals",
sum(case when bibNumber = 2608838 then 1 else 0 end) as "Food Inc",
sum(case when bibNumber = 2336383 then 1 else 0 end) as "Junebug",
sum(case when bibNumber = 2380691 then 1 else 0 end) as "inconvenient truth a global warning",
sum(case when bibNumber = 2127084 then 1 else 0 end) as "Pimsleur language programs Spanish 1 A the complete course",
sum(case when bibNumber = 2264211 then 1 else 0 end) as "Blink the power of thinking without thinking",
sum(case when bibNumber = 2513376 then 1 else 0 end) as "Outliers the story of success",
sum(case when bibNumber = 2327416 then 1 else 0 end) as "Grizzly man",
sum(case when bibNumber = 2327750 then 1 else 0 end) as "Eat pray love one womans search for everything across Italy India and Indonesia",
sum(case when bibNumber = 2285693 then 1 else 0 end) as "Freakonomics a rogue economist explores the hidden side of everything",
sum(case when bibNumber = 2333524 then 1 else 0 end) as "Three cups of tea one mans mission to fight terrorism and build nations one school at a time",
sum(case when bibNumber = 2407581 then 1 else 0 end) as "Back to black",
sum(case when bibNumber = 2398887 then 1 else 0 end) as "Jesus camp",
sum(case when bibNumber = 2430935 then 1 else 0 end) as "Planet Earth The complete series",
sum(case when bibNumber = 1923072 then 1 else 0 end) as "Guinness world records",
sum(case when bibNumber = 2331997 then 1 else 0 end) as "Chart your success on the COMPASS test"
from transactions 
where YEAR(checkOut) > 2007 and YEAR(checkOut) < 2014
GROUP BY YEAR(checkOut), MONTH(checkOut), mod(DAY(checkOut), 4)
ORDER BY YEAR(checkOut), MONTH(checkOut), DAY(checkOut)
-- ran in 295 secs
kaminski_hw4_3d_v1.zip
Processing sketch folder w/ screenshots
(1.54 MiB) Downloaded 298 times

fabian.offert
Posts: 5
Joined: Sat Jan 10, 2015 11:32 am

Re: Proj 4: 3D Volumetric, Spacial Visualization

Post by fabian.offert » Thu Feb 19, 2015 1:05 am

It's The Economy, Stupid! - 3D Visualization

Like the previous versions of my project, the visualization shows the possible effects of the 2007-2008 financial crisis on the choice of books borrowed by the patrons of the Seattle public library system. I examine four keywords and map their prevalence over time to the patron's general interest in topics related to economics (Dewey decimal numbers 330-339). For the 3D version, I tried to create an "analog" interface that takes up graphical elements and color schemes from hand-written plans and diagrams while still being interactive and animated.

Screenshots
p1.png
p2.png
p3.png
p4.png

Zipped Project

sketch.zip
(7.63 KiB) Downloaded 306 times

intae
Posts: 19
Joined: Tue Oct 14, 2014 10:56 am

Re: Proj 4: 3D Volumetric, Spacial Visualization

Post by intae » Thu Feb 19, 2015 1:29 am

"3D Graph: Comicbook checkouts by countries: 2006 - 2013 "

I dig into more about Dewey class 741.59 which is geographically classified code. The classification had not fixed, and simultaneulsy changed by librarians. Interestingly, Japan has the biggest number of volumes in cartoon section, even if the library located in the Seattle, there were some period that total checkout of Japanese mange were more than American's.

In some countries, famous artist take some categories, such as Hernandez_Jaime_Dean_Koontz_Graphic_Novel (741.59573),Palmiotti_Jimmy (741.5953) ,also, there is a section only for the "Seattle_publisher" (741.59797).

I used the MySQL result between 2006 to 2013 by every month, then arranged these numbers in the graph. The color stands for total checkout number, but the gap between two major cartoon countries Japan, US are far more than any other countries, the color variation lost its meaning. Instead, I put a number slide which shows number of actual check out by elevation.
Screen Shot 2015-02-22 at 6.21.14 PM.png
Screen Shot 2015-02-22 at 6.21.33 PM.png

Code: Select all

SELECT 


     SUM(case
        When deweyClass = '741.5941' then 1
        else 0 end) as British,

     SUM(case
        When deweyClass = '741.59411' then 1
        else 0 end) as Scotland,
    
	 SUM(case
        When deweyClass = '741.5942' then 1
        else 0 end) as England,
        
	 SUM(case
        When deweyClass = '741.5943' then 1
        else 0 end) as Germany,
        
	 SUM(case
        When deweyClass = '741.5944' then 1
        else 0 end) as France,
    
	 SUM(case
        When deweyClass = '741.5945' then 1
        else 0 end) as Italy,
        
	 SUM(case
        When deweyClass = '741.5946' then 1
        else 0 end) as Spain,
        
	 SUM(case
        When deweyClass = '741.5947' then 1
        else 0 end) as Eastern_Europe,
        
	 SUM(case
        When deweyClass = '741.5948' then 1
        else 0 end) as Northern_Europe,
        
	 SUM(case
        When deweyClass = '741.59481' then 1
        else 0 end) as Norway,
        
	 SUM(case
        When deweyClass = '741.59485' then 1
        else 0 end) as Sweden,
        
	 SUM(case
        When deweyClass = '741.59489' then 1
        else 0 end) as Finland,
        
     SUM(case
        When deweyClass = '741.59492' then 1
        else 0 end) as Netherlands,
        
	SUM(case
        When deweyClass = '741.59493' then 1
        else 0 end) as Belgium,
        
	 SUM(case
        When deweyClass = '741.59494' then 1
        else 0 end) as Swiss
        
            SUM(case
        When deweyClass = '741.5971' then 1
        else 0 end) as Canada,
        
	SUM(case
        When deweyClass = '741.5972' then 1
        else 0 end) as Mexico,
    
    SUM(case
        When deweyClass = '741.5973' then 1
        else 0 end) as United_States,
        
	SUM(case
        When deweyClass = '741.59794' then 1
        else 0 end) as Walt_Disney,
        
	SUM(case
        When deweyClass = '741.59797' then 1
        else 0 end) as Seattle_publisher,
        
	SUM(case
        When deweyClass = '741.5981' then 1
        else 0 end) as Brazil,
    
    SUM(case
        When deweyClass = '741.5982' then 1
        else 0 end) as Argentina,
    
     SUM(case
        When deweyClass = '741.5983' then 1
        else 0 end) as Chile,
        
	SUM(case
        When deweyClass = '741.5993' then 1
        else 0 end) as New_Zealand,
    
	SUM(case
        When deweyClass = '741.5994' then 1
        else 0 end) as Australia
        	 SUM(case
        When deweyClass = '741.5951' then 1
        else 0 end) as Korea,
    
	 SUM(case
        When deweyClass = '741.5952' then 1
        else 0 end) as Japan,
    
	 SUM(case
        When deweyClass = '741.5953' then 1
        else 0 end) as Palmiotti_Jimmy,
        
     SUM(case
        When deweyClass = '741.5954' then 1
        else 0 end) as India,
        
      SUM(case
        When deweyClass = '741.5955' then 1
        else 0 end) as Iran,
        
	 SUM(case
        When deweyClass = '741.5956' then 1
        else 0 end) as Israel,
     
     
     SUM(case
        When deweyClass = '741.59573' then 1
        else 0 end) as Hernandez_Jaime_Dean_Koontz_Graphic_Novel,
    
     SUM(case
        When deweyClass = '741.5959' then 1
        else 0 end) as Malaysia,
        
	 SUM(case
        When deweyClass = '741.5962' then 1
        else 0 end) as Egypt,
	
     SUM(case
        When deweyClass = '741.59666' then 1
        else 0 end) as Cote_dIvoire,

	SUM(case
        When deweyClass = '741.5968' then 1
        else 0 end) as South_Africa
            
FROM
    spl2.inraw
WHERE
	
			DATE(cout) >= '2006-01'
            and DATE(cout) < '2014-02'
		
            
GROUP BY
  YEAR(cout) MONTH(cout)


Code: Select all

/**
 * ControlP5 with PeasyCam support. tested with peasy 0.8.2
 *
 * by jeffg 2011
 */
 
//created by Mohit Hingorani for MAT259 Winter 2015
//edited by Intae Hwang for MAT259 W 2015 homework4

/*
A simple 3D demo to introduce the concepts of 3D graphics in processing
Introducing only peasycam

/**
 * ControlP5 Controller on top of 3D
 * demonstrates how to use controlP5 controllers on top of a 
 * OpenGL 3D scene.
 * by Andreas Schlegel, 2011
 * www.sojamo.de/libraries/controlP5
http://www.sojamo.de/libraries/controlP5/examples/extra/ControlP5withPeasyCam/ControlP5withPeasyCam.pde 
press n , l , t for nubers, lines and text repectively

*/
import peasy.test.*;
import peasy.org.apache.commons.math.*;
import peasy.*;
import peasy.org.apache.commons.math.geometry.*;

import controlP5.*;

import igeo.*;


PeasyCam cam;
ControlP5 cp5;

int buttonValue = 1;

// setting up fonts for text
PFont  myHelvetica12 = createFont("Helvetica",12, true);
PFont  myHelvetica20 = createFont("Helvetica",20, true);
PFont  myHelvetica48 = createFont("Helvetica",48, true);
//to store the data table
int [][] dataMatrix = null;

// to store the maximum possible value
int maxCount; 

// intialzing number or rows & columns , can be modified later
int numRows;
int numColumns;

//  text formatting variables 
float constantMarginOnRows = 160;
float constantMarginOnColumns = 100;
Table table; 

float blockWidth = 100;
float blockHeight = 100;   // initial 80

boolean textFlag= false;
boolean numberFlag = false;
boolean lineFlag = false; 

boolean y2006 = true;
boolean y2007 = true;
boolean y2008 = true;
boolean y2009 = true;
boolean y2010 = true;
boolean y2011 = true;
boolean y2012 = true;
boolean y2013 = true;

int ballelevation = 200;
int ballelevation1 = 0;
int checkout = 0;
boolean setMouseControlled = true;

String [] Month = {"Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul",
                   "Aug", "Sep", "Oct", "Nov", "Dec"};

int textColor = #4D2315;

String [] countries = {"United States",  "Japan",  "France",
      "Belgium","England","Canada","Italy","British","Korea","Mexico",
      "Norway","Finland","Cote dIvoire","Australia","Iran","Swiss",
      "Scotland","Seattle publisher","Sweden","Germany","Netherlands",  
      "South Africa","Palmiotti Jimmy","Malaysia","India","Spain",
      "Brazil","Argentina","Chile","Eastern Europe","Egypt",
      "Northern Europe", "New Zealand", "Israel"};
void text(float theValue) {
  textFlag= !textFlag;
}

void grid(float thevalue){
   lineFlag = !lineFlag;
}

void number(float thevalue){
   numberFlag = !numberFlag;
}

void y2006(float thevalue){
   y2006 = !y2006;
}

void y2007(float thevalue){
   y2007 = !y2007;
}

void y2008(float thevalue){
   y2008 = !y2008;
}

void y2009(float thevalue){
   y2009 = !y2009;
}

void y2010(float thevalue){
   y2010 = !y2010;
}

void y2011(float thevalue){
   y2011 = !y2011;
}

void y2012(float thevalue){
   y2012 = !y2012;
}

void y2013(float thevalue){
   y2013 = !y2013;
}

void controlEvent(ControlEvent theEvent) {
  //println(theEvent.getController().getId());
}

void draw() {
colorMode(RGB,100);
  background(12);
  translate( -width/2, -height/2, -1300);   // Peasycam intially sets lookAt() at (0,0,0) hence we need to transalte the axis
  pushMatrix();
 if ( lineFlag)
      {
      stroke(textColor,128);
      for ( int i =0 ; i <= blockWidth * (12) ; i+= blockWidth)
          {
           pushMatrix();                                                  //setting up a matrix to keep the current graphi coordinates
           translate(constantMarginOnRows, constantMarginOnColumns);      //translate to the correct position
           line( i,0,0,i, blockHeight * (numColumns-1)  ,0);              //creating a line (x,y,z)
           popMatrix();                                                   // going back to previos projection matrix   
          }           
      for ( int i = 0 ; i < blockHeight* numColumns ; i+= blockHeight)
          {
           pushMatrix();
           translate(constantMarginOnRows, constantMarginOnColumns); 
           line ( 0,i,0,blockWidth* 12 ,i,0);
           popMatrix();
          }
      }
      
      for ( int  i = 0 ; i< numRows ; i++)
      {  
        for ( int  j= 0 ; j< numColumns ; j++)
            {
              if( dataMatrix[i][j]!=0)
                {
                  int zTranslation = (dataMatrix[i][j]*255/ maxCount);
                  //println(zTranslation);
                  noStroke();
                  colorMode(HSB,100);
                  fill (dataMatrix[i][j],dataMatrix[i][j],dataMatrix[i][j], 92 );                    // setting up a color for the box
                  pushMatrix();                                                          // doing the same for all the boxes, sadly that is the only way to do this
                  translate(constantMarginOnRows, constantMarginOnColumns);    
                                   
                  if(i < 12 && y2006){
                  translate( i* blockWidth + blockWidth/2,  
                  j* blockHeight + blockHeight/2, 0*ballelevation+zTranslation*checkout);                                                                                                                                         // note the movement in z position, it is half of the box's z length, this is because the box is drawn at the center                                               
                  sphere(10);   
                  }                                  
                  if(i > 11 && i < 24 && y2007){
                  translate( (i-12)* blockWidth + blockWidth/2,  
                  j* blockHeight + blockHeight/2, 2*(ballelevation+ballelevation1)+zTranslation*checkout);                                                                                                                                         // note the movement in z position, it is half of the box's z length, this is because the box is drawn at the center                                               
                  sphere(10);   
                  }                
                  if(i > 23 && i < 36 && y2008){
                  translate( (i-24)* blockWidth + blockWidth/2,  
                  j* blockHeight + blockHeight/2, 4*(ballelevation+ballelevation1)+zTranslation*checkout);                                                                                                                                         // note the movement in z position, it is half of the box's z length, this is because the box is drawn at the center                                               
                  sphere(10);   
                  }
                  if(i > 35 && i < 48 && y2009){
                  translate( (i-36)* blockWidth + blockWidth/2,  
                  j* blockHeight + blockHeight/2, 6*(ballelevation+ballelevation1)+zTranslation*checkout);                                                                                                                                         // note the movement in z position, it is half of the box's z length, this is because the box is drawn at the center                                               
                  sphere(10);   
                  }
                  if(i > 47 && i < 60 && y2010){
                  translate( (i-48)* blockWidth + blockWidth/2,  
                  j* blockHeight + blockHeight/2, 8*(ballelevation+ballelevation1)+zTranslation*checkout);                                                                                                                                         // note the movement in z position, it is half of the box's z length, this is because the box is drawn at the center                                               
                  sphere(10);   
                  }
                  if(i > 59 && i < 72 && y2011){
                  translate( (i-60)* blockWidth + blockWidth/2,  
                  j* blockHeight + blockHeight/2, 10*(ballelevation+ballelevation1)+zTranslation*checkout);                                                                                                                                         // note the movement in z position, it is half of the box's z length, this is because the box is drawn at the center                                               
                  sphere(10);   
                  }
                  if(i > 71 && i < 84 && y2012){
                  translate( (i-72)* blockWidth + blockWidth/2,  
                  j* blockHeight + blockHeight/2, 12*(ballelevation+ballelevation1)+zTranslation*checkout);                                                                                                                                         // note the movement in z position, it is half of the box's z length, this is because the box is drawn at the center                                               
                  sphere(10);   
                  }
                  if(i > 83 && i < 96 && y2013){
                  translate( (i-84)* blockWidth + blockWidth/2,  
                  j* blockHeight + blockHeight/2, 14*(ballelevation+ballelevation1)+zTranslation*checkout);                                                                                                                                         // note the movement in z position, it is half of the box's z length, this is because the box is drawn at the center                                               
                  sphere(10);   
                  }
                               
                  if (numberFlag)
                      {
                       // box values
                       translate( 0,0, zTranslation*10);     // translating coordiantes to box position                      
                       textFont(myHelvetica12);
                       textAlign(CENTER,CENTER);
                       fill(0);       
                       text( dataMatrix[i][j] , 0,0  );
                      }     
                  popMatrix();
                }
            }
      }

    //supporting text
    // nochanges required here. As this is being drawn at the base plane
   
   if ( textFlag)
     {
     for ( int  i = 0 ; i< numColumns-1 ; i++)   // y varies
        {
          textFont(myHelvetica48);
          textAlign(RIGHT,CENTER);
           fill(textColor);
          text( countries[i], constantMarginOnRows*0.75 ,constantMarginOnColumns + blockHeight * i  +  blockHeight/2 );    
        }
               
     for ( int  i = 0 ; i < 12 ; i++)   // y varies
        {
          textFont(myHelvetica48);
          textAlign(CENTER,CENTER);
           fill(textColor); 
         
          text(Month[i], constantMarginOnRows + blockWidth*i+ 50 , constantMarginOnColumns/2);        
        }
     /*
     for ( int  i = 2006 ; i< 2014 ; i++)   // y varies
        {
          textFont(myHelvetica48);
          textAlign(CENTER,CENTER);
          fill(textColor);  
          text(i, constantMarginOnRows + blockWidth + 50 , constantMarginOnColumns/2, (i-2006)*2);        
        }
        */   
    textFont(myHelvetica48);
    textAlign(LEFT ,CENTER);
     fill(textColor);
    text( "3D Graph: Comicbook checkouts by countries: 2006 - 2013 " , constantMarginOnRows , constantMarginOnColumns/2-50 );   
     } 
  popMatrix();
  // makes the gui stay on top of elements
  // drawn before.
 
  gui();
 
  
   
  if(mouseX > 0  && mouseX < 261 && mouseY > 749 && mouseY < 840){
    cam.setMouseControlled(false);
  } else {
  cam.setMouseControlled(true);
  }
   
 /*
   cp5.addSlider("ballelevation1")
     .setPosition(30,750)
     .setSize(200,20)
     .setRange(0,200)
     .setValue(0)
     ;
     
     cp5.addSlider("checkout")
     .setPosition(30,780)
     .setSize(200,20)
     .setRange(0,200)
     .setValue(0)
     ;
  */
  
}

void gui() {
  hint(DISABLE_DEPTH_TEST);
  cam.beginHUD();
  cp5.draw();
  cam.endHUD();
  hint(ENABLE_DEPTH_TEST);
}

void setup() {
  size(1400, 840, OPENGL);
  cam = new PeasyCam(this, 100);
  cp5 = new ControlP5(this);
  
  cp5.addButton("text", 4, 30, 30, 90, 40).setId(1);
  cp5.addButton("grid", 4, 30, 80, 90, 40).setId(2);
  cp5.addButton("number", 4, 30, 130, 90, 40).setId(3);
  cp5.addButton("y2006", 4, 30, 200, 40, 30).setId(4);
  cp5.addButton("y2007", 4, 30, 235, 40, 30).setId(5);
  cp5.addButton("y2008", 4, 30, 270, 40, 30).setId(6);
  cp5.addButton("y2009", 4, 30, 305, 40, 30).setId(7);
  cp5.addButton("y2010", 4, 30, 340, 40, 30).setId(8);
  cp5.addButton("y2011", 4, 30, 375, 40, 30).setId(9);
  cp5.addButton("y2012", 4, 30, 410, 40, 30).setId(10);
  cp5.addButton("y2013", 4, 30, 445, 40, 30).setId(11);
  
  cp5.addSlider("ballelevation1")
     .setPosition(30,750)
     .setSize(200,20)
     .setRange(-200,200)
     .setValue(0)
     ;
     
     cp5.addSlider("checkout")
     .setPosition(30,780)
     .setSize(200,20)
     .setRange(0,70)
     .setValue(0)
     ;
  

  cp5.setAutoDraw(false);
  
  //if (frame != null) {
  //  frame.setResizable(true);          // resizable window
  //}
  
  //setting up the camera
  //cam= new PeasyCam( this,0,0,0,1000);       
  cam.setMinimumDistance(50);        // how near the camera can get 0,0,0
  cam.setMaximumDistance(13000);       // how far the camera can travel
               
  table = new Table();                 //allocating memory to new table 
  table = loadTable("data_all_06-13_all_month.csv", "header");  //loading table
  
  numRows = table.getRowCount();        //getting number of rows in the table
  numColumns = table.getColumnCount();  //getting the number of columns in table
  
  dataMatrix = new int [numRows][numColumns];  //allocating memory to the internal 2D array
  
  //copying the table data into 2D array
  for ( int  i = 0 ; i< numRows ; i++)
      {
        for ( int  j= 0 ; j< numColumns ; j++)
            {
              dataMatrix[i][j]  = table.getInt(i,j);          // - 1 as the first column is the day number             
              println("data value: "+ dataMatrix[i][j]);              
            }
      }
      
   println("data transferred");
   
   //finding the maximum count in the data
   
   maxCount = dataMatrix[0][0];   
    for ( int  i = 0 ; i< numRows ; i++)
      {
        for ( int  j= 1 ; j< numColumns ; j++)
            {
             if(maxCount<dataMatrix[i][j])
              {
                maxCount=dataMatrix[i][j];
              }
              
            }
      }
      
   println ( "maximum value is:"+ maxCount);
   println ( "rowCount: "+ numRows);
   println ( "columnCount: "+ numColumns);
   
}

British, '741.5941'
Scotland '741.59411'
England 741.5942'
Germany '741.5943'
France '741.5944'
Italy '741.5945'
Spain '741.5946'
Eastern_Europe '741.5947'
Northern_Europe '741.5948'
Norway '741.59481'
Sweden '741.59485'
Finland '741.59489'
Netherlands '741.59492'
Belgium '741.59493'
Swiss '741.59494'

Korea '741.5951'
Japan '741.5952'
Palmiotti_Jimmy '741.5953'
India '741.5954'
Iran '741.5955'
Israel '741.5956'
Hernandez_Jaime_Dean_Koontz_Graphic_Novel '741.59573'
Malaysia '741.5959'

Egypt '741.5962'
Cote_dIvoire '741.59666'
South_Africa '741.5968' then 1

Canada '741.5971'
Mexico ‘741.5972'
United_States '741.5973'
Walt_Disney '741.59794'
Seattle_publisher '741.59797'

Brazil '741.5981'
Argentina '741.5982'
Chile '741.5983'
New_Zealand '741.5993'
Australia '741.5994'
Attachments
data_all_06-13_all_month.csv
(8.33 KiB) Downloaded 271 times
datahomework1_4.zip
(1.51 MiB) Downloaded 270 times
Last edited by intae on Sun Feb 22, 2015 6:27 pm, edited 1 time in total.

james_schaffer
Posts: 5
Joined: Sat Jan 10, 2015 11:34 am

Re: Proj 4: 3D Volumetric, Spacial Visualization

Post by james_schaffer » Thu Feb 19, 2015 1:41 am

Tantalizing Clusters of Controversy UPDATED

For this phase of the project, I shelved the controversy score used previously and decided to use an unsupervised learning technique to determine if checkout behavior could indicate controversial - and therefore interesting - books. Unfortunately, it looks like SPL1 was taken down between this project and the last, so I no longer had access to page numbers and ratings, but I passed the deviation of checkout duration, topic, and popularity (checkout quanitity) to a k-means (https://code.google.com/p/kmeansclustering/) and heirarchical clustering algorithm (https://github.com/lbehnke/hierarchical-clustering-java). Heirarchical clustering proved far too slow for more than 4000 books, so it was ditched for the former.

K-Means clustering yielded approximately 20 clusters over deviation of checkout time, topic, and popularity for books in the philosophy category (total 7500, taken after 2005). In the visualization, deviation is plotted on the x-axis, logarithmic popularity on the y-axis, and topic on the z-axis. Each dot is a book, larger dots represent the cluster's centroid, and each cluster has a different color. Some interesting clusters can be seen in the high deviation and popularity range.
screen1.png
screen2.png
Clustering is an NP-hard problem, and k-means clustering is merely a heuristic that is prone to error. In general, visualizing the clusters yielded useful feedback for running the original k-means clustering. At first, the clusters seemed oddly shaped, and a bit of investigation showed that the original algorithm needed normalized values to work properly. Once that was corrected, drawing a line between each point and the mean of the cluster shows a few more surprisingly shaped clusters that could perhaps be improved through further manipulation of the k-means input.
screen3.png
Additionally, animation that cycles through the clusters was added.

UPDATE: Visuals have been tuned using additive image blending for a more arresting visual. Numbers 1-5 can be pressed for pre-defined camera views. X-Y-Z scales have been added.

From the visualization, we can draw a few conclusions.
1. Checkout quantity and the deviation of the checkout duration are correlated. Interesting books would therefore have a high deviation relative to their popularity.
2. The clustering is not perfect. The clusterer made "slices" of the data in the X and Z directions, clustering together data that is far apart (more clusters would be necessary to get a better fit). Using the 'spinies', poor clusters can quickly be identified.
3. Cluster 8 contains the most popular books with the highest checkout deviation (even though it is a split cluster). Although interaction is not implemented, the original dataset can be used to yield the book list, here are a few selections:

8, "Watch me grow Im one every parents guide to the enchanting 12 to 24 month old"
8, "Toward a psychology of being"
8, "Full catastrophe living using the wisdom of your body and mind to face stress pain and illness"
8, "Crucial conversations tools for talking when stakes are high"
8, "Awakening from grief finding the road back to joy"

Once again, the project is too large to attach. Try http://128.111.28.122/controversy_v4.zip "SeattleControversy" contains the Java project that was used to cluster the data before being passed to processing.
Last edited by james_schaffer on Sun Mar 08, 2015 7:49 pm, edited 1 time in total.

brocknoah
Posts: 5
Joined: Sat Jan 10, 2015 11:36 am

Re: Proj 4: 3D Volumetric, Spacial Visualization

Post by brocknoah » Thu Feb 19, 2015 2:35 am

My previous visualization showed how weekly checkouts of a subclass compared to its siblings within a larger parent class. I wanted to continue a similar path of ranking an item in a group. For this next search I would narrow down to a topic that might change over time. I wanted to avoid a situation with consistent polar opposites, but I wouldn't know until I searched.

I searched through 190 Modern Western Philosophy, and its 10 subclasses. Other topics I would like to explore are 640 Home & family management and 680 Manufacture for specific uses.

Instead of tracking weekly checkouts for one year, I capture an instance of a "ranking order" for each month for 6 years. Plot these points to track changes over time. In a boring system with no change or variation, each dewey class would fill a flat horizontal plane. A consistent checkout rate compared to others, each class would stay in the same place month after month.

You can toggle the size of the sphere. The size is either related to the number of checkouts, or all sizes are equal (expect 0 checkouts). You can set all spheres to be the same to focus more on the change over time compared to others, not compared to the number of checkouts. You also have the option to view a particular year or all years at once.

This query could improve if linked with a search engine. Scan news archives with the checkout dates and geographical area to see popular events.

SQL Query

Code: Select all

SELECT
SUM(CASE WHEN month(cout) = 1 THEN 1 ELSE 0 END) as January,
SUM(CASE WHEN month(cout) = 2 THEN 1 ELSE 0 END) as February,
SUM(CASE WHEN month(cout) = 3 THEN 1 ELSE 0 END) as March,
SUM(CASE WHEN month(cout) = 4 THEN 1 ELSE 0 END) as April,
SUM(CASE WHEN month(cout) = 5 THEN 1 ELSE 0 END) as May,
SUM(CASE WHEN month(cout) = 6 THEN 1 ELSE 0 END) as June,
SUM(CASE WHEN month(cout) = 7 THEN 1 ELSE 0 END) as July,
SUM(CASE WHEN month(cout) = 8 THEN 1 ELSE 0 END) as August,
SUM(CASE WHEN month(cout) = 9 THEN 1 ELSE 0 END) as September,
SUM(CASE WHEN month(cout) = 10 THEN 1 ELSE 0 END) as October,
SUM(CASE WHEN month(cout) = 11 THEN 1 ELSE 0 END) as Novermber,
SUM(CASE WHEN month(cout) = 12 THEN 1 ELSE 0 END) as December
FROM spl2.inraw
WHERE year(cout) ="2007"
AND (itemtype LIKE "%bk")
AND deweyClass is not null
AND deweyClass >= 190 AND deweyClass <200
group BY floor(deweyClass) order by floor(deweyClass) ASC
Year was changed from 2007 - 2012
Attachments
p4.zip
(7.21 KiB) Downloaded 220 times
8349.jpg
6954.jpg
2701.jpg
0705.jpg
Last edited by brocknoah on Thu Feb 19, 2015 8:49 pm, edited 2 times in total.

younkeehong
Posts: 4
Joined: Sat Jan 10, 2015 11:36 am

Re: Proj 4: 3D Volumetric, Spacial Visualization

Post by younkeehong » Thu Feb 19, 2015 5:16 am

Keehong Youn.

PILE of BOOKS

what time of the year do you feel wisdom-less?
When end of the year comes? When new year starts?
or when the leaves start to fall?
and for that, especially what topic of philosophy?

this visualization will show number of times loaned on z axis,
w.r.t. the specific subject within category of philosophy
and month in year.

also, longer the items loaned they will be more red-tinted.

interesting fact found is that while there's no significant change throughout the year,
loan rate throughout the topics vary very much.
for example, while induction & deduction related books were not loaned almost at all,
books related to conscious&subconscious topics were very often loaned.

for the visual part, I tried to give impression of piled books,
and also some of those flying herd of books like those in Harry Potter kind-of films.
So I got a texture on data cube while giving a bit of misalignment.
(which is not as big as to give misconception about data)
also put a background image of a bookshelf.
I know the background might distract the focusing to the data,
but it seems the whole scene is in good harmony.
Screen Shot 2015-02-18 at 23.00.22.png
Screen Shot 2015-02-18 at 22.59.21.png
Screen Shot 2015-02-18 at 22.58.46.png
Screen Shot 2015-02-18 at 22.57.58.png
Attachments
kh_hw4.zip
processing code
(980.1 KiB) Downloaded 210 times

jmd
Posts: 5
Joined: Sat Jan 10, 2015 11:26 am

Re: Proj 4: 3D Volumetric, Spacial Visualization

Post by jmd » Thu Feb 19, 2015 10:18 am

Autobiographies

This project explores the interest in autobiographies over the years. The code allows to sort this interest per higher year of activity and higher month of activity.

Query:

Code: Select all

SELECT 
    MONTH(cout),
    MONTH(cout),
    YEAR(cout),
    SUM(CASE
        WHEN title LIKE '%autobiography' THEN 1
        ELSE 0
    END) AS autobio

FROM
    spl2.inraw
WHERE
     DATE(cout) >= '2006-01-01'
        AND DATE(cout) <= '2013-12-31'
GROUP BY MONTH(cout) , YEAR(cout)
ORDER BY YEAR(cout) , MONTH(cout)
Images:
autobiography.png
detail.png
The data of each cell can be activated or deactivated with ’t’. Every checked out item in the database is being drawn as a particle within its own cell limits. I include the final image and another one to show how the 3D space can be understood once the cell data is shown.

(Databases included in the *.csv file with some more captures).
Attachments
three_dimensional_grid_01.zip
Processing code
(219.93 KiB) Downloaded 207 times
report_homework_1_2_3.pdf
Previous work
(187.36 KiB) Downloaded 341 times

Post Reply