11.21 Status of Work-in-Progress

Transforming Data: Cultural Strategies in DataMining
Instructor: George Legrady
Locked
glegrady
Posts: 160
Joined: Wed Sep 22, 2010 12:26 pm

11.21 Status of Work-in-Progress

Post by glegrady » Fri Nov 18, 2011 7:41 am

Post here a brief description of the work-in-progress. What is the status of the project's development? Is the work proceeding according to the schedule, or will you be re-adjusting the schedule?
George Legrady
legrady@mat.ucsb.edu

bowea324
Posts: 13
Joined: Tue Sep 13, 2011 10:58 am

Re: 11.21 Status of Work-in-Progress

Post by bowea324 » Fri Nov 18, 2011 1:06 pm

I plugged my idea of looking for trends in political texts prior to the 2008 election cycle into part of the visualization code from last week and the results were definitely of the 'interesting sort'. From 2005- 2008 it is easily recognizable that there is a growing trend in political texts checked out at the library every year. The basic result shows me that this project is worth pursuing and offers me a little leverage in building this analysis of political texts. I can draw conclusions that assert that people are using libraries for political research, that the libraries are attempting to assist in political research, or that the rise of political texts available are source-able from the library.

import de.bezier.data.sql.*;

MySQL msql;

void setup()
{
size( 600, 600 );
background(255);

String user = "mat259";
String pass = "V1sual1zat1on";

/*String [] monthLables = new String[12];
monthLables[0] = "Jan";
monthLables[1] = "Feb";
monthLables[2] = "Mar";
monthLables[3] = "Apr";
monthLables[4] = "May";
monthLables[5] = "Jun";
monthLables[6] = "Jul";
monthLables[7] = "Aug";
monthLables[8] = "Sep";
monthLables[9] = "Oct";
monthLables[10] = "Nov";
monthLables[11] = "Dec";*/
// regular way of defining an array

String [] monthLables = {"Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec"}; // inline version of defining an array (same as above)

String database = "spl_years";

msql = new MySQL( this, "tango.mat.ucsb.edu", database, user, pass );

if ( msql.connect() )
{
msql.query( "select month(ckoutDateTime), count(*) from transactions2005 where title like '%politics%' "+
"group by month(ckoutDateTime) order by month(ckoutDateTime)" );

int monthCount = 0;

while(msql.next()) {

int amonth = msql.getInt(1);
int count = msql.getInt(2);

fill(255 , 50, 2, 50);
stroke(0 ,50);
rect( monthCount * (width/12) , height , width /12 , count * -0.5);
println(count);

fill(0);
textAlign(CENTER);
text(monthLables[monthCount], 20+ monthCount * (width/12) , 550);

monthCount++;



}

msql.query( "select month(ckoutDateTime), count(*) from transactions2006 where title like '%politics%' "+
"group by month(ckoutDateTime) order by month(ckoutDateTime)" );

int month2Count = 0;

while(msql.next()) {

int amonth = msql.getInt(1);
int count = msql.getInt(2);

fill(255, 10, 90 , 50);
stroke(0 ,50);
rect( month2Count * (width/12) , height , width /12 , count * -0.5);
println(count);

fill(255);
textAlign(CENTER);
text(monthLables[month2Count], 20+ month2Count * (width/12) , 550);

month2Count++;


}

msql.query( "select month(ckoutDateTime), count(*) from transactions2007 where title like '%politics%' "+
"group by month(ckoutDateTime) order by month(ckoutDateTime)" );

int month3Count = 0;

while(msql.next()) {

int amonth = msql.getInt(1);
int count = msql.getInt(2);

fill(255, 52, 88 , 50);
stroke(0 ,50);
rect( month3Count * (width/12) , height , width /12 , count * -0.5);
println(count);

fill(255);
textAlign(CENTER);
text(monthLables[month3Count], 20+ month3Count * (width/12) , 550);

month3Count++;

}


msql.query( "select month(ckoutDateTime), count(*) from transactions2008 where title like '%politics%' "+
"group by month(ckoutDateTime) order by month(ckoutDateTime)" );

int month4Count = 0;

while(msql.next()) {

int amonth = msql.getInt(1);
int count = msql.getInt(2);

fill(255, 27, 300 , 50);
stroke(0 ,50);
rect( month4Count * (width/12) , height , width /12 , count * -0.5);
println(count);

fill(255);
textAlign(CENTER);
text(monthLables[month4Count], 20+ month4Count * (width/12) , 550);

month4Count++;

}
}
else
{
println( "Conection Failed!! ");
}
}

void draw()
{

}

alessandrarosecampos
Posts: 9
Joined: Tue Sep 13, 2011 11:09 am

Re: 11.21 Status of Work-in-Progress

Post by alessandrarosecampos » Sun Nov 20, 2011 4:58 pm

My primary goal for this week was to identify some potential articles, book, and proposals to look at in conjunction with Manovich's cultural analytics proposal. So far I'm looking at NSF's Cyberinfrastrucuture Vision Report published in 2007 (which Manovich mentions as a basis for the technology he intends to use in implementing his proposal--for example, petascale computing), James Thomas and Kristina Cook's "A Visual Analytics Agenda" (which uses the examples of 9/11 and Hurricane Katrina to emphasize the importance of analyzing and drawing conclusions from real-time, large data sets), the American Society of Learned Societies Commission's report of Cyberinfrastructure for the humanities and social sciences, Manovich's "Visual Techniques for Exploring Large Media Collections", Choamei Chen's "The Impact of the Work of Jim Thomas" (who was influential in the development of the field of visual analytics), the Google Analytics website, and Andrejevic's "iSpy: Surveillance and Power in the Interactive Era." So far this is quite a bit, so I'd like to refine my list and also add a bit more theoretical research to the mix.

putzb642
Posts: 10
Joined: Tue Sep 13, 2011 11:02 am

Re: 11.21 Status of Work-in-Progress

Post by putzb642 » Mon Nov 21, 2011 11:21 am

Progress report:

Currently I have all of the NEH grant reports for all of the 2000's. I began looking at Ben Fry's zip code processing code. This is giving me a good idea where to start once I start parsing the XML data from data.gov. Along with playing with that code I began working on getting the XML data parsed using the ofxXmlSettings add on for openFrameworks.

I have been toying with different designs that this could possibly end up looking like. One that I prefer is the idea of viewing a map of the US with he more dense colors representing the areas in the US that receive the most grants. This would be determined from the Zip Code information that is in the XML data. Another design I have been thinking about and sketching how it would work well would be design a Mondrian style design determined once again by the amount of grants. With both of these the way the information can be represented would be in the interim period with a mouse roll over action with the hopes of integrating openCV and allow motion to be the action that drives the information interaction.

As of now the current trajectory of the schedule is on course and next week should have a working prototype of one of the current designs (most likely the map, possible an abstract representation).

bowea324
Posts: 13
Joined: Tue Sep 13, 2011 10:58 am

Re: 11.21 Status of Work-in-Progress

Post by bowea324 » Mon Nov 21, 2011 11:43 am

First, in order to investigate political research in mass culture, I located the political texts that were most important or most popular during the 2008 election cycle. I narrowed my search to only include texts from the Federal Presidential Election, hoping to draw conclusions on the use of libraries for political research based on these results.

1) I ran a search within amazon.com and the new york times best seller list to discover the key texts for 2007 (just before the election):

http://www.amazon.com/Best-Political-Bo ... 8EJ30I17C1


I began to run a search to look at the basic movement of certain key political texts:

My first search was to look at the number of check outs for Al Gore’s text “The Assault on Reason.” Al Gore has a large reputation for his framing of political arguments prior to elections.

Here is my SQL search to sort the data by number of checkouts per month for the barcode indicated by the Seattle Public Library website:

select month(ckoutDateTime), count(*) from transactions2007
where title like '%assault on reason%' group by month(ckoutDateTime) order by month(ckoutDateTime)

I cleaned my search by adding the title to the search:

select month(ckoutDateTime), title, count(*) from transactions2007
where title like '%assault on reason%' group by month(ckoutDateTime) order by month(ckoutDateTime)

This process includes all of the CDs, DVDs, and books that were checked out in affiliation with Al Gore’s title:

select month(ckoutDateTime), title, count(*) from transactions2007
where title like '%assault on reason%' group by month(ckoutDateTime) order by month(ckoutDateTime)

In order to evaluate the number of checkouts for other key texts, I will continue this process and evaluate the results.

I have also indicated the following visualization code, in order to display the rise in checkouts for key political texts from 2004 – 2008, the results on the graph are grouped by month:

import de.bezier.data.sql.*;

MySQL msql;

void setup()
{
size( 600, 600 );
background(255);

String user = "mat259";
String pass = "V1sual1zat1on";

/*String [] monthLables = new String[12];
monthLables[0] = "Jan";
monthLables[1] = "Feb";
monthLables[2] = "Mar";
monthLables[3] = "Apr";
monthLables[4] = "May";
monthLables[5] = "Jun";
monthLables[6] = "Jul";
monthLables[7] = "Aug";
monthLables[8] = "Sep";
monthLables[9] = "Oct";
monthLables[10] = "Nov";
monthLables[11] = "Dec";*/
// regular way of defining an array

String [] monthLables = {"Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec"}; // inline version of defining an array (same as above)

String database = "spl_years";

msql = new MySQL( this, "tango.mat.ucsb.edu", database, user, pass );

if ( msql.connect() )
{
msql.query( "select month(ckoutDateTime), count(*) from transactions2005 where title like '%politics%' "+
"group by month(ckoutDateTime) order by month(ckoutDateTime)" );

int monthCount = 0;

while(msql.next()) {

int amonth = msql.getInt(1);
int count = msql.getInt(2);

fill(255 , 50, 2, 50);
stroke(0 ,50);
rect( monthCount * (width/12) , height , width /12 , count * -0.5);
println(count);

fill(0);
textAlign(CENTER);
text(monthLables[monthCount], 20+ monthCount * (width/12) , 550);

monthCount++;



}

msql.query( "select month(ckoutDateTime), count(*) from transactions2006 where title like '%politics%' "+
"group by month(ckoutDateTime) order by month(ckoutDateTime)" );

int month2Count = 0;

while(msql.next()) {

int amonth = msql.getInt(1);
int count = msql.getInt(2);

fill(255, 0, 0 , 50);
stroke(0 ,50);
rect( month2Count * (width/12) , height , width /12 , count * -0.5);
println(count);

fill(255);
textAlign(CENTER);
text(monthLables[month2Count], 20+ month2Count * (width/12) , 550);

month2Count++;


}

msql.query( "select month(ckoutDateTime), count(*) from transactions2007 where title like '%politics%' "+
"group by month(ckoutDateTime) order by month(ckoutDateTime)" );

int month3Count = 0;

while(msql.next()) {

int amonth = msql.getInt(1);
int count = msql.getInt(2);

fill(255, 0, 0 , 50);
stroke(0 ,50);
rect( month3Count * (width/12) , height , width /12 , count * -0.5);
println(count);

fill(255);
textAlign(CENTER);
text(monthLables[month3Count], 20+ month3Count * (width/12) , 550);

month3Count++;

}


msql.query( "select month(ckoutDateTime), count(*) from transactions2008 where title like '%politics%' "+
"group by month(ckoutDateTime) order by month(ckoutDateTime)" );

int month4Count = 0;

while(msql.next()) {

int amonth = msql.getInt(1);
int count = msql.getInt(2);

fill(255, 0, 0 , 50);
stroke(0 ,50);
rect( month4Count * (width/12) , height , width /12 , count * -0.5);
println(count);

fill(255);
textAlign(CENTER);
text(monthLables[month4Count], 20+ month4Count * (width/12) , 550);

month4Count++;

}
}
else
{
println( "Conection Failed!! ");
}
}

void draw()
{

}



I will continue to develop these patterns, and draw conclusions from visualizations and data representations.

In terms of time line- because the vast majority of this project has to deal with data mining, I will extend the mining process for one more week and the writing will begin in December.

rileb685
Posts: 6
Joined: Tue Sep 13, 2011 11:02 am

Re: 11.21 Status of Work-in-Progress

Post by rileb685 » Mon Nov 21, 2011 12:02 pm

I've fallen slightly behind schedule-- I had hoped to have my SQL queries in Processing by now, but I'm still working on refining them in Sequel Pro. Once I have a full set of useful queries, bringing them into Processing will be very simple (the tricky thing after that is visualizing those queries in an interesting way)

I've been trying to search for terms used in any of the subject fields (subject1, subject2, subject3, etc.) with queries structured liked this:

select * from transactions2010 where subject1 or subject2 or subject3 or subject4 like '%augustus emperor of rome%';

This, however, turns up a huge number of false positives, where nothing even remotely similar to Augustus appears under any of the subject lines.

Just searching for a single subject field returns the expected results:
select * from transactions2010 where subject1 like '%augustus emperor of rome%';

However, many of the historical figures I'm searching for only occur in subject2 or subject3, so I need to figure out what's going wrong here.

I'm also conducting some informal polling (among people who read my blog, so it's not terribly scientific) about who immediately comes to mind in connection with ancient history. I'll then coordinate this with SPL data and possibly google NGrams to see how much the zeitgeist is reflected.

User avatar
keats047
Posts: 10
Joined: Tue Sep 13, 2011 11:00 am

Re: 11.21 Status of Work-in-Progress

Post by keats047 » Mon Nov 21, 2011 12:37 pm

I'm currently working my way through Kant and Dewey's aesthetics to give myself a grounded space for discussion of the work of visual complexity and Ward Shelley.
http://plato.stanford.edu/entries/kant-aesthetics/
http://plato.stanford.edu/entries/dewey-aesthetics/
http://www.amazon.com/Art-as-Experience ... 0399531971
http://www.amazon.com/Critique-Judgemen ... 0199552460

Along with this, I've also been examining various pieces, attempting to find objects to compare and contrast.
http://www.visualcomplexity.com/vc/proj ... domain=Art
http://www.wardshelley.com/paintings/pa ... 20copy.jpg

(Process of shaping here, questions of where and how things are placed and spaced)

http://www.wardshelley.com/paintings/pa ... 20copy.jpg
http://www.visualcomplexity.com/vc/proj ... 20Networks

(Systemic visualizing versus representative visualizing)

beth.c.carlson
Posts: 9
Joined: Tue Sep 13, 2011 11:10 am

Re: 11.21 Status of Work-in-Progress

Post by beth.c.carlson » Mon Nov 21, 2011 1:22 pm

I have decided to create language visualizations from other forms of technology, primarily twitter, and compare them to the SMS visualizations I did. I want to compare the visualizations in order to draw conclusions about how different technologies affect the language we choose to use, the shortening of words, etc.

I found this twitter data set and I am currently going through it to clean up the clutter, but it is taking a long time:

http://caw2.barcelonamedia.org/node/24

I will then use the text to make comparison visualizations.

User avatar
deklerk
Posts: 14
Joined: Tue Sep 13, 2011 11:10 am
Location: New York, NY
Contact:

Re: 11.21 Status of Work-in-Progress

Post by deklerk » Mon Nov 21, 2011 1:59 pm

So far, I am still on schedule with my project. The Voronoi Diagram is drawing as points are placed on the map sequentially. I've constrained the diagram to plot spaces between graffiti locations that the City was unable to remove.

Labeling has proved a little challenging. I want the labels to appear as matching location-types are plotted to the map. While this mostly works, the process exponentially slows as the visualization unfolds. I worked with Sepand today in an attempt to optimize the script.

Here is a screenshot from the project:

Image
-----
Christo

athenallewellyn
Posts: 6
Joined: Tue Sep 13, 2011 11:08 am

Re: 11.21 Status of Work-in-Progress

Post by athenallewellyn » Wed Dec 21, 2011 10:10 am

In looking at my various projects: the Newark Art parade, as well as my personal artistic projects I realize that the underlying intention of these works is to create a space of shared experience, and a sense of togetherness and interdependence. I decided to work at the root of this issue, teaching myself how to create these spaces better, through an investigation of the tarot card #21, the Universe. This card is the last of the major arcana, and represents the culmination of the fools journey, which begins at #0. This culmination represents a moment in development of balance, where there is no co-dependency but pure independence and interdependence. This state of interdependence is a goal in all my projects, and life. By studying this card, I hope to gain insight to attaining this state. A visual mapping project is new to me, and the process of gathering the data has been extremely revelatory.

Locked