Simple Histogram

Data Analysis with Python – The Tools and the Data

Welcome to the Inaugural Post for my Learning Python Series

The Post is the start of a series of walk throughs from Start to Finish of my journey into Data Analysis and Data Science with Python


The Tools and Loading the Data

What are the tools to download in order to get started building in Python

How do I load the data and construct the domain

How do I do some basic analysis on the data to get a feel for the relationships.

The Next Part

The next post will utilize Panda to perform quicker more structure data analysis.



The Tools

I downloaded and installed the Anaconda distribution along with the Visual Studio Python Tools

The Anaconda distribution is a python distribution that contains many many scientific libraries.

The Data

Fortunately there is a massive amount of data that you can have fun and experiment with.

The UCI Machine Learning Repository has a huge amount of data.


  • Mice Protein Expression
    • Expression levels of 77 proteins measured in the cerebral cortex of 8 classes of control and Down syndrome mice exposed to context fear conditioning, a task used to assess associative learning.
    • Mice Protein
  • Car Evaluation Data Set
    • Derived from simple hierarchical decision model, this database may be useful for testing constructive induction and structure discovery methods.
  • Adult Data Set
    • Predict whether income exceeds $50K/yr based on census data. Also known as “Census Income” dataset.

I am going to use the Adult Data Set in my Examples.


Loading the Data and Graphing the Data

Grab the Data from

This will print out the workclass column in the data.

Resulting in an output like

In order to start to looking into the data we can use someone of Python’s built in magic to bucket the data and create some histograms.

Histograms will tell us how the data is distributed and start to give us clues about the shape of the data.

In order to create a histogram we will use the collections library to count the data


Once I get the file

I take my agelist and run it through Counter.

Counter allows for rapid tallying of data.  It returns a defaultdict object that list each age and how many occurences there were.

We then unzip the list to labels and values.  * reverses the zip operation.

We then call the, values) to show the graph.simpleagedistribution

We can see that our data is a right-skewed distribution.  When you look at the data you can see it is evenly distributed over the income generating population.  At around 18 it starts and starts to gradually tail off at the peak of around 40.


First Refactoring – Making the code a bit more compact and readable

I mapped each row to a named tuple in order to iterate through the data a bit more intuitively

First I created a dictionary of the columns in the data.


I then created a object that would represent the named tuple.

I used the EconRecord and the map function to apply EconRecord._make to every record in the collection.  Creating a new econ record for each row in the file.

The result is being able is being to aggregate the items in a bit more cleanly with more concise readable code.

The Next Series we will start to scatter plot and look for relationships in the data using Panda.



Python Lists

The Book – Data Science from Scratch

Beautiful Plots With Pandas and Matplotlib


CSV  – Python

Python Structure 

Read More

New Router – Asus RT-N16 and Tomato Awesomeness

Jeff Atwood has an excellent article recommending this router.

It can be found here.

In the article is a guide to flashing the firmware with tomato.

The one caveat that I would add is.

You have to reset the NVRAM via key presses which means

    1. Power off the RT-N66U
    2. Press and hold down the WPS button
    3. While holding the WPS button, plug in the power cable to turn RT-N66Uon
    4. Keep holding the WPS button for 30 seconds before releasing
      The router should reboot
    5. Congratulations. The NVRAM has been cleared.

I was not able to reach the web address so I was not able to perform the operation via the web admin, so the key press option was my only option.


In performing this process. I found the instructions and distribution of Easy Tomato to be fantastic.

Easy Tomato is by far the best router firmware I have ever used. 


It makes the things that should be easy, easy and allows the advanced settings to be found easily.


A couple of examples

You can block any and all adult sites with a single checkbox.(bam)

You can define groups and drag the devices on your network into groups.  Yeah I said drag(holy web 2.0 interface batman)

Each group can then have access rules defined for it.


I highly recommend Easy Tomato and the Asus RT-N16 router.

Read More

Superior Biking Trip 2013


On October 28th my brother and I set out on a Bike ride from his house to Superior WI. Our plan was to take the Gandy Dancer Trail from Milltown to Superior.  We brought gear to camp and food to eat.  The bikes were geared up and ready to go.

Here is the Google tour that goes over the bike route.(Google Earth Plugin is Required)

Google Tour

Day One – To Danbury

We set out on the Gandy Dancer Trail

It is a nice limestone trail that runs from St. Croix to Danbury.

Originally we thought it was a nice bike trail all of the way up to Superior.

We were wrong.  It turns to crap at Danbury.


A nice river bridge over the Yellow River

We ended up camping a ways down on the Gandy Dancer after Danbury.  It was a pretty cold night.  I bet it was in the neighborhood of 30.

In the morning our bikes had a healthy layer of frost.

Day Two – Danbury to Superior

Since the Gandy Dancer was a rough four-wheeler trail from Danbury to Superior we decided to take highway 35 up to Superior.

Highway 35 is a state highway that runs on the Wisconsin side.  It is pretty nice road.

It was about 40 and flurries on the way up.  Sun would have been great, but it was better than rain.

WP_20131029_001 (1)

We stopped at the Drydock in Chaffey to warm up.

Day Three – Superior

We decided to take a two night break in Superior and rest up.

After a night of camping, and being next to a hotel. It seemed pretty silly to not stay in a hotel.



During our time we biked to Grandma’s saloon and had dinner with our niece Allison.

Day Four – Duluth – Moose Lake

We decided to take the Willard Munger Trail back.

It is a nice paved trail from Duluth to Hinckley.


Stopping to change into Rain gear


There is a really nice rail bridge over the St. Louis River.


There is a fantastic Sausage shop in Mahtowa

TJ’s Country Store

The Cheddar Wurst is fantastic.


We stayed over night in Moose Lake at the Moose Lake Hotel.

Day Five – Moose Lake – Hinckley

The next day was pretty short and ended at about 1:00pm

Mom picked us up at Grand Casino Hinckley.


Overall it was a nice ride. It was too bad that the Gandy Dancer Trail was not able to take us all of way up.

It would have been possible to take 77 over to Hinckley from Danbury and then we could have taken the Willard Munger there and back.

Maybe next time…

Read More

My Top 5 Strengths


You love to learn. The subject matter that interests you most will be determined by your other themes and experiences, but whatever the subject, you will always be drawn to the process of learning. The process, more than the content or the result, is especially exciting for you. You are energized by the steady and deliberate journey from ignorance to competence. The thrill of the first few facts, the early efforts to recite or practice what you have learned, the growing confidence of a skill mastered—this is the process that entices you. Your excitement leads you to engage in adult learning experiences—yoga or piano lessons or graduate classes. It enables you to thrive in dynamic work environments where you are asked to take on short project assignments and are expected to learn a lot about the new subject matter in a short period of time and then move on to the next one. This Learner theme does not necessarily mean that you seek to become the subject matter expert, or that you are striving for the respect that accompanies a professional or academic credential. The outcome of the learning is less significant than the “getting there.”


You look for areas of agreement. In your view there is little to be gained from conflict and friction, so you seek to hold them to a minimum. When you know that the people around you hold differing views, you try to find the common ground. You try to steer them away from confrontation and toward harmony. In fact, harmony is one of your guiding values. You can’t quite believe how much time is wasted by people trying to impose their views on others. Wouldn’t we all be more productive if we kept our opinions in check and instead looked for consensus and support? You believe we would, and you live by that belief. When others are sounding off about their goals, their claims, and their fervently held opinions, you hold your peace. When others strike out in a direction, you will willingly, in the service of harmony, modify your own objectives to merge with theirs (as long as their basic values do not clash with yours). When others start to argue about their pet theory or concept, you steer clear of the debate, preferring to talk about practical, down-to-earth matters on which you can all agree. In your view we are all in the same boat, and we need this boat to get where we are going. It is a good boat. There is no need to rock it just to show that you can.


Excellence, not average, is your measure. Taking something from below average to slightly above average takes a great deal of effort and in your opinion is not very rewarding. Transforming something strong into something superb takes just as much effort but is much more thrilling. Strengths, whether yours or someone else’s, fascinate you. Like a diver after pearls, you search them out, watching for the telltale signs of a strength. A glimpse of untutored excellence, rapid learning, a skill mastered without recourse to steps—all these are clues that a strength may be in play. And having found a strength, you feel compelled to nurture it, refine it, and stretch it toward excellence. You polish the pearl until it shines. This natural sorting of strengths means that others see you as discriminating. You choose to spend time with people who appreciate your particular strengths. Likewise, you are attracted to others who seem to have found and cultivated their own strengths. You tend to avoid those who want to fix you and make you well rounded. You don’t want to spend your life bemoaning what you lack. Rather, you want to capitalize on the gifts with which you are blessed. It’s more fun. It’s more productive. And, counterintuitively, it is more demanding.


You are inquisitive. You collect things. You might collect information—words, facts, books, and quotations—or you might collect tangible objects such as butterflies, baseball cards, porcelain dolls, or sepia photographs. Whatever you collect, you collect it because it interests you. And yours is the kind of mind that finds so many things interesting. The world is exciting precisely because of its infinite variety and complexity. If you read a great deal, it is not necessarily to refine your theories but, rather, to add more information to your archives. If you like to travel, it is because each new location offers novel artifacts and facts. These can be acquired and then stored away. Why are they worth storing? At the time of storing it is often hard to say exactly when or why you might need them, but who knows when they might become useful? With all those possible uses in mind, you really don’t feel comfortable throwing anything away. So you keep acquiring and compiling and filing stuff away. It’s interesting. It keeps your mind fresh. And perhaps one day some of it will prove valuable.


Your Achiever theme helps explain your drive. Achiever describes a constant need for achievement. You feel as if every day starts at zero. By the end of the day you must achieve something tangible in order to feel good about yourself. And by “every day” you mean every single day—workdays, weekends, vacations. No matter how much you may feel you deserve a day of rest, if the day passes without some form of achievement, no matter how small, you will feel dissatisfied. You have an internal fire burning inside you. It pushes you to do more, to achieve more. After each accomplishment is reached, the fire dwindles for a moment, but very soon it rekindles itself, forcing you toward the next accomplishment. Your relentless need for achievement might not be logical. It might not even be focused. But it will always be with you. As an Achiever you must learn to live with this whisper of discontent. It does have its benefits. It brings you the energy you need to work long hours without burning out. It is the jolt you can always count on to get you started on new tasks, new challenges. It is the power supply that causes you to set the pace and define the levels of productivity for your work group. It is the theme that keeps you moving.

Learned from

Read More

My Utility for one user is now deployed across the enterprise

You created a simple Utility to aid in doing your job. You give it to one user, and he/she gives it to someone else. 

Everyone is singing your praises.  Great Job.  I love this app.

Now 30 people have it.  Cracks are appearing, the app is not ready for the load.

People are starting to complain.

You were the hero, now you are constantly defending yourself for an unstable application.


What do you do?

Sell it

  • Get your users to quantify the value.
  • Get Feedback on new features and figure out the value.
  • Estimate the time.
  • Acknowledge the risk (It could 20% more because I don’t understand widgets)
  • Put together a presentation
  • Sell it to people who are willing to pay.
  • Execute.

Read More

Developer Resources for Windows 8 and Window Phone 8

Technorati Tags:

Starter Kits

1. Apimash

All kinds of examples of using web services.

Lots of nice UI design.

19 Windows 8 Starter Kits

1 Windows 8.1 Starter Kit

10 Windows Phone 8 Starter Kits

Lots of Sample

More Samples


Windows 8 Libraries that will help with MVVM

1. MvvmLight

2. Obelisk – Persistence Library – Tombstoning and Permanent Persistence

3. Okra App Framework – Navigation Framework


Windows Phone 8 Libraries that will help with MVVM

1. MvvmLight


XAML Spy   – Costs a few bucks, but it is definately worth it.

Just like snoop for wpf.  This works for Windows 8 and Windows Phone 8


The Design Resources for Windows 8 are amazing!!!

Read More

Choosing a Cross Platform Mobile Framework for Dummies


There are currently many choices for creating Mobile Applications.

1. You can just create a website.

2. You can create an app in each platform using Native Tools.

3. You can use a Cross-Platform Solution that can reduce the coding necessary to deploy to all platforms.


The ones that I know about are

If you like C# and want to use Visual Studio then Xamarin is the clear choice.

Read More

Software Development and The Cycle of Crap


You sit down in front of a legacy application. Maybe something that has a fair bit of complexity.

Do you allow yourself to enter the “This is crap let’s rewrite it” Cycle ?


The “This is crap let’s rewrite it” cycle is simple. The app is in shambles, there are business rules everywhere, you think you understand the app and all of the intricacies. You suggest that rewriting will be simple. It is not.  You run into things…

You now have this


You try to put the worms back in the can. But it involves hacking because you are up against a deadline.

You end up with something better, but will another developer enter into the same cycle?




Things that can save you and others

Humility – You don’t know everything and maybe some of the things are there for a reason

Architecture – Gives you clues on the big picture.

Historical Info – Someone on the project gives you insight on why things are the way they are.

Read More

Android Development – Architecture (MVVM or MVP)

Some Background

Most of my experience has been doing .net development. For the last few years I have done client development: WPF, Silverlight, and Windows Forms. Prior to that I was doing mostly development.

I recently started getting my feet wet in Android development and one of the first steps in any platform is to get comfortable with the parts of the applications and how they talk to each other.

Android is new to me so I will define some terms to start off with.


Activities: application component that provides a screen with which users can interact in order to do something.

Layout:  A layout is the architecture for the User Interface. Either defined by XML or in Code.

Intents: Messaging object that can be used inside of an application or from application to application.

The Xml layouts allows for methods to be implicitly wired to methods in the  Activity that loaded the resource.  Similar to Commanding in XAML.

How do you classify an Activity in terms of UI patterns?
Is it a Controller, View, Model, ViewModel, Presenter, does it fall outside of a UI Pattern, or is it a combination of many?

Is it a Controller?
In an MVC application you have a request come in and the controller handles the routing. It is pretty clear example of a controller.  It is entirely possible that you could write the app in a way that when an Activity is called it determines the view, but this seems like a stretch. I would see the relationship between an Activity and a View to be One-to-One.

In this case I would see the OS being the Controller.

So in my opinion it is not a Controller.

Is it a View?

Since you can create UI’s in an Activity using code, you could have a situation where it acts like a view, but you would lose the separation of concerns gained by using XML.


Is it a Presenter or a ViewModel?

If you consider the notification and management of the listadapter as Binding then it would be a ViewModel

ArrayAdapter<Customer> adapter = new ArrayAdapter<Customer>(this,android.R.layout.simple_list_item_1,customers);



If you don’t consider this a form of binding then it is probably more of a presenter.

In my opinion it feels more like MVVM but then my background in WPF could be skewing my opinion.


MVVM vs. MVP vs. MVC

Read More

F# Baby Steps – Fibonacci


F# is the kind of language that can really energize someone who is excited about just sitting down and powering out a solution.

Here is an example of how to sum some fibonacci numbers.


let rec fibonacci n = 

    if n < 2 then 1 

    else fibonacci (n-2) + fibonacci(n-1)


//create a recursive function called fibonacci

//that takes the parameter n 

// if n < 2 then 1  so this is the base case.

//else fib (n – 2) + fib (n – 1)


printfn "fibonacci = %A" (fibonacci 10)

Read More