Creating a Segregated Searchable/Editable Map in ArcGISOnline

I would like to create a map that stores core company location data, that anyone in the company can search and a select few can add data points

ArcGISOnline has many features to jump start your project.

The Architecture/Plan

SolutionArchitecture

Basic Parts

These are my interpretations of the pieces*

  • Feature Layer
    • A Feature Layer is the hosted data.  It is where you would upload your shapefile/geodatabase.
    • It is the data store.  A Web Map is made up of one to many Feature Layers.
  • Web Map
    • It the most basic viewer of the Feature Data.  It is where you add your features and define how your points show up.
  • Web Mapping Application
    • There are two ways to create
    • Web App Builder for ArcGIS
      • Allows you add various widges
    •  Templates
      • A template is a pre-built website that has a purpose built function.  ESRI has put all of their templates on GitHub so you can download and customize them in order to meet your mapping needs.
      • Examples
        • Basic Viewer – Presents a map in a general purpose app with a collection of essential tools including edit and print.  Download
        • Compare Analysis – Compares geographic phenomena at one or more locations with multiple side-by-side maps.  Download
        • Crowdsource Manager – Provides the ability to review crowd sourced information and update attributes such as status, assignment, etc. Download

Domain

Lets Get Started


 

Step One

Create a Layer that will store the data.

Since I live in Minnesota I created a Shapefile with the Coordinate System

Projected Coordinate System:
Name: NAD_1983_UTM_Zone_15N
Geographic Coordinate System:
Name: GCS_North_American_1983
Here is the Data I am working with

ShapefileData

I created Attributes for

ProjName, CreateDate, Company, ProjType

So we have some good things to search for once the data is uploaded.

 

The First Step is to zip up my shape file data.

It is all of the data that begins with your shape file name.

UploadShapefile

In My Content go to Add Item / Select My Computer and choose the zip file you just created.

Now we have a Feature Layer that we can use as a foundation for our web maps and templates.

You can go into the Feature Layer click on Edit and Check the box for “Enable editing and allow editors to: Add, update, and delete features”

Note: You can re-upload an updated shape file by clicking on overwrite and picking a file.  This is also a nice way to manage updates if you don’t need a way for novices to edit the data.


 

Step Two

In the Content Area go to Create

Now create a new Map.  — This will create a new Web Map

This will bring into Edit Mode.  Go to the top of the now editable web map and add the layer we just created in the previous step.

MonumentWebMap

Save it


 

Creating the Editor

I initially tried the template Basic Editor.  I found this template to be pretty useful in terms of editor.

The nice part is you can add points and edit the data.

The Search is not very good, when you enable it.  If you search for a feature like Company Name, it will go to the first point it finds, and you cannot go to the next result.

In contrast the Find/Edit/Filter Template works great for filtering, but does not allow you to add points.

Trade-offs:(

Due to the trade-offs, and some of the weird complexity I decided to use the Web App Builder for ArcGIS.  This allowed me to customize the filters and Edit Widget. I am sure the downside to the Web App Builder is the code is not available so I can’t create a brand new variant.

I have no desire to do that, so this should be good.

The Filter Setup is a breeze, and once it is done it looks great.  You can even customize how the results look.  A drawback on some of the templates.

 

Filter Setup

ResultsFilter

Once we are finished you have a nice filter interface with a marginally slick editor interface(who are we kidding, editing a map nicely in an web app is a pretty hard task)

WebAppBuilder_EditFilter


 

Creating the Searchable Viewer

I had tried using some of the Filter Templates

At the end of the day I found the Web App Builder worked great.

I just created a Web App Builder and did not include the edit widget.

In this case I could just use sharing to manage who had permissions to what functions.

 

At this point I am done.

I created the feature layer, Web Map, and two web apps that should expose segrated edit/filter capabilities to your different user types.

 

Data Analysis with Python – The Tools and the Data

Welcome to the Inaugural Post for my Learning Python Series

The Post is the start of a series of walk throughs from Start to Finish of my journey into Data Analysis and Data Science with Python

 

The Tools and Loading the Data

What are the tools to download in order to get started building in Python

How do I load the data and construct the domain

How do I do some basic analysis on the data to get a feel for the relationships.

The Next Part

The next post will utilize Panda to perform quicker more structure data analysis.

 

 

The Tools

I downloaded and installed the Anaconda distribution along with the Visual Studio Python Tools

The Anaconda distribution is a python distribution that contains many many scientific libraries.

The Data

Fortunately there is a massive amount of data that you can have fun and experiment with.

The UCI Machine Learning Repository has a huge amount of data.

Examples

  • Mice Protein Expression
    • Expression levels of 77 proteins measured in the cerebral cortex of 8 classes of control and Down syndrome mice exposed to context fear conditioning, a task used to assess associative learning.
    • Mice Protein
  • Car Evaluation Data Set
    • Derived from simple hierarchical decision model, this database may be useful for testing constructive induction and structure discovery methods.
  • Adult Data Set
    • Predict whether income exceeds $50K/yr based on census data. Also known as “Census Income” dataset.

I am going to use the Adult Data Set in my Examples.

 

Loading the Data and Graphing the Data

Grab the Data from

http://archive.ics.uci.edu/ml/datasets/Adult

import csv

with open('C:\adult.test','r') as f:
    for line in f: 
        reader = csv.reader(f)
        for row in reader: 
            age = row[0]
            workclass = row[1]
            fnlweight = row[2]
            education = row[3]
            educationnum = row[4]
            maritalstatus = row[5]
            occupation = row[6]

            print workclass

This will print out the workclass column in the data.

Resulting in an output like

 State-gov
 Federal-gov
 Private
 Private
 Private
 Local-gov
 Private
 Local-gov

In order to start to looking into the data we can use someone of Python’s built in magic to bucket the data and create some histograms.

Histograms will tell us how the data is distributed and start to give us clues about the shape of the data.

In order to create a histogram we will use the collections library to count the data

 

def create_histogram(labels, values, bucket_size, title):
    plt.bar(labels, values)
    plt.title(title)
    plt.show()
    

agelist = list()

with open ('C:\Users\Jon\Documents\adult.test','r') as f:
	for line in f:
		reader = csv.reader(f)
		for row in reader:
			try:
				age = row[0]
				agelist.append(age)

			except IndexError:
				print("something")

agelistfloat = [float(x) for x in agelist]
agedist = Counter(agelist)

labels, values = zip(*agedist.items())

valueslistfloat = [float(x) for x in values]
labelsliststring = [float(x) for x in labels]

create_histogram(labelsliststring,valueslistfloat,5,"Age Distribution Simple")

Once I get the file

I take my agelist and run it through Counter.

Counter allows for rapid tallying of data.  It returns a defaultdict object that list each age and how many occurences there were.

We then unzip the list to labels and values.  * reverses the zip operation.

We then call the pyplot.bar(labels, values) to show the graph.

Simple Histogram

We can see that our data is a right-skewed distribution.  When you look at the data you can see it is evenly distributed over the income generating population.  At around 18 it starts and starts to gradually tail off at the peak of around 40.

 

First Refactoring – Making the code a bit more compact and readable

I mapped each row to a named tuple in order to iterate through the data a bit more intuitively

First I created a dictionary of the columns in the data.

    economic_columns = ['age', 'workclass', 'fnlwght', 'education', 'educationNum', 'maritalStatus', 'occupation', 'relationship', 'race', 'sex', 'capitalGain', 'capitalLoss', 'hoursPerWeek','nativeCountry', 'income']

    EconRecord = collections.namedtuple('econ',economic_columns)

 

I then created a object that would represent the named tuple.

    rowlist = list()

    for econ in map(EconRecord._make, csv.reader(open('C:\Users\Jon\Documents\adult.txt', "r"))):
        rowlist.append(econ)

I used the EconRecord and the map function to apply EconRecord._make to every record in the collection.  Creating a new econ record for each row in the file.

The result is being able is being to aggregate the items in a bit more cleanly with more concise readable code.

    agelistint = [int(x.age) for x in rowlist]
    agedist = Counter(agelistint)
    labels, values = zip(*agedist.items())

The Next Series we will start to scatter plot and look for relationships in the data using Panda.

 

References 

Python Lists

The Book – Data Science from Scratch

Beautiful Plots With Pandas and Matplotlib

Collections 

CSV  – Python

Python Structure