Creating a Segregated Searchable/Editable Map in ArcGISOnline

I would like to create a map that stores core company location data, that anyone in the company can search and a select few can add data points

ArcGISOnline has many features to jump start your project.

The Architecture/Plan

SolutionArchitecture

Basic Parts

These are my interpretations of the pieces*

  • Feature Layer
    • A Feature Layer is the hosted data.  It is where you would upload your shapefile/geodatabase.
    • It is the data store.  A Web Map is made up of one to many Feature Layers.
  • Web Map
    • It the most basic viewer of the Feature Data.  It is where you add your features and define how your points show up.
  • Web Mapping Application
    • There are two ways to create
    • Web App Builder for ArcGIS
      • Allows you add various widges
    •  Templates
      • A template is a pre-built website that has a purpose built function.  ESRI has put all of their templates on GitHub so you can download and customize them in order to meet your mapping needs.
      • Examples
        • Basic Viewer – Presents a map in a general purpose app with a collection of essential tools including edit and print.  Download
        • Compare Analysis – Compares geographic phenomena at one or more locations with multiple side-by-side maps.  Download
        • Crowdsource Manager – Provides the ability to review crowd sourced information and update attributes such as status, assignment, etc. Download

Domain

Lets Get Started


 

Step One

Create a Layer that will store the data.

Since I live in Minnesota I created a Shapefile with the Coordinate System

Projected Coordinate System:
Name: NAD_1983_UTM_Zone_15N
Geographic Coordinate System:
Name: GCS_North_American_1983
Here is the Data I am working with

ShapefileData

I created Attributes for

ProjName, CreateDate, Company, ProjType

So we have some good things to search for once the data is uploaded.

 

The First Step is to zip up my shape file data.

It is all of the data that begins with your shape file name.

UploadShapefile

In My Content go to Add Item / Select My Computer and choose the zip file you just created.

Now we have a Feature Layer that we can use as a foundation for our web maps and templates.

You can go into the Feature Layer click on Edit and Check the box for “Enable editing and allow editors to: Add, update, and delete features”

Note: You can re-upload an updated shape file by clicking on overwrite and picking a file.  This is also a nice way to manage updates if you don’t need a way for novices to edit the data.


 

Step Two

In the Content Area go to Create

Now create a new Map.  — This will create a new Web Map

This will bring into Edit Mode.  Go to the top of the now editable web map and add the layer we just created in the previous step.

MonumentWebMap

Save it


 

Creating the Editor

I initially tried the template Basic Editor.  I found this template to be pretty useful in terms of editor.

The nice part is you can add points and edit the data.

The Search is not very good, when you enable it.  If you search for a feature like Company Name, it will go to the first point it finds, and you cannot go to the next result.

In contrast the Find/Edit/Filter Template works great for filtering, but does not allow you to add points.

Trade-offs:(

Due to the trade-offs, and some of the weird complexity I decided to use the Web App Builder for ArcGIS.  This allowed me to customize the filters and Edit Widget. I am sure the downside to the Web App Builder is the code is not available so I can’t create a brand new variant.

I have no desire to do that, so this should be good.

The Filter Setup is a breeze, and once it is done it looks great.  You can even customize how the results look.  A drawback on some of the templates.

 

Filter Setup

ResultsFilter

Once we are finished you have a nice filter interface with a marginally slick editor interface(who are we kidding, editing a map nicely in an web app is a pretty hard task)

WebAppBuilder_EditFilter


 

Creating the Searchable Viewer

I had tried using some of the Filter Templates

At the end of the day I found the Web App Builder worked great.

I just created a Web App Builder and did not include the edit widget.

In this case I could just use sharing to manage who had permissions to what functions.

 

At this point I am done.

I created the feature layer, Web Map, and two web apps that should expose segrated edit/filter capabilities to your different user types.

 

New Router – Asus RT-N16 and Tomato Awesomeness

Jeff Atwood has an excellent article recommending this router.

It can be found here.

In the article is a guide to flashing the firmware with tomato.

http://www.shadowandy.net/2012/03/asus-rt-n66u-tomatousb-firmware-flashing-guide.htm

The one caveat that I would add is.

You have to reset the NVRAM via key presses which means

    1. Power off the RT-N66U
    2. Press and hold down the WPS button
    3. While holding the WPS button, plug in the power cable to turn RT-N66Uon
    4. Keep holding the WPS button for 30 seconds before releasing
      The router should reboot
    5. Congratulations. The NVRAM has been cleared.

I was not able to reach the web address so I was not able to perform the operation via the web admin, so the key press option was my only option.

 

In performing this process. I found the instructions and distribution of Easy Tomato to be fantastic.

http://www.easytomato.org/installing-easytomato-on-your-router

Easy Tomato is by far the best router firmware I have ever used. 

 

It makes the things that should be easy, easy and allows the advanced settings to be found easily.

 

A couple of examples

You can block any and all adult sites with a single checkbox.(bam)

You can define groups and drag the devices on your network into groups.  Yeah I said drag(holy web 2.0 interface batman)

Each group can then have access rules defined for it.

 

I highly recommend Easy Tomato and the Asus RT-N16 router.

Letter to Rookie Developers

Dear Rookie Developers

First off let me start by saying I am still a Rookie in a lot of ways. I still make plenty of stupid mistakes and I still forget to submit my hours on time:)

I have been on a few development projects and have learned a lot about how to develop software. Admittedly I still am not sure on what is the best way to develop software, but I have some advice on the pitfalls and things that I think were bright spots on my rise to programming mediocrity.

Get help early and often.

Don’t be embarrased to ask a senior developer how he/she would implement a solution. I do it all of the time. None of us really know the best solution, we just know the best solution in relation to the solutions that we have seen.  Push for code reviews they are an invaluable tool into the mind of another developer, try and study the code before hand so you have some context on the problem at hand.

Read code

Scott Hanselman talks about it all of the time.

Code is interesting, how else are you going to get the best practices that are going to keep you on the good side of your developer comrades.

Be proud of your work, but don’t be afraid to throw it in garbage.

It is great to be proud, but we need a healthy mix of pride and realism.  A lot of times I will write a method and just delete it and rewrite it. It you are doing TDD then the second time your code will probably be a lot cleaner and more efficient.

Read Code Complete

There is a lot of great advice in Code Complete that every developer should be familiar with.

Start out thinking about testing

When you start coding, think about how easy it will be test this method. The more you do that the easier the jump TDD will become.

Learn new languages

As computer scientists we are abstract thinkers. That is one of the many reasons that math is taught to us.  It gears us up for the abstractions that are on the horizon. By learning new programming languages we learn on how to approach a problem on a whole new level. It can also help us identify improvements that can be made in our process. Just look at what has happened in the Java world, and the .net world. Seems like we have a constant trickle of Java tools being ported over to the .net world.

These are just a couple of things you can do to improve your craft. But keep in mind development is like a trade. When you first start out we are all just apprentices, we learn from others on the best ways to approch problems, and code. We perfect our craft and hopefully one day we will be masters at our trade.

Database Edition now available from msdn

Just noticed that Database Edition is available to download with my msdn subscription.

Before it was about $3000 to buy, I wonder why there are just giving it away now?

I supect it has to do with the fact that currect dbas are heavily entrenched in their tools and practices, and don’t want to change.  By giving it away for free, they can get the developers on board and get them to show the dbas how nice the tool works.

Conveying Complexity with Simple examples or analogies

Google just released there new browser Google Chrome

It is beatifully simple piece of coding, in which each tab is broken into its own process.

Each process can then spawn child process which correspond to various plugins, such as shockwave flash, etc.

This is a great way to isolate your processes from other errors, essentially it is a mini-os wrapped in a browser.

This can be a difficult concept to wrap your head around and the cartoon that was produced is an excellent way to understand the nuts and bolts.

Some cartoons that convey difficult concepts.

Google Chrome

Why we need architects

What is perpendicular storage

Design Patterns

SOA is something that is often defined using analogies such as Legos,  Transportation, or the post office.

It pays to think of things at their simplest level, it will really be helpful for you when attempting to convince others or your brilliance:)

Getting started with the Google App Engine.

The tutorial goes over some basic concepts surrounding Google’s App Engine Framework, demonstrates using the Google App Engine to store data, and using Django templates to create a GeoRss feed that is consumed by Google maps.

Setup your environment

I chose eclipse as my ide.
The nice thing about eclipse is if you add the lib directories of whatever you are using (including the Google App Engine) you will get some intellisense.

Download the necessary components.
Google app engine
Eclipse
Installing Pydev
The documentation helped, but the link was bad. I used http://pydev.sf.net/updates/.

The guts of the python file


import cgi
import os
import wsgiref.handlers

from google.appengine.api import users
from google.appengine.ext import webapp
from google.appengine.ext import db
from google.appengine.ext.webapp import template

_DEBUG=True

class Business(db.Model):
    ......

def main():
    application = webapp.WSGIApplication([
            ('/',MainPage),
            ('/createbusiness.do',BusinessSignup),
            ('/georssfeed.xml',GeoRssFeed)
            ],debug=_DEBUG)
    wsgiref.handlers.CGIHandler().run(application)

if __name__ == "__main__":
    main()

The main method is where we map our urls to the classes we have defined within the python file.
Each class that handles requests should have a get or a post method.
When a get or a post occurs it will be routed automagically to the appropriate method.

Creating the table

 
class Business(db.Model):
    name = db.StringProperty()
    description = db.StringProperty(multiline=True)
    url = db.URLProperty()
    location = db.StringProperty()
    latitude = db.StringProperty()
    longitude = db.StringProperty()
    address = db.StringProperty()
    created = db.DateTimeProperty(auto_now_add=True)

Description can contain line breaks so we specify multiline=True
Created is of type DateTime and has the property auto_now_add set to true
created is set to the current time the first time the model instance is stored in the datastore, unless the property has already been assigned a value.

There is also an auto_now property that can be used to set the current time each time the record is created or updated. Useful for modified dates.


Handling the request

In one of googles examples(Task List) they used a base class for the request.
Here is my modified version.

class BaseRequestHandler(webapp.RequestHandler):
    """Supplies a common template generation function"""
    def generate(self,template_name,template_values={}):
        values = {
                  'request': self.request,
                  'debug': self.request.get('deb'),
                  'application_name': 'Local Business Directory'
                  }
        
        values.update(template_values)
        directory = os.path.dirname(__file__)
        path = os.path.join(directory,os.path.join('templates',template_name))
        self.response.out.write(template.render(path,values,debug=_DEBUG))

This does a few nice things.

        values = {
                  'request': self.request,
                  'debug': self.request.get('deb'),
                  'application_name': 'Local Business Directory'
                  }
        values.update(template_values)

This sets up an array of base values that will be passed into the template.
In other methods that use base request, we will add other objects to this array. So our html templates can process data.
The last line values.update is where the two arrays gets merged.

        path = os.path.join(directory,os.path.join('templates',template_name))

In this application I created a templates folder to separate the html from the code. This line just adds the template_name to the /templates path.

    self.response.out.write(template.render(path,values,debug=_DEBUG))

And finally
Write the request out.

Using the BaseRequestHandler

class MainPage(BaseRequestHandler):
    def get(self):
        
        #Get all of the businesses
        businesses = Business.all().order('-created')
        
        self.generate('index.html', {
                                     'businesses': businesses
                                     })   

Here is a simple example of querying all of the businesses ordered by created date.
We then call the generate method on the BaseRequestHandler, passing in our additional objects, along with the template name.

Using Templates
The Google App Engine uses the Django templating engine. W00t

The for loop

{% for athlete in athlete_list %}
    
  • {{ athlete.name }}
  • {% endfor %}

    The if statement(there are several varieties)

    {% if athlete_list %}
        Number of athletes: {{ athlete_list|length }}
    {% else %}
        No athletes.
    {% endif %}
    
    {% ifequal user.id comment.user_id %}
        ...
    {% endifequal %}
    

    In the spirit of python, there are a lot of functions that Django gives you.
    Examples:
    timesince: Formats a date as the time since that date (e.g., “4 days, 6 hours”).
    phone2numeric: Converts a phone number (possibly containing letters) to its numerical equivalent. For example, ‘800-COLLECT’ will be converted to ‘800-2655328’.

    More Information on Django templates

    In order to display a list of businesses I am just using a simple for loop and creating a row each time.

    {% for business in businesses %} <td class="main"
    {{ business.name }}
    {% endfor %}
    Business Name Address(Address, City, State) Description Url Location Latitude Longitude
    {{ business.address }} {{ business.description }} {{ business.url }} {{ business.location }} {{ business.latitude }} {{ business.longitude }}

    Entering Data

    Two pieces of code were necessary for this
    Plumbing in the python file

    class BusinessSignup(webapp.RequestHandler):    
        def post(self):
            business = Business()
            
            business.name = self.request.get("txtBusinessName")
            business.address = self.request.get('txtAddress')
            business.description = self.request.get('txtDescription')
            business.url = self.request.get('txtUrl')
            business.location = self.request.get('txtLocation')
            business.latitude = self.request.get('txtLatitude')
            business.longitude = self.request.get('txtLongitude')
            
            business.put()
            self.redirect('/')
    

    This just grabs from the data from the request and sets each property on our business object.
    Then calls put.
    put is an instance method that saves the data to the database.
    delete, to_xml, is_saved, are a couple of other useful instance methods.

        
    

    Tells the form to post to the specified address.

    Bringing it all together

    application: yourapplication
    version: 1
    runtime: python
    api_version: 1
    
    handlers:
    - url: /static
      static_dir: static
    
    - url: /.*
      script: localbusinesslocator.py
    

    The app.yaml is where your external url mapping occurs.
    If you wanted to use several python files, this is where that would happen.
    More Info can be found here

    Testing the application
    usr/local/google_appengine/dev_appserver.py /sourcedirectory/

    Hopefully this fills in some gaps left by Googles tutorial.

    The next installment of the series will go over displaying the data in the GeoRss format and displaying it on google maps.

    Modeling using Enterprise Architect

    Recently I have been introduced to a new modeling tool called Enterprise Architect.
    It is truly a great product for a great price.
    It has full support for UML 2.1.

    They have a robust and easy solution for distributed teams that have a need to access design artifacts.

    Artifact Store

    Local File or Shared Network Store
    As with any modeling software you can store a file containing your models.

    DBMS
    You can easily configure SQL server to be your backend store, which allows a multi-user access to your modeling projects.

    Source Control
    Either you can use a DBMS in conjuction with a Version Control Product (Shared Model)
    or you
    use a local file based solution with a Version Control.
    The major source control providers are supported out of the box, and you can add support for work item tracking in tfs with an optional plugin.

    The price is very reasonable at $250 for an enterprise seat.
    Compared with $5000 for Rational Software Architect, which is comparable to Power Designer and ErWin.

    If you want to do some serious modeling without having to sacrifice your yearly budget. I would suggest getting a few copies of enterprise architect for your team.

    Recovering data from a computer that won’t boot and has a bad dvd rom

    If the computer will not boot and the dvd drive is bad. You will have to find an alternate way to boot the computer.

    I used a thumbdrive since the bios supported it.

    First get the thumbdrive bootable.

    In order to get access to the windows drives you should do a
    fdisk -l to see the physical devices.

    You will have to mount whichever drive you need to get the data from.
    mkdir /mnt/windows1
    mount /dev/hda1 /mnt/windows1 -t ntfs -r

    Once you have the drive you need you can get your backup device setup.

    I used a usb hard drive(beware some only turn on when active –MyBook)

    Now you can do a copy
    cp -r /mnt/windows1 /mnt/usbharddrive

    Voila you have your valuable data again.