Location Tracking/Processes/Observation Processing

From SOWNWiki
Jump to: navigation, search

This is the process gone through to upload observation data into the database. It reqires data in the format described in input. Its purpose is to provide a generic front end for various collectiopn progams like Kismet or a real time mobile device. Inputs are one-off observations. They are made by an interface at a particular location in time and space. Although a location is not neccecary as the system will attempt to work out the location of the observsation. This interface sees multiple networks and also provides information about those networks.


We are only interested in wireless networks at the moment

For Wireless Networks

One observation contains multiple interfaces which can contain multiple virtual access points. For most interfaces they will only broadcast one SSID but for devices like the SOWN(at)HOME nodes which broadcast more than one SSID with different security parameters on each virtual interface it is neccecary to have this separate table. An Observation need not have any Interfaces and an Interface need not have any AP Information.


  • Date - ISO 8601 Time Stamp that represents the UTC date/time the observation was made (YYYY-MM-DD hh:mm:ssZ)
  • Location - The location of the observation (lon,lat,alt)
  • Interface which made the observation - MAC address of the interface which made this observation


  • The interface observed - The MAC address of the interfae observed
  • Power of the recieved signel in dBm

Access Point information

XML field XML type Database field Database type Required? Description
ssid text ssid text yes The SSID the AP is broadcasting
hidden yes/no hidden_ssid bool no This is yes if the SSID of the AP is being broadcast or no if it is not
encryption none/WEP/WPA PSK/WPA Enterprise/802.1x encryption_id small_int no This is the type of encryption being used. It is sent as human readable text in the XML but stored as an integer in the database
public Not known/Free/Commercial/Private public small_int no This is the type of access provided by this access point. Not known indicates the observer does not know what type of AP this is. If this tag is not sent then it is assumed that the type is not known. Free access points are APs which provide continuous free access. They meybe run by a commercial company and require free registration. Commercial APs are APs for which money must be provided to access them. Private APs are APs which are not intended for public use - regardless of the ability of the public to access them.
dhcp yes/no dhcp bool no This indicates whether or not DHCP (v4) is available on this AP.
ip6 yes/no ip6 bool no This indicates whether or not IPv6 addresses are advertised by this node either through router advertisement or through DHCPv6
channel signed integer frequency integer no In XML this is the channel the AP is broadcasting on. In the database this is the frequency in kHz
frequency integer frequency integer no In XML this is the frequency in kHz. In the database this is also the frequency in kHz.
observation_interface integer no In XML this is determined by the interface this AP is nested under. In the database this is a lionk to the id of the interface broadcasting this AP.
username text username varchar(30) no 802.1x password
passphrase text passphrase text no The WPA/WEP passphrase or the 802.1x password
certificate text certificate text no The certificate issued by this AP to identify itself
technology text technology integer no The type of technology used as the carrier eg 802.11g

Note: The frequency of the AP need only be specified as either a channel or frequency. If both are specified the frequency is assumed to be the correct one.

Inputing the data into the database

The data from an observation is fed into the observation database with the following XML file:

<?xml version="1.0" encoding="UTF-8"?>
 <sown_location_system_observation xmlns="http://www.sown.org.uk/schemas/sls/0.2">
Fatal error: The format of the coordinate could not be determined. Parsing failed.


Environment Section

This section is introduced in 0.2 so that the location database can take the information from the SOWN wireless survey. The environment variable looks as follows:


Humans is the % of people in the area. For example a full room is 100. An empty street is 0.

Description is a description of the area

AP Section

The section contained within the


tags is used to provide information about the individual SSIDs (Virtual Access Points) being broadcast by the interface. Most access points will only broadcast one SSID. Several modern access points have the ability to broadcast more than one SSID with the same physical interface. This is used by SOWN[at]home nodes to provide both web form authenticated access and 802.1x protected access. It is also commonly used by companies to provide guest access alongside corporate access over their wireless infrastructure. The XML tags used in this section are specified in the section Access_Point_information. The XML is shown below:



The input data is taken and the following steps are performed:

  1. Does the interface making the observation already exist?
      1. Yes: Continue
      2. No: Add the interface and continue
  2. Loop through all the observed interfaces performing the following on each:
    1. Is the Interface already in the system?
      1. Yes: Continue
      2. No: Add the interface and continue
    2. Add the observation details to the database

PHP Files


Once an XML file which conforms to the 0.1 specification for data transfer in the location system has been generated either by an application or from processing a third party observation file it can be fed in the xml_observation_upload.php page via a HTTP post. This page will take the XML input and enter it into the database.


This file takes input in the form of two files from a kismet observation. These files are Kismet-date.xml and the Kismet-date.gps file. The first of the files gives details about all the MAC addresses observed. The second file is the GPS trace which provides a record of the areas covered the MAC addresses seen there. The purpose of this page is take the two files and produce a 0.1 specification XML file which can be fed into xml_observation_upload.php


Currently this page takes PlaceLab data and enters it into the database. It needs to be modified to convert the placelab data into standard XML for input into the database. It is doubtful whether this will be of any benifit so it is not scheduled to done any time soon.


This page takes text files which are derived from WiGLE. This was used to initialy provide data with which to produce maps. Unfortunatly the data from WiGEL consists of innacurate estimations of AP locations and is licenced to restrictivly to be of any use. Once the system goes into production with our own observed data we will delete all the WiGLE data and this page will no longer be of any use.


This page is very much work in progress. It is hoped that work on this may be incorperated into a third year project.

This probably should not be on this page as it is related to processing the observations to get a location for the access points. The page recieves an input from the database server/location processing dispatch server which contains details about the observations of a particular access point. The page then processes the data and returns what it believes the location of the access point is. The idea behind this system is that it allows distributed processing of the observation data and also meens that a third party developer need not know about the database that is used to store the data. The actual data sent to and recieved from this page has yet to be finalised. Also the page itself does not actualy work as documented on the page.

Scratchpad PHP files


This was a test to see if a text representation of a Genom object could be transmitted and then decoded. It can!