Difference between revisions of "Change: Weather Data"

From HiveTool
Jump to: navigation, search
(Created page with "Currently, in version 0.1-0.2, the program on the hive computer reads the local weather from weather underground in xml, parses out the variables that are logged locally and a...")
 
 
(3 intermediate revisions by the same user not shown)
Line 2: Line 2:
  
 
Now that a central database is used and the hive computers are often behind firewalls and run more like data collectors, this is not efficient.  Instead of the hive computer downloading the weather data and uploading it to the database, it make more sense for the data base to access the weather data.
 
Now that a central database is used and the hive computers are often behind firewalls and run more like data collectors, this is not efficient.  Instead of the hive computer downloading the weather data and uploading it to the database, it make more sense for the data base to access the weather data.
 +
 +
 +
The current upload:
 +
 +
 +
<nowiki>
 +
<hive_data>
 +
        <hive_observation>
 +
                <hive_id>XP002</hive_id>
 +
                <hive_observation_time>2015-11-22 02:55:01</hive_observation_time>
 +
                <hive_weight_lbs>37.68</hive_weight_lbs>
 +
                <hive_temp_c>15.90</hive_temp_c>
 +
                <hive_relative_humidity>55.11</hive_relative_humidity>
 +
                <hive_ambient_temp_c>8.45</hive_ambient_temp_c>
 +
                <hive_ambient_relative_humidity>70.23</hive_ambient_relative_humidity>
 +
                <hive_ambient_luminance>5.00</hive_ambient_luminance>
 +
                <hive_ambient_precip_in>0.00</hive_ambient_precip_in>
 +
        </hive_observation>
 +
<current_observation>
 +
<credit>Weather Underground Personal Weather Station</credit>
 +
<credit_URL>http://wunderground.com/weatherstation/</credit_URL>
 +
<image>
 +
<url>http://icons.wunderground.com/graphics/bh-wui_logo.gif</url>
 +
<title>Weather Underground</title>
 +
<link>http://wunderground.com/weatherstation/</link>
 +
</image>
 +
<location>
 +
<full>Hawk Ridge, Clayton, GA</full>
 +
<neighborhood>Hawk Ridge</neighborhood>
 +
<city>Clayton</city>
 +
<state>GA</state>
 +
<zip/>
 +
<latitude>34.889847</latitude>
 +
<longitude>-83.403343</longitude>
 +
<elevation>2146 ft</elevation>
 +
</location>
 +
<station_id>KGACLAYT6</station_id>
 +
<station_type>RainWise MK-III</station_type>
 +
<observation_time>Last Updated on November 22, 2:55 AM EST</observation_time>
 +
<observation_time_rfc822>Sun, 22 Nov 2015 07:55:02 GMT</observation_time_rfc822>
 +
<weather/>
 +
<temperature_string>44.1 F (6.7 C)</temperature_string>
 +
<temp_f>44.1</temp_f>
 +
<temp_c>6.7</temp_c>
 +
<relative_humidity>89</relative_humidity>
 +
<wind_string>Calm</wind_string>
 +
<wind_dir>ENE</wind_dir>
 +
<wind_degrees>67</wind_degrees>
 +
<wind_mph>0.0</wind_mph>
 +
<wind_gust_mph>9.0</wind_gust_mph>
 +
<pressure_string>27.80" (941.3 mb)</pressure_string>
 +
<pressure_mb>941.3</pressure_mb>
 +
<pressure_in>27.80</pressure_in>
 +
<dewpoint_string>41.0 F (5.0 C)</dewpoint_string>
 +
<dewpoint_f>41.0</dewpoint_f>
 +
<dewpoint_c>5.0</dewpoint_c>
 +
 +
<heat_index_string/>
 +
<heat_index_f/>
 +
<heat_index_c/>
 +
 +
 +
<windchill_string/>
 +
<windchill_f/>
 +
<windchill_c/>
 +
 +
<solar_radiation/>
 +
<UV/>
 +
<precip_1hr_string>0.00 in (0.0 mm)</precip_1hr_string>
 +
<precip_1hr_in>0.00</precip_1hr_in>
 +
<precip_1hr_metric>0.0</precip_1hr_metric>
 +
<precip_today_string>0.00 in (0.0 cm)</precip_today_string>
 +
<precip_today_in>0.00</precip_today_in>
 +
<precip_today_metric>0.0 cm</precip_today_metric>
 +
<history_url>http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KGACLAYT6</history_url>
 +
<ob_url>http://www.wunderground.com/cgi-bin/findweather/getForecast?query=34.889847,-83.403343</ob_url>
 +
</current_observation>
 +
 +
</hive_data>
 +
</nowiki>
 +
 +
Could be reduced to:
 +
 +
 +
<nowiki>
 +
<h>
 +
<hi>XP002</hi>
 +
<ht>2015-11-22 02:55:01</ht>
 +
<hl>37.68</hl>
 +
<hc>15.90</hc>
 +
<hh>55.11</hh>
 +
<ac>8.45</ac>
 +
<ah>70.23</ah>
 +
<al>5.00</al>
 +
<ap>0.00</ap></h>
 +
</nowiki>
 +
 +
 +
or
 +
 +
<nowiki>
 +
<h><hi>XP002</hi><ht>2015-11-22 02:55:01</ht><hl>37.68</hl><hc>15.90</hc><hh>55.11</hh><ac>8.45</ac><ah>70.23</ah>
 +
<al>5.00</al><ap>0.00</ap></h>
 +
</nowiki>
 +
 +
 +
----
 +
  
 
A problem was observed with the script that reads the xml file posted by the hive.  It was failing with this error
 
A problem was observed with the script that reads the xml file posted by the hive.  It was failing with this error

Latest revision as of 00:30, 22 November 2015

Currently, in version 0.1-0.2, the program on the hive computer reads the local weather from weather underground in xml, parses out the variables that are logged locally and appends the weather xml file to the hive data xml file.

Now that a central database is used and the hive computers are often behind firewalls and run more like data collectors, this is not efficient. Instead of the hive computer downloading the weather data and uploading it to the database, it make more sense for the data base to access the weather data.


The current upload: 


<hive_data>
        <hive_observation>
                <hive_id>XP002</hive_id>
                <hive_observation_time>2015-11-22 02:55:01</hive_observation_time>
                <hive_weight_lbs>37.68</hive_weight_lbs>
                <hive_temp_c>15.90</hive_temp_c>
                <hive_relative_humidity>55.11</hive_relative_humidity>
                <hive_ambient_temp_c>8.45</hive_ambient_temp_c>
                <hive_ambient_relative_humidity>70.23</hive_ambient_relative_humidity>
                <hive_ambient_luminance>5.00</hive_ambient_luminance>
                <hive_ambient_precip_in>0.00</hive_ambient_precip_in>
        </hive_observation>
	<current_observation>
		<credit>Weather Underground Personal Weather Station</credit>
		<credit_URL>http://wunderground.com/weatherstation/</credit_URL>
		<image>
		<url>http://icons.wunderground.com/graphics/bh-wui_logo.gif</url>
		<title>Weather Underground</title>
		<link>http://wunderground.com/weatherstation/</link>
		</image>
		<location>
		<full>Hawk Ridge, Clayton, GA</full>
		<neighborhood>Hawk Ridge</neighborhood>
		<city>Clayton</city>
		<state>GA</state>
		<zip/>
		<latitude>34.889847</latitude>
		<longitude>-83.403343</longitude>
		<elevation>2146 ft</elevation>
		</location>
		<station_id>KGACLAYT6</station_id>
		<station_type>RainWise MK-III</station_type>
		<observation_time>Last Updated on November 22, 2:55 AM EST</observation_time>
		<observation_time_rfc822>Sun, 22 Nov 2015 07:55:02 GMT</observation_time_rfc822>
		<weather/>
		<temperature_string>44.1 F (6.7 C)</temperature_string>
		<temp_f>44.1</temp_f>
		<temp_c>6.7</temp_c>
		<relative_humidity>89</relative_humidity>
		<wind_string>Calm</wind_string>
		<wind_dir>ENE</wind_dir>
		<wind_degrees>67</wind_degrees>
		<wind_mph>0.0</wind_mph>
		<wind_gust_mph>9.0</wind_gust_mph>
		<pressure_string>27.80" (941.3 mb)</pressure_string>
		<pressure_mb>941.3</pressure_mb>
		<pressure_in>27.80</pressure_in>
		<dewpoint_string>41.0 F (5.0 C)</dewpoint_string>
		<dewpoint_f>41.0</dewpoint_f>
		<dewpoint_c>5.0</dewpoint_c>
		
		<heat_index_string/>
		<heat_index_f/>
		<heat_index_c/>
		
		
		<windchill_string/>
		<windchill_f/>
		<windchill_c/>
		
		<solar_radiation/>
		<UV/>
		<precip_1hr_string>0.00 in (0.0 mm)</precip_1hr_string>
		<precip_1hr_in>0.00</precip_1hr_in>
		<precip_1hr_metric>0.0</precip_1hr_metric>
		<precip_today_string>0.00 in (0.0 cm)</precip_today_string>
		<precip_today_in>0.00</precip_today_in>
		<precip_today_metric>0.0 cm</precip_today_metric>
		<history_url>http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KGACLAYT6</history_url>
		<ob_url>http://www.wunderground.com/cgi-bin/findweather/getForecast?query=34.889847,-83.403343</ob_url>
	</current_observation>

</hive_data>

Could be reduced to: 


<h>
<hi>XP002</hi>
<ht>2015-11-22 02:55:01</ht>
<hl>37.68</hl>
<hc>15.90</hc>
<hh>55.11</hh>
<ac>8.45</ac>
<ah>70.23</ah>
<al>5.00</al>
<ap>0.00</ap></h>


or 

<h><hi>XP002</hi><ht>2015-11-22 02:55:01</ht><hl>37.68</hl><hc>15.90</hc><hh>55.11</hh><ac>8.45</ac><ah>70.23</ah>
<al>5.00</al><ap>0.00</ap></h>



A problem was observed with the script that reads the xml file posted by the hive. It was failing with this error 

mismatched tag at line 14, column 2, byte 719 at /usr/local/lib/perl5/site_perl/5.8.8/i686-linux/XML/Parser.pm line 187

Weather Underground was down. cURL was returning an error message instead of the xml page. The xml file sent to the database was corrupted: 

        <hive_observation>
                <hive_id>XP001</hive_id>
                <hive_observation_time>2014/01/09 19:10:01</hive_observation_time>
                <hive_observation_time_rfc822>Mon, 08 Jul 2013 11:20:03 GMT</hive_observation_time_rfc822>
                <hive_weight_lbs>+ 66.5 lb</hive_weight_lbs>
                <hive_temp_c>22.3</hive_temp_c>
                <hive_ambient_temp_c>5.1</hive_ambient_temp_c>
                <hive_ambient_relative_humidity>62.2</hive_ambient_relative_humidity>
        </hive_observation>
 <HTML><HEAD><TITLE>Error</TITLE></HEAD><BODY> 
 An error occurred while processing your request.<p>
 Reference #97.45ff4f17.1389312638.28562732
 </BODY></HTML>
 [repeated 5 more times]
 </hive_data>
Retrieved from "http://18.212.38.72/index.php?title=Change:_Weather_Data&oldid=2063"