Live Data Graphing - Revision history

Mparadis at 19:00, 7 June 2017

2017-06-07T19:00:08Z

← Older revision		Revision as of 19:00, 7 June 2017
Line 277:		Line 277:
	Ready to run the code itself? Here it is:		Ready to run the code itself? Here it is:

	[{{SERVER}}/~~wiki~~/images/6/64/Temp_visual_logger.py Temp_visual_logger.py]		[{{SERVER}}/docs21/images/6/64/Temp_visual_logger.py Temp_visual_logger.py]

	You can run the code as described by operating system on the [[Language - Python]] page.		You can run the code as described by operating system on the [[Language - Python]] page.

Mparadis: /* Putting it All Together */

2013-06-24T20:51:36Z

Putting it All Together

← Older revision		Revision as of 20:51, 24 June 2013
Line 277:		Line 277:
	Ready to run the code itself? Here it is:		Ready to run the code itself? Here it is:

	[~~[File:~~Temp_visual_logger.py]]		[{{SERVER}}/wiki/images/6/64/Temp_visual_logger.py Temp_visual_logger.py]

	You can run the code as described by operating system on the [[Language - Python]] page.		You can run the code as described by operating system on the [[Language - Python]] page.

Burley at 17:54, 8 August 2012

2012-08-08T17:54:12Z

← Older revision		Revision as of 17:54, 8 August 2012
Line 96:		Line 96:
	===Libraries Needed===		===Libraries Needed===

	First, make sure you have followed the Getting Started guide for your Temperature Sensor, found its ~~product page on our~~ [~~{{SERVER}} main website~~].		First, make sure you have followed the Getting Started guide for your Temperature Sensor, found in its [[:Category:UserGuide\|user guide]].

	To run the code in this guide, you will need the following installed:		To run the code in this guide, you will need the following installed:

Cora at 14:10, 29 June 2012

2012-06-29T14:10:46Z

Cora at 21:42, 27 June 2012

2012-06-27T21:42:44Z

← Older revision		Revision as of 21:42, 27 June 2012
Line 1:		Line 1:
	<!-- CATEGORIES -->		<!-- CATEGORIES -->
	[[Category:fileIO]]		[[Category:fileIO]]
			[[Category:Application Guides]]

	The project described here is for using various Python libraries for creating a MATLAB-like graphing program, which scrolls the graph in real time.		The project described here is for using various Python libraries for creating a MATLAB-like graphing program, which scrolls the graph in real time.

Cora: Created page with " Category:fileIO The project described here is for using various Python libraries for creating a MATLAB-like graphing program, which scrolls the graph ..."

2012-06-25T17:03:33Z

Created page with " Category:fileIO The project described here is for using various Python libraries for creating a MATLAB-like graphing program, which scrolls the graph ..."

New page

[[Category:fileIO]]

The project described here is for using various Python libraries for creating a MATLAB-like graphing program, which scrolls the graph in real time.

{|
|Practical concepts covered are (click on links to see other projects on that topic):
* Data Logging
** [[:Category:FileIO|Writing to a text file]]
** Using Phidgets with Python
** (Alternative to) Using Phidgets with MATLAB
* Analyzing your Phidget Data
|width="45px"| 
|[[Image:Livegraph-temp-screenshot.png|600px|link=|alt=]]
|}

{| style="border:1px solid darkgray;" cellpadding="5px;"
|-
|'''Time:'''
|About one day of work installing libraries and adapting code
|-
|'''Special Needed Tools:'''
|None
|-
|'''Materials and Phidgets:'''
|One Phidget Thermocouple board, USB cable, and thermocouple
|}

As with any of our [[Application Guides|described projects]], Phidgets takes care of the electrical component design. Here, we show how to collect the data and display it in a slick manner.

If you just want to log data and save it to a file, you probably want to start with our [[Data Logging With a Thermocouple]] application guide, which does exactly that. This application guide focuses on the graphical display of data over time.

__TOC__

==Introduction==

This project is a live GUI graphing application that displays data in real time, in a scrolling graph. This type of application - written in Python - can capture low-speed data and display it, and hence is a good alternative to MATLAB graphics when you wish to use events in your code. Python also offers fast numerical processing via the NumPy library and so can be a numerical alternative to MATLAB as well.

Although we use a Phidget thermocouple board (such as the [http://www.phidgets.com/products.php?product_id=1051 Phidget 1051 1-Thermocouple Board] or the [http://www.phidgets.com/products.php?product_id=1048 Phidget 1048 4-Thermocouple Board]), this code can be adapted for nearly any Phidget, as shown by the use of it for a Phidget DC Motor Controller in the [[#Extending with Other Phidgets and Widgets|Extra Credit section]].

The GUI program has automatic axes scaling and scrolling as this smaller-than-reality (and low resolution) animated GIF shows:

[[File:out_sm.gif|link=|alt=]]

==Phidgets==

To use the code in this guide as-is, you will need to have a Phidget Temperature Sensor and a thermocouple.

Some example ways to set this up are:
{| style="border:1px solid darkgray;" cellpadding="5px;"
|- style="background: #c0c0c0;"
!Example Setup
!Strengths
|- style="background: #efefef;"
|
*[http://www.phidgets.com/products.php?product_id=1051 1051 - PhidgetTemperatureSensor 1-Input], plus
*[http://www.phidgets.com/products.php?product_id=3107 3107 - TPK-01 Bead Probe K-Type Thermocouple (-50°C to +200°C)]
| Basic thermocouple setup
|- style="background: #efefef;"
|
*[http://www.phidgets.com/products.php?product_id=1048 1048 - PhidgetTemperatureSensor 4-Input], plus
*[http://www.phidgets.com/products.php?product_id=3107 3107 - TPK-01 Bead Probe K-Type Thermocouple (-50°C to +200°C)]
| Ability to measure multiple points
|- style="background: #efefef;"
|
*[http://www.phidgets.com/products.php?product_id=1051 1051 - PhidgetTemperatureSensor 1-Input], plus
*[http://www.phidgets.com/products.php?product_id=3110 3110 - TPK-01H Bead Probe K-Type Thermocouple (-50°C to +785°C)]
| Wide temperature range thermocouple
|}

Of course, these are only examples - any thermocouple with either Temperature Sensor Phidget board will work.

We will log data from thermocouple port 0. On a one-input board, this is the only thermocouple input; on a four-input board, make sure you have a thermocouple plugged in to sensor port 0:

[[Image:App guide templogger 1048Leads.png|600px|link=|alt=]]

Once you have your Phidget assembled with a thermocouple in Port 0, continue on to installing libraries.

==Setting up and Installing Libraries==

The libraries used here are primarily Python libraries. This makes them a good alternative to the graphics in MATLAB. Python and MATLAB share many commonalities, such as:
* They are interpreted languages
* They have good online documentation and a large user base
* They run on all major operating systems, without much code difference
* You can use them to write code to do matrix analysis, or GUI display

However, there are a few advantages to using Python, especially with Phidgets, instead of MATLAB:
* The Phidget API in MATLAB is essentially using the C library calls from MATLAB. In Python, you can make calls directly.
* You can listen to and use events in Python, but not in MATLAB.
* Python is free.
* Python has an extensive number of libraries to interface your Phidget with other programs, such as R, GRASS and PostGIS, Octave (free MATLAB), online CGI applications, and more.

In this guide, we will use some of the GUI and matrix/vector math libraries in Python. These will allow us to create a full application which graphs, scales, exports data, exports images, and more in under 350 lines of code.

===Libraries Needed===

First, make sure you have followed the Getting Started guide for your Temperature Sensor, found its product page on our [http://www.phidgets.com main website].

To run the code in this guide, you will need the following installed:
# [http://www.python.org Python]
# The Phidget Libraries installed for [[Software Overview#Operating System Support|your operating system]]
# The Phidget Python module installed and tested, as described on our [[Language - Python]] page
# [http://www.wxpython.org WxPython]
# [http://matplotlib.sourceforge.net Matplotlib]
# [http://numpy.scipy.org NumPy and SciPy] (these provide much of the the matrix and array functionality in MATLAB)

Installing hints by operating system are below.

===Windows and Mac OS X===

First, figure out which version of Python you want to download.

'''Note''' that as of the writing of this guide, there was only ''one'' version of Python which would run NumPy/SciPy, Matplotlib, and WxPython, and it is 32-bit:
* WxPython [http://wxpython.org/download.php#stable runs with Python 2.6 or 2.7]
* Matplotlib [http://matplotlib.sourceforge.net/users/installing.html#build-requirements runs with Python 2.X (2.4 or later, but not 3), and then only 32 bit]
* NumPy [http://www.scipy.org/Download seems to prefer Python 2.6], on reports from our customers.
However, NumPy 1.5 release notes says it is compatible with later Python 2.X versions, so you may be able to get it to work.

So as of this writing, you'll want to download Python 2.6, 32 bit version.

If this has changed, on Windows and Mac, it can be quite obtuse to figure out which versions you need for all of the software. If you can use a 32-bit version, the 32-bit packages from [http://www.enthought.com/products/epd.php Enthought Python] are free and include all of the needed libraries except the Phidget components. They also offer 64-bit versions for a subscription as of this writing, but you must do your own research there as we are not affiliated with them. You can also try [http://code.google.com/p/pythonxy/ Python(x,y)], another option to get all of the libraries installed on a non-Linux system.

Once you have the right version of Python installed, you can install the Phidget libraries as discussed in the Linux section of our [[Language - Python]] page.

===Linux===

To install these libraries on Linux, it usually suffices to do:

<div class=source>
<syntaxhighlight lang=bash>
sudo apt-get install python python-wxtools python-matlibplot python-numpy
</syntaxhighlight>
</div>

And then you can install the Phidget libraries as discussed in the Linux section of our [[Language - Python]] page.

==Algorithm==

The algorithm is stripped down to:
# A single timer which re-scales and re-draws the plot (i.e. the [[#GUI|GUI]]), and
# A Phidget event function which writes [[#Data|data]] to the array that gets drawn

For simplicity here, and because only one thread is ever writing to the data array, mutual exclusion is not used. This can create interesting background behaviour, as discussed in the extra credit [[#Mutual Exclusion|Mutual Exclusion]] section below.

===GUI===

The best way to learn what the GUI does and how it works is perhaps to refer to the code itself and play with it. But we offer some explanation here for the [[#Back-End Rendering|back end]] and for the [[#Set Up and Scaling|setup and scaling]] of the graph.

====Back-End Rendering====

To render graphics in Python, we simply pass commands to the back-end. In this case, we'll be passing commands to matplotlib, which will in turn pass its commands to the GUI back-end. Hence, we import both WxPython (our backend) and matplotlib at the same time:

<div class="source">
<syntaxhighlight lang=python>
import matplotlib # Provides the graph figures
matplotlib.use('WXAgg') # matplotlib needs a GUI (layout), we use wxPython

from matplotlib.figure import Figure
from matplotlib.backends.backend_wxagg import FigureCanvasWxAgg as FigCanvas
</syntaxhighlight>
</div>

After setting up the main structure for the GUI (see the code for plot layout, etc), the actual ''drawing'' that occurs, which makes the graph scroll - happens via a timer. This timer triggers a re-draw of the canvas (via the {{Code|onRedrawTimer()}} function). Here is where we attach the timer to the redraw function:

<div class="source">
<syntaxhighlight lang=python>
# This times fires every 100 ms to redraw the graphs
self.redrawTimer = wx.Timer(self)
self.Bind(wx.EVT_TIMER, self.onRedrawTimer, self.redrawTimer)
self.redrawTimer.Start(100)
</syntaxhighlight>
</div>

Then, the redrawTimer function can simply be:

<div class="source">
<syntaxhighlight lang=python>
def onRedrawTimer(self, event):
self.drawPlot()
</syntaxhighlight>
</div>

And in drawPlot(), we can read the array, set the axes, and draw it. Because the [[#Data|data is updated by Phidget events]], we just grab whatever is in the array and not worry about doing the updating of the data itself in the GUI. If a new point was added

In the attached code, we also have a handler in {{Code|onRedrawTimer}} to catch exceptions for when the array lengths do not match (i.e. we are reading the arrays when they are being written to). This is discussed more in the [[#Mutual Exclusion|Mutual Exclusion]] section below.

====Set Up and Scaling====

The colour and style of the lines and plot are handled in an initialization function once - i.e. {{Code|initPlot()}}. Then those settings are remembered and activated within matplotlib when we call the set_data and draw functions later for every redraw. For example, the yellow temperature line style (the blue line is a running average) is set in {{Code|initPlot()}} as:

<div class="source">
<syntaxhighlight lang=python>
self.plotData = self.axes.plot(
self.data,
linewidth=1,
color=(1, 1, 0),
)[0]
</syntaxhighlight>
</div>

This is also the line where the actual data array connected with the line - i.e. {{Code|self.data}} - gets attached for the first time.

Then the {{Code|drawPlot()}} function (via the redraw timer) draws that line with:

<div class="source">
<syntaxhighlight lang=python>
self.plotData.set_data(np.array(self.time), np.array(self.data))
self.canvas.draw()
</syntaxhighlight>
</div>

The only data that gets drawn visually to the user is the data between the x/y minimum and maximum values on the plot. So we can get the plot to scale automatically in the vertical direction by setting the axes to the limits of the data, plus a buffer - this happens in {{Code|drawPlot()}} right before the actual refresh draw occurs.

We can also get the x-data to scale the same way, such that the far right side of the x-axis always matches the last point in our data. If the x-axis is always a fixed width, this makes the graph appear as if the data are scrolling through it. You can play around with these parameters to make the vertical axis fixed, or the x-axis expand as data is collected, or any number of display options.

===Data===

The data for a temperature sensor is straightforward as you simply have to display the sensor value. Other Phidgets, such as the [[#Extending with Other Phidgets and Widgets | example below]] using a Phidget Motor Controller, need various levels of calculation prior to display.

The main complexity here is creating a data array which remains the same size. If we just keep adding (appending) data to an array

If you have a normal desktop-power computer, writing to a data array within your event handling function should work fine. In this example, the GUI components put a large load on your processor, so if your computer is not relatively fast, you might have to set up a [[General Phidget Programming#Logic Code|data polling method]] rather than using events. But here, we attach the function {{Code|temperatureChanged}} to the Phidget data handler:

<div class="source">
<syntaxhighlight lang=python>
thermocouple.setOnTemperatureChangeHandler(temperatureChanged)
</syntaxhighlight>
</div>

And then our {{Code|temperatureChanged()}} function records the time and further calls an {{Code|updateTemperature()}} function with the time and temperature. The {{Code|updateTemperature()}} function looks like this:

<div class="source">
<syntaxhighlight lang=python line start=1>
def updateTemperature(self,temperature,time):

avgSum = 0
window = self.avgOver if self.avgOver < len(self.data) else len(self.data)
for a in range(window):
avgSum = avgSum + self.data[(-1 * a)]
avgSum = avgSum / window
self.avgData.append(float(avgSum))

self.data.append(float(temperature))
self.time.append(float(time))
self.totalTime = time

if len(self.data) > self.maximumArray:
global looped
if not looped:
print "Starting re-use of graph arrays for efficiency..."
looped = True
self.time.pop(0)
self.avgData.pop(0)
self.data.pop(0)

self.fileTime.append(float(time))
self.fileTemperature.append(float(temperature))
</syntaxhighlight>
</div>

* Lines 3-7 calculate the average. This is over the latest data set of length {{Code|window}}.
* Lines 8-12 appends all the data of interest to each array.
** The average goes onto {{Code|self.avgData}}
** The time goes onto {{Code|self.time}}
** The temperature goes onto {{Code|self.data}}
* Lines 14-21 checks to see if the array exceeded our maximum.
** If it does, the first data points are popped of and thereby discarded.
* Lines 23-24 save both time and temperature again. The {{Code|fileTime}} and {{Code|fileTemperature}} variables save all data to export to a text file later if desired.

And that's it for recording data. In this way, all data is saved in the file variables, and a moving array of fixed length handles the data to be shown on the graph.

Using a fixed-width moving array makes the code run almost forever. The file variables get longer and longer, but nothing is done with them until a text file is written. Not using a fixed-width array for the displayed data, however, will eventually cause enough lag in the GUI redraws to slow - and then stall - the graphical output.

==Putting it All Together==

Ready to run the code itself? Here it is:

[[File:Temp_visual_logger.py]]

You can run the code as described by operating system on the [[Language - Python]] page.

==Extra Credit==

===Exporting Data===

If you run the program, you may note that there is a {{Code|File}} menu that appears in the normal upper menu location. The actual location depends on your OS - it may be on the upper top bar on Mac or Linux Unity.

The File menu has two options of interest here - Save as Image, and Save as CSV. If you download and read the code, you'll see the functions (near the bottom of the file) that provide the drive behind this, and they're quite simple. This allows you to save ''all'' of the current data in a CSV, or, as an image, the part of the data on the current graph:

[[Image:Livegraph-temp-saveaspng.png|800px|link=|alt=]]

===Extending with Other Phidgets and Widgets===

Using the code as a place to start, you don't just have to have one scrolling graph, you can have many. You can also add buttons, sliders, input boxes, etc. and process on them to drive your Phidget and see the output.

For example, here is the live graph program adapted to control a DC motor for testing control loops - in this case, a PID (Proportional, Integral, Derivative) constant rotation loop. The inputs at the bottom allow the loop to be changed, and the two graphs allow both the state of the motor, and the state of the control loop, to be displayed at the same time:

[[Image:Livegraph-motor-example.png|800px|link=|alt=]]

===Mutual Exclusion===

We don't include mutual exclusion in the attached code. The best way to implement this code (and the most safe, for a commercial application) would be to write to the data array in a thread which communicates with the GUI thread that reads it. Reading would not be allowed until the writing thread had completed writing an entire event.

Without this mutual exclusion, there is a chance that the "draw" thread will trigger and the data from the Time or Data array without the other array matching it in length. This array mismatch error can be caught, and that draw event skipped. This is an annoyance, but will not mess up the data arrays themselves, because only one thread is writing to them (the event thread from the Phidget).

If, however, you were to deploy this code long-term, or wanted more than one function to write to the arrays, you would need to implement some sort of mutually exclusive method of writing to the data array.

← Older revision		Revision as of 14:10, 29 June 2012
Line 28:		Line 28:
	\|}		\|}

	As with any of our [[Application Guides\|described projects]], Phidgets takes care of the electrical component design. Here, we show how to collect the data and display it in a slick manner.		As with any of our [[:Category:Application Guides\|described projects]], Phidgets takes care of the electrical component design. Here, we show how to collect the data and display it in a slick manner.

	If you just want to log data and save it to a file, you probably want to start with our [[Data Logging With a Thermocouple]] application guide, which does exactly that. This application guide focuses on the graphical display of data over time.		If you just want to log data and save it to a file, you probably want to start with our [[Data Logging With a Thermocouple]] application guide, which does exactly that. This application guide focuses on the graphical display of data over time.
Line 38:		Line 38:
	This project is a live GUI graphing application that displays data in real time, in a scrolling graph. This type of application - written in Python - can capture low-speed data and display it, and hence is a good alternative to MATLAB graphics when you wish to use events in your code. Python also offers fast numerical processing via the NumPy library and so can be a numerical alternative to MATLAB as well.		This project is a live GUI graphing application that displays data in real time, in a scrolling graph. This type of application - written in Python - can capture low-speed data and display it, and hence is a good alternative to MATLAB graphics when you wish to use events in your code. Python also offers fast numerical processing via the NumPy library and so can be a numerical alternative to MATLAB as well.

	Although we use a Phidget thermocouple board (such as the [~~http://www.phidgets.com~~/products.php?product_id=1051 Phidget 1051 1-Thermocouple Board] or the [~~http://www.phidgets.com~~/products.php?product_id=1048 Phidget 1048 4-Thermocouple Board]), this code can be adapted for nearly any Phidget, as shown by the use of it for a Phidget DC Motor Controller in the [[#Extending with Other Phidgets and Widgets\|Extra Credit section]].		Although we use a Phidget thermocouple board (such as the [{{SERVER}}/products.php?product_id=1051 Phidget 1051 1-Thermocouple Board] or the [{{SERVER}}/products.php?product_id=1048 Phidget 1048 4-Thermocouple Board]), this code can be adapted for nearly any Phidget, as shown by the use of it for a Phidget DC Motor Controller in the [[#Extending with Other Phidgets and Widgets\|Extra Credit section]].

	The GUI program has automatic axes scaling and scrolling as this smaller-than-reality (and low resolution) animated GIF shows:		The GUI program has automatic axes scaling and scrolling as this smaller-than-reality (and low resolution) animated GIF shows:
Line 55:		Line 55:
	\|- style="background: #efefef;"		\|- style="background: #efefef;"
	\|		\|
	*[~~http://www.phidgets.com~~/products.php?product_id=1051 1051 - PhidgetTemperatureSensor 1-Input], plus		*[{{SERVER}}/products.php?product_id=1051 1051 - PhidgetTemperatureSensor 1-Input], plus
	*[~~http://www.phidgets.com~~/products.php?product_id=3107 3107 - TPK-01 Bead Probe K-Type Thermocouple (-50°C to +200°C)]		*[{{SERVER}}/products.php?product_id=3107 3107 - TPK-01 Bead Probe K-Type Thermocouple (-50°C to +200°C)]
	\| Basic thermocouple setup		\| Basic thermocouple setup
	\|- style="background: #efefef;"		\|- style="background: #efefef;"
	\|		\|
	*[~~http://www.phidgets.com~~/products.php?product_id=1048 1048 - PhidgetTemperatureSensor 4-Input], plus		*[{{SERVER}}/products.php?product_id=1048 1048 - PhidgetTemperatureSensor 4-Input], plus
	*[~~http://www.phidgets.com~~/products.php?product_id=3107 3107 - TPK-01 Bead Probe K-Type Thermocouple (-50°C to +200°C)]		*[{{SERVER}}/products.php?product_id=3107 3107 - TPK-01 Bead Probe K-Type Thermocouple (-50°C to +200°C)]
	\| Ability to measure multiple points		\| Ability to measure multiple points
	\|- style="background: #efefef;"		\|- style="background: #efefef;"
	\|		\|
	*[~~http://www.phidgets.com~~/products.php?product_id=1051 1051 - PhidgetTemperatureSensor 1-Input], plus		*[{{SERVER}}/products.php?product_id=1051 1051 - PhidgetTemperatureSensor 1-Input], plus
	*[~~http://www.phidgets.com~~/products.php?product_id=3110 3110 - TPK-01H Bead Probe K-Type Thermocouple (-50°C to +785°C)]		*[{{SERVER}}/products.php?product_id=3110 3110 - TPK-01H Bead Probe K-Type Thermocouple (-50°C to +785°C)]
	\| Wide temperature range thermocouple		\| Wide temperature range thermocouple
	\|}		\|}
Line 96:		Line 96:
	===Libraries Needed===		===Libraries Needed===

	First, make sure you have followed the Getting Started guide for your Temperature Sensor, found its product page on our [~~http://www.phidgets.com~~ main website].		First, make sure you have followed the Getting Started guide for your Temperature Sensor, found its product page on our [{{SERVER}} main website].

	To run the code in this guide, you will need the following installed:		To run the code in this guide, you will need the following installed: