Fitting a Line to Data with JSvg (and a brief discussion of JavaScript Arrays)

In the last post, we covered the basics of plotting a scattergraph using JSvg. In this post we will fit a line to the data from the last post as well as review some concepts.
When plotting anything on most 2D graphs (not including pie charts), one needs x-y coordinate pairs. This is even true in bar graphs and histograms (although generally the programs used to plot these other graph types will automatically generate the x-axis data). In JavaScript, the easiest way to store such data is in arrays. A one-dimensional JavaScript array can be likened to a list. It can be a list of numbers:

     var myArray = [1,2,3,4,5];

or

     var myArray = [1,5,23,43,2,7];

or even

     var myArray = [,,,2,3,4];

Where in the last case, the first three elements of the array are undefined. Arrays can also be lists of other things such as strings (representing characters or words):

     var myArray = ['Bob','Chris','a','b','c'];

In fact, arrays can be lists of other arrays:

     var arr1 = [1,2,3];
     var arr2 = [4,5,6];
     var arr3 = ['a,'b','c','d'];
     var myArray = [arr1,arr2,arr3];

Notice in the last case that the outer array (myArray) contains the other previously defined arrays. Also notice that not all arrays contain the same sort of thing. The first two inner arrays contain numbers, while the last one contains strings (in this case letters). Notice also, that the inner arrays are not all of the same length. This is allowable in JavaScript, and once you get used to it, it brings a great deal of flexibility to the table. In this last example, the array, myArray, could also be written like this:

     var myArray = [ [1,2,3], [4,5,6], ['a','b','c','d'] ];

or for clarity:

     var myArray = [ 
                     [1,2,3], 
                     [4,5,6], 
                     ['a','b','c','d'] 
                   ];

although this latter form takes up more space. To see how to access an element of this array, we could type into the Web Console:

     >> myArray = [ [1,2,3], [4,5,6], ['a','b','c','d'] ];

which would return:

     >> [objectArray]

Clicking on [objectArray] in the console results in the following:

At the right hand side, the contents of the [objectArray] are shown.  You can see that it consists of three inner (or sub) arrays and that its length is three.  The indices of the sub arrays are shown at the left  as 0, 1, and 2.  If we click on the third item in the array (index of 2), we see the following:

which exposes the content of the third inner array.  Let's say that we wish to access the content of the first inner array (index 0) from command line, we could type:

     myArray[0]

Which would result in another [object Array]. If we wished to access the first element of this array, we would type:

     myArray[0][0]

And in this case, since it is just a number, we would see its value (1) returned above in the console.

In general, in the future, we will use the language "zeroeth element" to refer to index 0 of an array, "first element" to refer to index 1 of an array and so on.  This is in keeping with JavaScript's convention to start indexing arrays at zero rather than one. 

After this rather lengthy preamble, we are ready to discuss fitting a line to the data from before.  The previous script was as follows:

     
    var R = 0.02275;   // m
    var Ncoils = 100;  // Number of coils
    var Ntrials = 6;   // Number of trials 
                   //   (presume we sample resistance every 100 coils)
    var pi = Math.PI;  // Use pi as a short hand for the JavaScript constant Math.PI
    var L = new Array();   // Define the array.  It's empty now.  
    for (var i=0; i< Ntrials; i++) {
         var Ntot = (i+1)*Ncoils;  // Total number of coils this sample.
         L[i] = 2*pi*R*Ntot;   // This is the ith value of length in the array
    }

    resist = [0.6,1.1,1.6,2.1,2.7,3.5];   // Ohms

    // Graphics commands
    optsGraphTitles[0].name = 'Length (m)';    // x-axis title
    optsGraphTitles[1].name = 'Resistance (\\Omega )';  // y-axis title
    optsGraphTitles[2].name = 'Resistance vs Length';   // main title
    optsGraphTitles[3].name = 'for a coil of wire';     // sub title

    optsLegend = false;

    drawSeriesPlot([L],[resist], 0,0, 'resistivityPlot.svg');

The idea is that we would be fitting a line to the x-y data represented by the array L and resist respectively. The big picture for physics students is that we are trying to determine the resistivity of the wire based on measurements of electrical resistance at given lengths, and knowledge of the wire's cross-sectional area (in this case: A = 4.64e-7 m²). The governing equation here is:

R = ρ L/A

where ρ is the resistivity. So if y corresponds to R and x corresponds to L, then ρ/A is the slope.
For quick fits, we can use the JSvg function "doFit()" which returns, you guessed it, an array! The syntax of the command is:

     
     fitObj = doFit(xvector, yvector, fitOrder, precisionOfoutput);

where xvector is the x-array, yvector is the y-array, fitOrder is 1 for linear, 2 for quadratic, 3 for cubic (and so on), and precisionOfoutput is the number of decimal places to show in the output. The function doFit returns an array that contains the following:

     
     fitObj[0] = 1D array containing the predicted y values for each x data point
     fitObj[1] = equation of fit (t is default variable)  (String)
     fitObj[2] = 1D array containing the coefficients of fit starting with highest order term 

There is a lower level function called by doFit() named polyCor() which contains much of the statistical information about the fit, but that's for another post.
So to modify the code from the prior and get a fit, we need to call doFit() with the appropriate arguments, and then extract the information we wish to display. Finally, we need to put it on our graph.

     
    var fitObj = doFit(L,resist,1,4);

We can then extract the objects we need and calculate the resistivity as follows:

     
// ********************* New code block ***********************************

    var fitObj = doFit(L,resist,1,4); 
    var fitData = fitObj[0];
    var fitEqu = fitObj[1];
    var slope = fitObj[2][0];

    var A = 4.64e-7;    // m^2
    var rho = slope*A;

    fitEqu = 'R = '+fitEqu.replace('t','L');
// ******************** End new code block ********************************

In the last equation, we have done some other formatting. By default the equation uses "t" as the x-variable. Let's replace it with "L", and then prepend "R = " on the front end to actually make it into an equation.
The only other item of substance is getting this information onto the graph. Recall that the function drawSeriesPlot() takes 2D arrays as its first two arguments. We recast this line as follows:

     
    drawSeriesPlot([L,L],[resist,fitData], 0,0, 'resistivityPlot2.svg');

What we have done is added another 1D array to both the x and y arrays in order to graph a second series on the plot. Below is the final code with all the bells and whistles.

     
    var R = 0.02275;   // m
    var Ncoils = 100;  // Number of coils
    var Ntrials = 6;   // Number of trials 

                       //   (presume we sample resistance every 100 coils)
    var pi = Math.PI;  // Use pi as a short hand for the JavaScript constant Math.PI
    var L = new Array();   // Define the array.  It's empty now.  
    for (var i=0; i< Ntrials; i++) {
         var Ntot = (i+1)*Ncoils;  // Total number of coils this sample.
         L[i] = 2*pi*R*Ntot;   // This is the ith value of length in the array
    }

    var resist = [0.6,1.1,1.6,2.1,2.7,3.5];   // Ohms

    // ********************* New code block ***********************************

    var fitObj = doFit(L,resist,1,4); 
    var fitData = fitObj[0];
    var fitEqu = fitObj[1];
    var slope = fitObj[2][0];

    var A = 4.64e-7;    // m^2
    var rho = slope*A;

    fitEqu = 'R = '+fitEqu.replace('t','L');
    // ******************** End new code block ********************************

    // Graphics commands
    optsGraphTitles[0].name = 'Length (m)';    // x-axis title
    optsGraphTitles[1].name = 'Resistance (\\Omega )';  // y-axis title
    optsGraphTitles[2].name = 'Resistance vs Length';   // main title
    optsGraphTitles[3].name = '\\rho = '+rho.toExponential(3)+' \\Omega m; '  // sub title

    // optsLegend = false;   // Old code
    optsLegend = true;      // default
    optsSeries[0].name = 'Data'
    optsSeries[1].itype = 1;      // make it a line
    optsSeries[1].name = fitEqu;  // show the equation of fit  
    xprec = 1;               // Set the precision on the x axis

    // Old code
    //drawSeriesPlot([L],[resist], 0,0, 'resistivityPlot.svg');
    // New code
    drawSeriesPlot([L,L],[resist,fitData], 0,0, 'resistivityPlot2.svg');

The resulting graph looks like this.

 

 

 In the next post, I'll give some more sample codes for plotting fits and expose some more graphics parameters for adjusting your plots' appearances.  Cheers!

My First Script: Plotting Data

This post is the first in a series designed to show you how to use OT/JSvg to analyze data.  There's a whole lot to JSvg, so rather than attempt to cover everything at once, we'll take it piecemeal.  Perhaps the simplest analysis used in many science classes the plotting of data.  OT/JSvg makes this relatively easy to do using regular JavaScript, the linga franca of the internet.  But to get started, you will need to get the Analysis Console open first.  In Firefox, go to the Tools option in the Menu Bar, and select "Start Analysis Console".  If you don't see it, you don't have OT/JSvg properly installed, and I'll refer you to the previous post.   At this point, you will see the console pop up.

Figure 1: A blank console after it has first popped up.

The purpose of the console is to allow users to easily make use of the JSvg libraries and see results in an organized manner.  The console also allows you to open up both the Web Console, a command line interface that allows you to type commands and do calculations using the scope of the window, and ScratchPad, a simple JavaScript programming editor with syntax highlighting.  Today, we're going to use ScratchPad to write the code.  The results will be displayed in the Analysis Console.

Let's start with the data.  In one particular physics lab, students measure resistance of various lengths of wire.   Of course, as wire is conductive, it takes a whole lot of wire to provide a measurable resistance with most multimeters.  So we use coils of wire rather than linear lengths.  Then the students make their measurements of resistance every so many coils and calculate the length based on the number.  If we know the radius of the coil, and the number of coils, it is pretty straight forward to calculate the total length.

    var R = 0.02275;   // m
    var Ncoils = 100;  // Number of coils
    var pi = Math.PI;  // Use pi as a short hand for the JavaScript 
                       // constant Math.PI
    var L = 2*pi*R*Ncoils;   // This is the total length;

But wait! We can do better than this. Let's say that we make six measurements at increments of 100 coils. We could write out each calculation, OR, we could do one better and use a loop and arrays. Loops allow you to repeat calculations an arbitrary number of times without having to type out each one. In short, they are time savers. Arrays allow you to store results. While in general, they are of a related type (for example lengths of wire), in JavaScript arrays can consist of all kinds of unrelated objects. More on that later. Let's change the code to this.

    var R = 0.02275;   // m
    var Ncoils = 100;  // Number of coils
    var Ntrials = 6;   // Number of trials 
                       //   (presume we sample resistance every 100 coils)
    var pi = Math.PI;  // Use pi as a short hand for the JavaScript constant Math.PI
    var L = new Array();   // Define the array.  It's empty now.  
    for (var i=0; i< Ntrials; i++) {
         var Ntot = (i+1)*Ncoils;  // Total number of coils this sample.
         L[i] = 2*pi*R*Ntot;   // This is the ith value of length in the array
    }

The loop is called a for loop and it works like this. Starting with an index (i) of 0, do this loop while i is less than Ntrials (6 times), incrementing i by 1 each time (i++). The result is that the array L will contain six values based on a formula that was executed six times, once per loop. The formula

    Ntot = (i+1)*Ncoils

calculates the total number of coils in that iteration. The first time, since the loop index i is zero, the result will be:

    Ntot = (0+1)*100 = 100 coils.

The second time, when the index is one, the result will be:

    Ntot = (1+1)*100 = 200 coils.

And so on. We will use the L array as our x values. For y values we have our measured resistances.

   resist = [0.6,1.1,1.6,2.1,2.7,3.5];   // Ohms

Note that there are six values in this array and that they are separated by commas. To this point, one could run the ScratchPad code without the help of JSvg since it's pure JavaScript. However to plot these data, we need to use JSvg graphics functions. The simplest scatterplot function available (without going low-level) is

    drawSeriesPlot([2D xarray], [2D yarray], 0,0, filename)

where the first argument is your x data, and the second is your y data. These are 2D arrays rather than 1D arrays by default so that if you have multiple series of data, you can plot them in a single command. The third and fourth arguments of the function are deprecated. For now, just use zeroes every time you call the command. The final argument is the filename in string form. This means that there should be quotes around the name. It is important that you specify the ".svg" extension to ensure that the graph displays properly. So here's the plotting command.

    drawSeriesPlot([L],[resist], 0,0, 'resistivityPlot.svg');

The 1D arrays L and resist are each enclosed in brackets to make them into 2D arrays. If you are starting with 2D arrays, you will not need to do this. When you run this command your graph will appear in the upper left-hand pane of the console (current graph), and when you mouse-over it, its background will turn orange. Clicking on the orange will make it appear in the main frame.

Figure 2: Results thus far

 You will notice a couple of things at this point: first, that there are no labels on the axes, and second, that there is a legend with a series called "unnamed0" displayed.  Let's add the titles via code (and later we'll see how to add them post-coding). The following commands create titles for your graph.

    optsGraphTitles[0].name = 'Length (m)';    // x-axis title
    optsGraphTitles[1].name = 'Resistance (\\Omega )';  // y-axis title
    optsGraphTitles[2].name = 'Resistance vs Length';   // main title
    optsGraphTitles[3].name = 'for a coil of wire';     // sub title

Figure 3: The graph after titles have been added.

If you're sharp, you'll notice that the \\Omega turned into an upper case omega in the y-axis title.  Generally, you can create Greek letters in this manner.  However, you will also need to remember to leave a space after the letter for the parser to work correctly.

We still have the issue of the legend.  One generally doesn't use the legend for single series plots.  So let's remove it.  Set the variable optsLegend to false to suppress its appearance.

    
    optsLegend = false;

The final graph (for today) will look as follows:

Figure 4: The final graph, no legend and with titles.

It is, of course, possible to change things around without dropping to ScratchPad. When your graph is displayed within the Analysis Console, you can click to change most items. Clicking on a title allows you to change its font or position. Clicking on an axis allows you to set the color of the grid lines, the number of divisions on the axis, the use of minor ticks, the precision of the displayed numbers, and so on. The font of the axis matches the font of the axis title. Should you forget to add titles, mouse over the upper left hand side of the graph, and after a couple of seconds, a title dialog will appear. In the meantime, here's the code for the day.

    
    var R = 0.02275;   // m
    var Ncoils = 100;  // Number of coils
    var Ntrials = 6;   // Number of trials 
                   //   (presume we sample resistance every 100 coils)
    var pi = Math.PI;  // Use pi as a short hand for the JavaScript constant Math.PI
    var L = new Array();   // Define the array.  It's empty now.  
    for (var i=0; i< Ntrials; i++) {
         var Ntot = (i+1)*Ncoils;  // Total number of coils this sample.
         L[i] = 2*pi*R*Ntot;   // This is the ith value of length in the array
    }

    var resist = [0.6,1.1,1.6,2.1,2.7,3.5];   // Ohms

    // Graphics commands
    optsGraphTitles[0].name = 'Length (m)';    // x-axis title
    optsGraphTitles[1].name = 'Resistance (\\Omega )';  // y-axis title
    optsGraphTitles[2].name = 'Resistance vs Length';   // main title
    optsGraphTitles[3].name = 'for a coil of wire';     // sub title

    optsLegend = false;

    drawSeriesPlot([L],[resist], 0,0, 'resistivityPlot.svg');

In the next post, we'll talk about how to fit a line to these data. Cheers!

Another Update

One of the hard parts about developing Firefox extensions is that you are developing for a moving target.  The internals of Firefox (which you want to be able to access) are very complex, and often despite trying to keep up with the development roadmap, one still sometimes misses a step.  When you add that to a complicated codebase like OT/JSvg, it's a recipe for trouble every so often.

So last night, I was running my physics lab and everything was going along smoothly.  Students had updated OT/JSvg or installed it for the first time without any issues and were busily collecting data.  Code editing went smoothly: students entered their data into arrays and were plotting the results, when one of the students running Windows noted that the left hand panes where one can click to view recent graphs had stopped working.  After going around to all the machines which I had dutifully updated to FF26.0 and OT/JSvg 0.11.3, I saw that it was true for all the Windows users.  The sole MacOSX person had no such problem nor did I (running Linux). 

The code does OS detection as it loads and sets the variable OS with its findings.  Then later when OS specific code is run, it makes use of this.  It was clear that this was borked.  After some exploring I found that there is new (??) internal object called OS which contains a wealth of info about the operating system.  Therefore, tests as such:

if (OS == 'Windows') {

// Do windowy stuff here

} else {

// Do posix (Linux/MacOSX) stuff here

}

were breaking since OS was an object and not a string.  Having discovered this, I fixed the JSvg part of the code (for a quick release) and am now moving into looking at the OT portion.

While I was at it, I pushed forward some cosmetic changes to the Analysis Console to make things more findable or otherwise legible.  There are now also tooltips for all the buttons.  Here's a preview:

Graph of last night's data with updated GUI.

Anyhow, that's it for now.  Enjoy the 0.11.4 release.

Installing OpenTrack/JSvg

Being a Firefox extension, OpenTrack is relatively easy to install.  If you are already driving Firefox, you can simply point your browser here to download and install the latest edition.  The video below shows the installation process.


 (If you're not seeing a video, here is the link)

Due to Mozilla's very cool extension architecture, once OT is installed, updates are pushed out automatically unless you really don't want this.  Considering that many updates are either bug fixes or new features, it's not clear that you wouldn't want OT to auto update.

A note on this:  If you are like many people and never shut down your browser, you may not get automatic updates.  In which case, you can either click on Tools|Add-ons to get the following tab.

 Near the search bar on the upper right hand side, there is a tool symbol.  This can be clicked to check for updates.  Note that you can also turn automatic updates on or off.

Finally, to start up the Analysis Console, be sure your Firefox menu bar is visible at the top, go into the Tools pulldown menu, and select (in green, except for Mac) "Start Analysis Console."    This will open a window as shown below.

 

The Analysis Console exists to give you access to the statistics and graphics functions embedded in OpenTrack/JSvg.  At this point you have several options:

• If all you want to do is graph something without a lot of fanfare, click on the "Create Plot" button.  
• If you want to do some calculations (that are not persistent), click on the "Toggle Console" button.  The console works as a very nice line calculator. For example, you can type: "a = 2; b=3; c = a*b;" to do a simple math problem.
• If you want to write code, click on the "Open Scratchpad" button.  ScratchPad runs JavaScript code in the context of the window behind it. (The same is true for the Web Console.)  All of the functions and graphics commands of OpenTrack/JSvg are available to you so long as this is true.

 You can also see sample plots and very simple analyses as well as get a sense of some of the functions that are available by pressing the labeled buttons.    In the next post, I will go through some simple examples involving the use of JSvg.

[Edit] One functionality in this release has been curtailed by an internal change in variable.  Mozilla has now turned OS into an object, so my own OS sniffing is sometimes burping as of OT 0.11.3.  I've got a patch in the works, which seems to be working as of now.   I'll post an update tomorrow.

What is JSvg?

In 2005, I was asked by a colleague to write a 2D video analysis package that would work on any operating system and which was freely available to students.  In order to display kinematic results from tracking, a graphing library was needed.  At the time, there was no jQuery, Raphael, svgjs or other Javascript framework for rapid development or graphics.  Nor were there any statistical libraries available.  To meet the requirements of relatively easy development, easy access for users, and since data were to be processed locally, Firefox (XUL+Javascript) was chosen as the platform.  Likewise, the SVG graphical format was chosen since it was possible to create vector graphics that were scriptable and with little work, could be publication quality.  The result of this effort is JSvg.

Fig 1: View of original OT from 2005

JSvg has, to this point, been part of the OpenTrack (OT) Firefox extension.  You won't find OT in Mozilla's addons due to its size (more than 30,000 lines of code), and the need for quick fixes and rereleases mid-semester.  There was no time for the Mozilla approval process.  Rather OT has been released via two sources simultaneously: Sourceforge, and niiler.com (xpi file). The first site was chosen for visibility whereas the second was chosen for automatic updating (which Sourceforge doesn't allow). 

While still embedded as part of OpenTrack, since 2005, JSvg has evolved to become a rather robust graphical and analysis program, run locally as a Firefox extension.  It makes ample use of Firefox's Web Console and Scratchpad and has recently obtained a console for easy navigation of results.

Figure 2: Some of the latest from JSvg showing analysis capabilities comparing different distributions.

So what can it do?  In addition to the many types of plots, it is also capable of running a number of statistics (t-tests, ANOVA, regression, descriptive stats...), signal processing (smoothing, fitting, FFT, derivatives), linear algebra, and more.

The next version of JSvg will be released shortly, both alongside OpenTrack, and for the first time, as its own extension.  This blog will serve as a tutorial to JSvg showing users how to do certain analyses, coding, or graphics.  In a number of cases, I will show benchmarks versus the R statistical language (which is my old standby).  [Note: due to limitations of Javascript, the precision of JSvg results compared to R are not nearly as good.  But that said, the results are often comparable to two or three sig figs.]

Until next time.