1

In my trading software I generate a file like the following:

10:00:00.0000001    1.25333
...........................
...........................
14:58:48.5024961    1.23456
14:58:48.5024987    1.23455
14:58:48.5025003    1.23460
14:58:48.5025108    1.23458
...........................
...........................
17:59:59.9997788    1.21111

I have millions of items in total. I want to see graphic, especially "extremums" (time is "X" and value is "Y"). I need ability to "zoom" to interesting parts of the graphic etc.

Probably I can do that in Excel or I should better use some specific software. What can you recommend?

As I generate the file myself, I can change format as needed (use ";" as a delimeter, change time format etc.)

Update: How do I run octave in Windows?

Oleg@OlegPC ~
$ octave
GNU Octave, version 3.6.1
Copyright (C) 2012 John W. Eaton and others.
This is free software; see the source code for copying conditions.
There is ABSOLUTELY NO WARRANTY; not even for MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE.  For details, type `warranty'.

Octave was configured for "i686-pc-cygwin".

Additional information about Octave is available at http://www.octave.org.

Please contribute if you find this software useful.
For more information, visit http://www.octave.org/help-wanted.html

Read http://www.octave.org/bugs.html to learn how to submit bug reports.

For information about changes from previous versions, type `news'.

warning: unable to open X11 DISPLAY
octave:1>
1

I would use Matlab, but I guess that a little bit to scientific and might not be available to you. Octave seems to be a open source similar to Matlab.

The plot itself would be the easiest plot(x,y) having all abilities to zoom and pan. The data import wizard should also allow you to import cvs type of files if you just change your output a little bit.

x = [1.00 1.10 1.11 1.12 1.12 1.2]

y = [0.709364830858073 0.754686681982361 0.276025076998578 0.679702676853675 0.655098003973841 0.162611735194631]

Having the 1.12 twice:

enter image description here

10
  • will it corrrectly process situation when I have several Y for the same X? it's possible because sometimes Y changes more often than precision of X. Jul 18 '12 at 8:44
  • I don't know what you take as correctly but it can handle multiple Y to the same x. see edit
    – eactor
    Jul 18 '12 at 9:26
  • do you know how can I run octave on windows? I receive unable to open X11 DISPLAY error (updated description) Jul 18 '12 at 10:00
  • for an x11 display you need linux window manager like xming, but i would recommend a virtual ubuntu using virtualbox
    – eactor
    Jul 18 '12 at 10:29
  • i'm using cygwin, how can I do that using cygwin? Jul 18 '12 at 10:47
0

I'm not a statistic buff but maybe RapidMiner can help you. Alternative you can take a look a the list of statistical packages

0

You can plot millions of points in R and Python. They both have thriving and engaging communities.

However that brings around another question. Should you? I sometimes draw huge datasets, but having a large number of data points is usually as informative as sampling down. If you want to keep certain extremes, you could weigh them up so that when sampling randomly, they come up more often. Or even take them aside and pluck them into your final sub-sample. Also, having less points to draw will make production time faster.

Not the answer you're looking for? Browse other questions tagged or ask your own question.