Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

In my trading software I generate a file like the following:

10:00:00.0000001    1.25333
...........................
...........................
14:58:48.5024961    1.23456
14:58:48.5024987    1.23455
14:58:48.5025003    1.23460
14:58:48.5025108    1.23458
...........................
...........................
17:59:59.9997788    1.21111

I have millions of items in total. I want to see graphic, especially "extremums" (time is "X" and value is "Y"). I need ability to "zoom" to interesting parts of the graphic etc.

Probably I can do that in Excel or I should better use some specific software. What can you recommend?

As I generate the file myself, I can change format as needed (use ";" as a delimeter, change time format etc.)

Update: How do I run octave in Windows?

Oleg@OlegPC ~
$ octave
GNU Octave, version 3.6.1
Copyright (C) 2012 John W. Eaton and others.
This is free software; see the source code for copying conditions.
There is ABSOLUTELY NO WARRANTY; not even for MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE.  For details, type `warranty'.

Octave was configured for "i686-pc-cygwin".

Additional information about Octave is available at http://www.octave.org.

Please contribute if you find this software useful.
For more information, visit http://www.octave.org/help-wanted.html

Read http://www.octave.org/bugs.html to learn how to submit bug reports.

For information about changes from previous versions, type `news'.

warning: unable to open X11 DISPLAY
octave:1>
share|improve this question
add comment

3 Answers

up vote 1 down vote accepted

I would use Matlab, but I guess that a little bit to scientific and might not be available to you. Octave seems to be a open source similar to Matlab.

The plot itself would be the easiest plot(x,y) having all abilities to zoom and pan. The data import wizard should also allow you to import cvs type of files if you just change your output a little bit.

x = [1.00 1.10 1.11 1.12 1.12 1.2]

y = [0.709364830858073 0.754686681982361 0.276025076998578 0.679702676853675 0.655098003973841 0.162611735194631]

Having the 1.12 twice:

enter image description here

share|improve this answer
    
will it corrrectly process situation when I have several Y for the same X? it's possible because sometimes Y changes more often than precision of X. –  javapowered Jul 18 '12 at 8:44
    
I don't know what you take as correctly but it can handle multiple Y to the same x. see edit –  eactor Jul 18 '12 at 9:26
    
do you know how can I run octave on windows? I receive unable to open X11 DISPLAY error (updated description) –  javapowered Jul 18 '12 at 10:00
    
for an x11 display you need linux window manager like xming, but i would recommend a virtual ubuntu using virtualbox –  eactor Jul 18 '12 at 10:29
    
i'm using cygwin, how can I do that using cygwin? –  javapowered Jul 18 '12 at 10:47
show 5 more comments

I'm not a statistic buff but maybe RapidMiner can help you. Alternative you can take a look a the list of statistical packages

share|improve this answer
add comment

You can plot millions of points in R and Python. They both have thriving and engaging communities.

However that brings around another question. Should you? I sometimes draw huge datasets, but having a large number of data points is usually as informative as sampling down. If you want to keep certain extremes, you could weigh them up so that when sampling randomly, they come up more often. Or even take them aside and pluck them into your final sub-sample. Also, having less points to draw will make production time faster.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.