Coding Tools

To set up the coding environment for a new NM Dev project, please follow the following steps.

JDK

NM Dev is a Java based code. Before we can code using the library, we need to install the latest Java Development Kit (JDK). If you skip this step, you can download it together with NetBeans in the next step.

NetBeans

NetBeans is our preferred IDE for Java programming. You may download and install JDK and then NetBeans. Or you can download “NetBeans with JDK” directly.

NetBeans can be downloaded from this link. If you have no Java programming experience, choose the one labeled “Java SE”.

Run the installer.

TortoiseSVN

Download TortoiseSVN.

Run the installer. More information on svn can be found in this wiki.

After installing TortoiseSVN, right click in Explorer in the empty space in the folder you want to put your project in. Click “SVN checkout” to check out project.

The following example checks out AlgoQuant, a NM Dev powered project. You will use the URL given to you instead. In most cases, you do NOT need to check out AlgoQuant as it will be automatically downloaded by Maven when you build the project.

Coding in NetBeans

Launch NetBeans. Open your project. You can right click on a package/folder to create a new Java class to start coding. If you are asked to modify the AlgoQuant code, copy and paste the code in your project and do the editing there. Do NOT modify source code in AlgoQuant directly.

To build your project, right click on the project and hit “Clean and Build”. Alternatively, you can hit this button on the top bar.

To run your project, you need to create JUNIT test cases. We run our project by running the test cases. To create a JUNIT test case file, hit “CTRL+SHIRT+U”. This will create a test file for you where you will put your test cases. You can right click on the test file and run it – either in normal or debug mode.

Debugger

https://netbeans.org/features/java/debugger.html

https://netbeans.org/kb/docs/java/debug-visual.html

Commit Code

When you are done with coding a milestone, you can submit your code by committing code. Right click the project folder. Select “SVN Commit”.

Type in some meaningful messages to let people know what changes and files you make and add. Click “OK” to commit.

STATS

This is a collection of advanced statistics algorithms.

  • cointegration
  • covariance selection
  • factor analysis
  • hidden Markov model (Rabiner)
  • hidden Markov model (mixture models: Beta, Binomial, Exponential, Gamma, Log-Normal, Normal, Poisson)
  • hypothesis testing
  • Kalman filter
  • LASSO (Least Absolute Shrinkage and Selection Operator)
  • linear regression
  • principal component analysis
  • stochastic process simulation
  • time series analysis

FAQs

Maven

  • How do I exclude NM Dev’s dependencies in my own Java project?
    • NM Dev depends on some third-party modules:
      • akka-2.1.4
      • joda-time-2.8.2
      • kryo-2.22
      • heaps-2.0 (from teneighty.org)

      By default, Maven includes these transitive dependencies in the classpath. If you do not want them to be included in the classpath of your project, you may exclude them explicitly by adding an “exclusion” in the dependency section:

      <dependencies>
        <dependency>
          <groupId>com.numericalmethod</groupId>
          <artifactId>suanshu</artifactId>
          <version>${suanshu.version}</version>
          <exclusions>
            <exclusion>
              <groupId>com.esotericsoftware.kryo</groupId> <!-- exclude kryo -->
              <artifactId>kryo</artifactId>
            </exclusion>
          </exclusions>
        </dependency>
      </dependencies>

      See this guide for more information.

Licensing

  • Is there a way to have *one* license file shared by multiple machines?
    • Yes, one license file can contain multiple MAC addresses so you can deploy/copy the same file to multiple machines.
  • Is it possible to have a license that runs on cloud machines?

Coding

  • Why did you choose to have indexes for matrices starting from 1 and not zero as for most structures in java (and c, …)? This complicates the adaptation of code to use NM Dev matrix models and prone to a lot of “+-1” errors in the indexes (for instance when simply iterating over an array *and* a matrix at the same time…)
    • Only the computer scientists count from 0. Everyone else on the planet earth, including the mathematicians, counts from 1. One reason for the computer scientists to count from 0 is indexing to an array element by adding an offset to a starting address. Back in the old C generation, suppose we do, double[] arr = new double[10]; , arr is also a reference to an address of a double or an array of doubles. The address of the 2nd element in the array is computed by adding 8 bytes to arr, i.e., arr + (2 – 1)* 8. In other words, the 2nd object is 1 unit (8 bytes) offset from the starting address. Therefore, an array index in C, C++, Java, C# is the number of offsets from a starting address. It is not the index of the object. NM Dev philosophy is to match symbolically the math equations as much as possible. Most math equations count from 1. All matrices and vectors count from 1. We think that this matching is important for code readability and verification against publications. When we compare our code to the papers, we need not mentally compute this +/-1. Otherwise, the code is very prone to errors, especially when there is a lot of indexing in a complex equation. To avoid doing the +/-1 in coding to some extend, I suggest that the users skip using [0], and count from 1. For example,
      double[] v1 = new double[]{Double.NaN, 1., 2., 3., 4., 5.};
      Vector v2 = new DenseVector(1., 2., 3., 4., 5.);
      Vector v3 = new DenseVector(3);
      for (int i = 1; i < = v1.length; ++i) {
                      v3.set(i, v1[i] + v2.get(i));
      }
      

Misc

 

Groovy

SuanShu Scripting in Groovy

We can make Groovy into one all-powerful math computing environment for numerical analysis.

Groovy is an object-oriented programming language for the Java platform. It is dynamically compiled to Java Virtual Machine bytecode and therefore inter-operates with SuanShu and other Java libraries. Most Java code is also syntactically valid Groovy. Groovy uses a Java-like but simplified syntax. For example, Groovy syntax removes type (dynamic type), semicolons (optional), return keyword (optional). If you already know Java, the learning effort will be minimal. Therefore, Groovy is a popular scripting language for the Java platform.

 

Setup

This section describes how to set up in Groovy to use SuanShu to do numerical computing.

  1. You need a copy of suanshu.jar and a valid SuanShu license. You may download and purchase our products here.
  1. Download the latest stable version of Groovy from the Groovy website, here.
  1. Run the installer. When choosing the destination folder, please use one that contains no space and special characters. We recommend for example “C:\Groovy\Groovy-1.7.5”.

groovy_destination_folder

4.  Configure the environment variables as in the picture below.

 

groovy_environment_variables

5.  Set up the environment variable JAVA_HOME to point to where your JDK is installed. For example, “C:\Program Files (x86)\Java\jdk1.6.0_20”.

groovy_java_home

  1. Create the Groovy ${user.home} folder. The user.home system property is set by your operating system. For example, it can be C:\Users\Haksun Li on Vista, or C:\Document and Settings\Haksun Li on XP. Create the folder “.groovy”. Please note that when you rename on Windows, you need to type “.groovy.” (a “.” at the end) otherwise Windows will complain (a bug?). Then create the “lib” folder in “.groovy”.
  1. Setup SuanShu. Copy and paste the SuanShu jar files, e.g., “suanshu.jar” in .groovy/lib. Copy the SuanShu license file “suanshu.lic” into .groovy.

groovy_folder

 

8.  Modify the Groovy shortcuts to start in the .groovy folder. I created two shortcuts on desktop: one for the Groovy shell (groovysh.exe) and another for the Groovy Console (GroovyConsole). The start-in folder is the current/working directory for Groovy. The SuanShu license management will look for the license file in the current directory.

groovy_startin_folder

9.  Congratulations! You are now set up with Groovy and ready to go. You can start up the Groovy shell by double clicking on groovysh.exe (or the shortcut you created). Type this to say ‘hi’.

 

[box type=”shadow”]

println hi

[/box]

groovy_shell

 

  1. You may modify the properties on the shell screen by clicking on the Groovy icon at the upper left hand corner. I changed mine to white background and black font.

groovy_shell_properties

 

Tutorials

The best way to learn a language is by imitation and practice. We have prepared some examples to get you started on scripting SuanShu in Groovy. You may want to consult the references below to learn more about Groovy.

import

For those converting from the Matlab/Octave/R/Scilab, the hardest thing to get used to is “import”. To use any Java class in Groovy, we can either type the fully qualified name such as

[box type=”shadow”]

com.numericalmethod.suanshu.matrix.doubles.dense.DenseMatrix

[/box]

We need to do this every time we want to create a DenseMatrix. This is very cumbersome.

Alternatively, we can declare the namespace of DenseMatrix by “importing” them once for all.

[box type=”shadow”]

import com.numericalmethod.suanshu.matrix.doubles.dense.DenseMatrix

[/box]

Or, we can do

[box type=”shadow”]

import com.numericalmethod.suanshu.matrix.doubles.dense.*

[/box]

to import all classes in the package import com.numericalmethod.suanshu.matrix.doubles.dense.

Although the keyword “import” saves some typing, it can get cumbersome if you need to import many classes. The Groovy shell partially solves this problem by letting users to put the most often used imports in the shell startup script, “groovysh.rc”. This file is found in the .groovy folder (or you can create one in the folder).

We recommend the SuanShu users to include these imports. You can simply copy and paste to append them in your groovysh.rc.

 

 

References

Maven

We do not include the javadoc files in the zipped package because Maven can automatically download it for you (when you need it) from our repository server. In NetBeans, you can also force it to download the javadoc by clicking the “Dependencies > suanshu-2.2.0 > Download Javadoc”:

netbeans-download-javadoc

Or, right click “Dependencies” and choose “Download Javadoc” for downloading the javadoc of ALL dependencies:

netbeans-download-all-javadoc

A little “J” means the javadoc is available offline. Then, by clicking “View Javadoc”, you can see the downloaded javadoc in your browser offline.

netbeans-view-javadoc

Eclipse also has similar functions:

eclipse-download-javadoc

Time Series Analysis

  • various moving average filters
  • univariate/multivariate time series analysis
    • ARIMA modeling, simulation, fitting, prediction
    • GARCH modeling, simulation, fitting, prediction
    • Auto-Covariance computation

Linear Regression

  • OLS regression
  • logistic regression
  • GLM regression
    • family: Binomial, Gamma, Gaussian, Inverse Gaussian, Poisson
    • quasi-family: Binomial, Gamma, Gaussian, Inverse Gaussian, Poisson
  • GLM model selection

Hypothesis Testing

  • Kolmogorov-Smirnov
  • D’Agostino
  • Jarque-Bera
  • Lilliefors
  • Shapiro-Wilk
  • One-way ANOVA
  • T
  • Kruskal-Wallis
  • Siegel-Tukey
  • Van der Waerden
  • Wilcoxon rank sum
  • Wilcoxon signed rank
  • Breusch-Pagan
  • Glejser
  • Harvey-Godfrey
  • White
  • ADF
  • Bartlett
  • Brown-Forsythe
  • F
  • Levene