This lesson is still being designed and assembled (Pre-Alpha version)

An introduction to NEMO

Introducing NEMO

Overview

Teaching: 5 min
Exercises: 0 min
Questions

  • What is NEMO and how would I use it

Objectives

  • Introducing NEMO

Background

As soon as computers became available to the research community in the 50s, astronomers tackled the problem how to compute the orbits of stars and planets under their common force of gravity: the N-body problem.

The first programs were written in FORTRAN (von Hoerner, Aarseth, van Albada and many others), and we have some of those available for you to experiment with in NEMO!

NEMO

NEMO is a software environment to enable you to study Stellar Dynamics in general and the N-body problem in particular. It has several hundred programs to initialize stellar N-body systems, integrate them in time, study their orbits, convert them to images, and compare them to observations etc.etc.

You will probably be writing scripts to orchestrate a simulation and perform an analysis. We have examples in bash, csh, and python.

NEMO has excellent programs to import data from other codes, as well as export data such that other codes can process our data. We also employ tables in text or CSV format, often an end-point before a plot is made. NEMO also has a good infrastructure to write new programs.

Although NEMO is not an acronym (its etymology is from an attempt to pun nbody to nobody, which became NEMO in latin), a good friend could not help himself and dub it Not Everyone Must Observe, in the vain of this being the first theory package next to a few observers packages back in 1986.

Key Points

  • Stellar Dynamics

  • the N-body problem

  • NEMO

Loading NEMO in your Unix Shell

Overview

Teaching: 5 min
Exercises: 5 min
Questions

  • How do I use NEMO in a Unix Shell

Objectives

  • Loading NEMO interactively in your shell

  • Ensure NEMO is present each time you start a new shell

In this lesson we will load NEMO into your shell environment.

Loading NEMO

It is assumed somebody has already installed NEMO. The only thing we need to know is the root directory where NEMO has been installed. So let us assume somebody told us that NEMO has been installed in /opt/nemo_4.1.1 The following command would then load NEMO in your bash shell

$ source /opt/nemo_4.1.1/nemo_start.sh

so really all you need is to find the location of the nemo_start.sh file. There is a similar file for the csh shell, for the die-hards.

Obviously you should test if NEMO commands are now available. The tsf (Type Structured File) program is the de-facto test for this.

$ tsf

Insufficient parameters, try 'help=', 'help=?' or 'help=h' or 'man tsf',
Formatted man page might also be here: https://teuben.github.io/nemo/man_html/tsf.1.html
Usage: tsf in=??? ...
type contents of a (binary) structured file

Making it persistent

Each time your (bash) shell is loaded, it will read some startup files, in which you can add the before mentioned source command. This way each time you start a new shell, NEMO will be ready. For bash this should be ~/.bashrc for interactive shells (interactive and login shells can give for confusing startup procedures).

There is a potential danger to this method. Side-effects for other programs. Probably should say more about that.

Key Points

  • Getting started with NEMO

Installing NEMO

Overview

Teaching: 5 min
Exercises: 10 min
Questions

  • How do I install NEMO (can be skipped)

Objectives

  • Installing NEMO

Installation

Obviously this episode can be skipped if somebody has already pre-installed NEMO for you.

Installing from github

The simplest will be

$ git clone https://github.com/teuben/nemo
$ cd nemo
$ ./configure

with

lots of output. This will take less than a minute.

followed by the unusual

$ make build

with

lots of output. This will take a few minutes.

Loading NEMO in your Shell

NEMO is now ready to be loaded in the shell, as we discussed in the previous episode:

$ source nemo_start.sh

Checking NEMO

A regression test is useful to see if some important tools are working correctly, not only if they work, but also if they give the answer we expect them to give.

$ make check

TESTSUITE: OK   /home/teuben/NEMO/nemo/src/image/fits
TESTSUITE: OK   /home/teuben/NEMO/nemo/src/image/io
TESTSUITE: OK   /home/teuben/NEMO/nemo/src/image/misc
TESTSUITE: OK   /home/teuben/NEMO/nemo/src/image/rotcur
TESTSUITE: OK   /home/teuben/NEMO/nemo/src/image/trans
TESTSUITE: OK   /home/teuben/NEMO/nemo/src/image/wcs
TESTSUITE: OK   /home/teuben/NEMO/nemo/src/kernel/fortran
TESTSUITE: OK   /home/teuben/NEMO/nemo/src/kernel/io
TESTSUITE: OK   /home/teuben/NEMO/nemo/src/kernel/misc
TESTSUITE: OK   /home/teuben/NEMO/nemo/src/kernel/tab
TESTSUITE: OK   /home/teuben/NEMO/nemo/src/nbody/cores
TESTSUITE: OK   /home/teuben/NEMO/nemo/src/nbody/evolve/aarseth/nbody0
TESTSUITE: OK   /home/teuben/NEMO/nemo/src/nbody/evolve/aarseth/tools
TESTSUITE: OK   /home/teuben/NEMO/nemo/src/nbody/evolve/dehnen
TESTSUITE: OK   /home/teuben/NEMO/nemo/src/nbody/evolve/directcode
TESTSUITE: OK   /home/teuben/NEMO/nemo/src/nbody/evolve/flowcode
TESTSUITE: OK   /home/teuben/NEMO/nemo/src/nbody/evolve/hackcode/hackcode1
TESTSUITE: OK   /home/teuben/NEMO/nemo/src/nbody/evolve/hackcode/treecode1
TESTSUITE: OK   /home/teuben/NEMO/nemo/src/nbody/evolve/scfm
TESTSUITE: OK   /home/teuben/NEMO/nemo/src/nbody/evolve/sellwood/tools
TESTSUITE: OK   /home/teuben/NEMO/nemo/src/nbody/image
TESTSUITE: OK   /home/teuben/NEMO/nemo/src/nbody/init
TESTSUITE: OK   /home/teuben/NEMO/nemo/src/nbody/io
TESTSUITE: OK   /home/teuben/NEMO/nemo/src/nbody/io/starlab
TESTSUITE: OK   /home/teuben/NEMO/nemo/src/nbody/io_nemo
TESTSUITE: FAIL /home/teuben/NEMO/nemo/src/nbody/reduc
TESTSUITE: OK   /home/teuben/NEMO/nemo/src/nbody/trans
TESTSUITE: OK   /home/teuben/NEMO/nemo/src/nbody/xyz
TESTSUITE: OK   /home/teuben/NEMO/nemo/src/orbit/misc
TESTSUITE: OK   /home/teuben/NEMO/nemo/src/orbit/potential
TESTSUITE: OK   /home/teuben/NEMO/nemo/usr/wolfire
Results in /home/teuben/NEMO/nemo/tmp/test23706/testsuite.log
A copy in /home/teuben/NEMO/nemo/testsuite.log is kept as well as the latest.
BSF regression results that failed:
BSF: FAIL   0 0 0 0 0 bench.dat
BSF: EXPECT 0.00196205 0.391904 -1.22753 2 10469 bench.dat
BSF regressions that are not OK: 1

in this example there was a failure.. The file testsuite.log will need to be consulted to see what the failure was.

Benchmarking

How fast is your computer? To get an idea how fast your computing platform is, we have deviced a NEMOBENCH5 benchmark which is an indication of the “speed” of NEMO

$ make bench5

with

CPU_USAGE  directcode  :  4.66  4.65  0.00  0.00  0.00  1718721840
CPU_USAGE  gyrfalcON   :  4.91  4.91  0.00  0.00  0.00  1718722307
CPU_USAGE  hackcode1   :  4.93  4.92  0.00  0.00  0.00  1718722799
CPU_USAGE  orbint      :  4.74  4.74  0.00  0.00  0.00  1718723292
CPU_USAGE  potcode     :  4.67  4.65  0.01  0.00  0.00  1718723767
CPU_USAGE  treecode1   :  4.61  4.61  0.00  0.00  0.00  1718724234
NEMOBENCH5 score:   1051.89

On this particular machine a score of 1052 was obtained. Larger values are better. This NEMOBENCH5 was modeled after the idea of geekbench5, in our case each of the codes was tuned such that it would take 5sec CPU on a i5-1135G7 intel CPU, to which the NEMOBENCH5 value was normalized to 1000.

Updating NEMO

Once NEMO has been installed, and assuming this has been done with git, there are two useful things to know about applying an update.

1. A single NEMO program (tsf)

In the case you have been told a single program was updated, you can use the mknemo command to update the source code and compile. This command can be given from any directory, no need to change directory somewhere in NEMO. Viz.

$ mknemo -u tsf

2. All of NEMO

In case there are many updates, it is best to update NEMO internally. For most situations this will be:

$ cd $NEMO
$ git pull
$ make rebuild

as before, this rebuild will take a few minutes.

Key Points

  • Installing NEMO

  • Updating NEMO

NEMO's command line interface (CLI)

Overview

Teaching: 5 min
Exercises: 5 min
Questions

  • How do I use NEMO from the command line

Objectives

  • Learning NEMO command line interface

Any NEMO command can be run from the command line. Here is the tsf (“type structured file”) program, which would display the contents of NEMO’s binary files in some human readable way. So lets try issuing the command

$ tsf 

Insufficient parameters, try 'help=', 'help=?' or 'help=h' or 'man tsf',
Formatted man page might also be here: https://teuben.github.io/nemo/man_html/tsf.1.html
Usage: tsf in=??? ...
type contents of a (binary) structured file

so obviously it needs some arguments. And there some reminders on how and where to get more help:

$ tsf --help

give

tsf in=??? maxprec=false maxline=4 allline=false indent=2 margin=72 item= xml=f octal=f VERSION=3.3c

Keyword Order

The order of the keywords can be important, as the keyword= portion can be left off if given in this order. However, from the moment a keyword is named, they all need to be named. python has a similar philosphy. An example of this:

$ tsf foo.dat true 10 margin=80 item=Image

Getting More Help

More help can also be obtained via the help= option. Try for yourself the help=h and help=M options for the tsf program.

The help=? gives help what help options exist. If your shell gets confused by the question mark, try help=\?

Solution

The help=h will give you a line by line summary of the keyword, some one-line help and the default in square brackets.

         tsf help=h

         margin           : righthand margin  [72]
         in               : input file name  [???]
         maxprec          : print nums with max precision  [false]
         maxline          : max lines per item  [4]
         allline          : print all lines (overrides maxline)  [false]
         indent           : indentation of compound items  [2]
         margin           : righthand margin  [72]
         item             : Select specific item []
         xml              : output data in XML format? (experimental) [f]
         octal            : Force integer output in octal again? [f]
         VERSION          : 3-feb-2014 PJT  [3.3c]

The help=M will give you a classic unix man page

         tsf help=h


        TSF(1NEMO)

        NAME
             tsf - type a structured file

        SYNOPSIS
             tsf in=file [parameter=value] ...

        DESCRIPTION
             tsf types  the contents of a structured binary file . . .
        . . .

Such a Unix manual page - short of reading the source code - will have the most information, including examples, and sometimes a TLDR section.

Source Code?

Ultimately the best documentation can be the source code. In the remainder of this episode you will discover how to easily find and look at the source code

mknemo

The mknemo command will help in updating NEMO when new code is available

      mknemo tsf

will not only compile a new version of the tsf program, but also show where the source code lives. Have a look at the source code with your favorite Unix command or editor.

Solution

      MKNEMO> Searching tsf.c: 
      found one: /home/teuben/NEMO/nemo/src/kernel/io/tsf.c
      gcc -g -O2 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE  -fpic -rdynamic  -Dlinux -DSYSV  -I/home/teuben/NEMO/nemo/inc -I/home/teuben/NEMO/nemo/lib -I/home/teuben/NEMO/nemo/opt/include -I/usr/include/hdf   -Wall -Wimplicit-function-declaration  -D_GNU_SOURCE -std=c99  -o tsf tsf.c -L/home/teuben/NEMO/nemo/lib -L/home/teuben/NEMO/nemo/opt/lib          -lnemo -ldl  -lreadline -lhistory -lncurses `pkg-config --libs cfitsio` -lm   -L/usr/lib/gcc/x86_64-linux-gnu/11 -L/usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/11/../../../../lib -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/11/../../.. -lgfortran -lm -lquadmath

On the 2nd line you will see the filename, so in my case I can do

       more /home/teuben/NEMO/nemo/src/kernel/io/tsf.c

Key Points

  • Run a NEMO command

  • Get help from a NEMO command

Files and Pipes

Overview

Teaching: 5 min
Exercises: 5 min
Questions

  • What kind of files does NEMO use

Objectives

  • File Types

  • Unix Pipes

NEMO stores most of its data in binary format. We have already see that the tsf program can display such data. These files are hierarchical, named, and type tagged to promote interoperability. Later in this episode we will see there are a few types (e.g. SnapShot, Orbit, Image). Here is a SnapShot:

$ mkplummer out=p10.dat nbody=10 seed=123
$ tsf in=p10.dat

but since the first two keywords are in order, it’s much easier to skip the keyword= portion for those. The seed= keyword is much further down the list, so we explicitly pass it with the keyword name:

$ mkplummer p10.dat 10 seed=123
$ tsf p10.dat

char Headline[28] "init_xrandom: seed used 123"
char History[43] "mkplummer p10.dat 10 seed=123 VERSION=3.0b"
set SnapShot
  set Parameters
    int Nobj 10 
    double Time 0.00000 
  tes
  set Particles
    int CoordSystem 66306 
    double Mass[10] 0.100000 0.100000 0.100000 0.100000 0.100000 
      0.100000 0.100000 0.100000 0.100000 0.100000 
    double PhaseSpace[10][2][3] -0.609486 -0.221687 -0.450963 
      -0.0766784 -0.209397 0.396561 4.80925 -0.486182 -1.52575 
      0.418474 0.117766 0.230193 0.545314 0.0795464 0.0632245 
      0.0867019 -0.109128 -0.282036 0.377209 -0.213318 -0.0118330 
      . . .
  tes
tes

but we can also use this in a Unix pipe

$ mkplummer - 10 seed=123 | tsf -

and it should give exactly the same output, since the same seed=123 was used.

Example Data Files

Here we give some examples of some file types we have or can generate in NEMO. We invite you to explore them.

Binary Tables

There are some binary tables to store a distribution function for initializing N-body realizations of certain model galaxies.

$ tsf $NEMODAT/k5isot.dat

Snapshot

We have already seen this one. They are used to store a snapshot of an N-body simulation.

$ mkplummer - 10 seed=123 | tsf -

Image

$ ccdgen - noise 1,0.1 | tsf -

Orbit

$ mkorbit - x=1 vy=1 potname=plummer | tsf -

this will just be a single point (initial conditions), but in general an Orbit will store many samples in time, much like a series of snapshots for an N-body simulation. This example also shows the use of a potential descriptor with the potname= keyword.

Key Points

  • File Types

  • Unix pipes