Databases

You can operate on a database from the marcel command line. Marcel links commands through pipes carrying tuples. Relational databases, of course, operate on tables -- sets of tuples -- so the integration of marcel pipelines and database access is very clean.

For example, suppose you have a table recording process information over time:

create table process(time int,  -- seconds since the epoch

                     pid int,

                     command varchar,

                     primary key(time, pid))

You can add data by using marcel commands to get process information, piping the data into a sql statement which inserts to this process table:

timer 1 | args [t: \

    ps | \

    map (p: (t, p.pid, p.commandline)) | \

    sql 'insert into process values(%s, %s, %s)]'

  • timer 1: Generate a timestamp every 1 second.

  • args [t: Bind each incoming timestamp to the pipeline parameter t.

  • ps: Generate a list of Process objects.

  • map (p: (t, p.pid, p.commandlnie): For each Process, p, generate a tuple comprising the timestamp t, and from the Process, the pid and commandline.

  • sql ...: Insert a tuple containing timestamp, pid and command line into the process table.

You can also retrieve data and pipe it into marcel commands, e.g.

sql "select time, count(*) from process \

     where command like '%python%' group by time" \

| out '{}: {}'

This runs a query, and then pipes the output to a marcel command to format the result as "time: count".

  • github
  • Twitter
  • email