Databases

​

You can operate on a database from the marcel command line. Marcel links commands through pipes carrying tuples. Relational databases, of course, operate on tables -- sets of tuples -- so the integration of marcel pipelines and database access is very clean.

​

For example, suppose you have a table recording process information over time:

​

create table process(time int,  -- seconds since the epoch

                     pid int,

                     command varchar,

                     primary key(time, pid))

​

You can add data by using marcel commands to get process information, piping the data into a sql statement which inserts to this process table:

​

timer 1 | args (| t: \

    ps | \

    map (p: (t, p.pid, p.cmdline)) | \

    sql 'insert into process values(%s, %s, %s)' |)

​

• timer 1: Generate a timestamp every 1 second.

• args (| t: ... |): Bind each incoming timestamp to the pipeline parameter t.

• ps: Generate a list of Process objects.

• map (p: (t, p.pid, p.commandline): For each Process, p, generate a tuple comprising the timestamp t, and from the Process, the pid and commandline.

• sql ...: Insert a tuple containing timestamp, pid and command line into the process table.

​

You can also retrieve data and pipe it into marcel commands, e.g.

​

sql "select time, count(*) from process \

     where command like '%python%' group by time" \

| out '{}: {}'

​

This runs a query, and then pipes the output to a marcel command to format the result as "time: count".