Introduction

The MSR data server is one means for a client to interact and communicate with a process encapsulated by the pdserv library, possibly running in real time.

The client attaches to the pdserv server via a TCP network socket, by default on port 2345. Communication is bi-directional.

The communication syntax between client and server follows an XML-like format.

Upon connection, the server sends an initial greeting and waits for commands from the client.

Command stream from client to server

Commands are sent by the client to the server in a single XML element. The XML-like command format in BNF is:

command := '<' tag attribute* '/>'
tag := name
attribute := ' ' name '="' value '"'
name := any character except the obvious (space, angled brackets, qoutes)

Example of setting a parameter value:

<write_parameter index="55" value="123457"/>

Note: The command parser is very relaxed (attributes can have single ('), double (") or even no quotes if the value does not contain spaces, the closing can be '>' or '/>', boolean attributes can use the attribute name stand alone without '="1"'), but do not rely on this feature! This is only useful when a human interacts on the network socket directly. Programs must always use the strict definition as shown above. Unrecognized commands return a warning: @iverbatim <silly/> <warn num="1000" text="unknown command" command="silly"/> @endiverbatim Unrecognized attributes are silently ignored. There are state changing and query commands, and they may or may not generate a reply. Commands are processed in order of reception. It is not required to synchronize commands to replies, although the client will do that for its own purpose. To determine whether a command was processed by the server, the id attribute is used within the command element. The reply (if one is generated) will have an id attribute with the same value. Additionally an <tt>\<ack\></tt> element with the same id attribute value is sent immediately after command is processed. For example, here are two commands sent in one line: @icode <rk index="99" id="rk99"/><rk index="3" id="rk3"/> <ack id="rk99"/> <channel index="3" name="/osc/amplitude/0/0" datasize="8" typ="TDBL" bufsize="1000" task="0" HZ="100" time="1565346298.661905" value="1.358077306222715e-312" id="rk3"/> <ack id="rk3"/> @endicode The first line contains two @ref rk "`<rk>`" (query a signal) commands with an id attribute. The first is unsuccessful and produces no reply, however the <tt>\<ack id="rk99"/\></tt> is replied before successfully processing the second query. Note the id attribute in <tt>\<channel\></tt>. This mechanism can be used to synchronize communication between client and server, especially when the commands do not produce a reply. @subsection reply-stream Reply stream from server to client The reply stream is well formed XML that can be processed with a streaming XML parser like SAX or expat. A message is a single XML element that may contain child elements. \note Although the server sends a greeting in the form of a @ref connected "`<connected>`" element, it is a complete element with a closing and is thus a finished document! Normal parsers will stop there and need to be reset for parsing to continue. Thus parsers must first be primed with any well formed opening element (like <tt>\<msr\></tt>) which is never closed. Content generated by the server is ASCII which is valid UTF-8, although variable path names are encoded in native program format. To avoid problems, use UTF-8 throughout. The communication from server to client is interleaved with solicited and unsolicited messages: - Solicited messages are replies generated by the server due to a command. An example is a parameter value poll to which the server replies. - Unsolicted messages are sent ad-hoc by the server to the client as required. Examples are: - The client configured the server to monitor a signal or parameter for value change. The value is sent as soon as it changes without the client explicitely requesting it. - Error, warning, info, etc events generated by the process are sent when they occur. - All clients are immediately notified of a parameter value changes. The client must carefully separate the return stream to distinguish between these message types. The problem with unsolicited messages is that they occur at any point in time without any cause from the client. Thus the client has to monitor this stream continuously or run the risk of its input stream overflowing and thus being cut off by the server. For most clients (GUI's, etc) this is no problem, but for clients that want to excercise synchronous single threaded communication could choke. To alleviate this problem, unsolicited messages can be disabled using the @ref remote_host "`<remote_host>`" command. @subsection data-types Data types When querying the server for its variables (@ref rk "signals" or @ref rk "parameters"), its data type and binary size (sizeof operator) is returned in the typ and datasize attributes respectively. An example of quering a parameter: @icode <rp index="4"/> <parameter index="4" name="/osc/amplitude/Limit" datasize="8" typ="TDBL" flags="7" mtime="0.000000" value="20"/> @endicode The following table maps these types to the corresponding C data type. <table class="markdownTable"> <tr class="markdownTableHead"> <th class="markdownTableHeadNone"> typ

datasize

c-type

TUCHAR

1

uint8_t

TCHAR

1

int8_t

TUSHORT

2

uint16_t

TSHORT

2

int16_t

TUINT

4

uint32_t

TINT

4

int32_t

TULINT

8

uint64_t

TLINT

8

int64_t

TFLT

4

float

TDBL

8

double

COMPOUND

n

struct {}

Dimension

The dimensionality of a variable is reflected in attributes when querying the variables.

Generally a variable can be scalar or non-scalar. Scalar variables contain only one of the data types as shown above (a compound can also be a scalar). If a variable contains more than one element, it is said to be non-scalar.

For non-scalar variables, the reply has the following additional attributes:

typ has either _VECTOR or _MATRIX is appended to the data type
orientation contains either
- VECTOR
- MATRIX_ROW_MAJOR
anz: element count (relevant for matrices and vectors) (German: anzahl)
cnum: column count (only for matrices)
rnum: row count (only for matrices)

The data values are ordered in a C-like fasion. Adjacent values are the last dimension of a multi-dimensional array: for the

double array[4][5];

with 4 rows and 5 columns, the 5's will be adjacent (row major).

An example of a scalar:

<channel index="32" name="/SawTooth" datasize="1" typ="TUCHAR"

bufsize="100" task="0" HZ="10"/>

An example of a vector:

<parameter index="5" name="/Event/State" datasize="4" typ="TUINT_LIST"
    anz="5" cnum="5" rnum="1" orientation="VECTOR" dir="1" flags="7"
    mtime="0.000000" value="0,0,0,0,0"/>

An example of a matrix:

<channel index="2" name="/osc/amplitude" datasize="8" typ="TDBL_MATRIX"
    anz="25" cnum="5" rnum="5" orientation="MATRIX_ROW_MAJOR"
    dir="1" bufsize="1000" task="0" HZ="100"/>

Incedentally, the dir flag in the last two examples show that there are more variables listed under its name (see the list command).

Binary value representation

Binary values are transmitted in native machine format. Check the endian flag in the greeting.

MSR uses two methods of binary encoding:

hexdec-format

This format simply walks though the value's entire memory space byte by byte and prints it as a 2-digit hex value.

Example: pi = 182D4454FB210940 on a little endian platform.

This representation is used between client and server interaction.

To query the value of a variable, set the hex boolean attribute in the command and the value is returned in the hexvalue reply attribute.

To set the value of a parameter, use the hexvalue instead of the value** attribute in the command.

Base64 encoding

The value is encoded as base64 string which is more efficient than hexdec (four base64 vs six hexdec characters per three bytes binary data).

Example: pi = GC1EVPshCUA= on a little endian platform.

This representation is used for streaming subscribed signals when the coding attribute is set to Base64

Atomized variables

To support old clients that can only handle scalar variables, the server can atomize non-scalar variables in a path beneath it. This behaviour is set in the configuration file.

Atomized variables can cause an exponential explosion (think of a 10x10 matrix yielding 100 separate variables)

When requesting a directory list, use the attribute noderived to suppress listing of directories containing atomized variables only.

Commands

ping

Ping the server. The server will reply with its current time. This function is used to detect network presence of the server.

Command attributes: none

Reply: <ping/> with attributes:

time (after v3.0): Server reception time

Example:

<ping/>

write_parameter

Alias: wp

Set the value of a parameter. The connection must be authorized, see access in remote_host

Command attributes:

index [uint] (m*): Parameter index (see read_parameter)
name [string] (m*): Parameter path
hexvalue [string] (m**): Hexadecimal representation of the value
value [value list] (m**): Comma separated list of doubles
startindex [uint] (o): Linear starting index when writing a non-scalar parameter. Out of range indices are ignored.
aic [bool] (o): If true, suppress parameter update notification. This is used when parameters need to be written at high frequency.

*) One of index (dominant) or name is mandatory, otherwise nothing happens.

**) One of hexvalue (dominant) or value is mandatory, otherwise nothing happens.

Values are written until either the parameter's end is reached or input value is exhaused.

Reply: none (possibly a parameter update notification follows)

See also: pdserv_parameter()

Example:

<wp index="6" hexvalue="DEADBEEF"/>

read_parameter

Alias: rp

Read a parameter's value value and attributes.

Command attributes:

name [string] (m*): Parameter path
index [uint] (m*): Parameter index (name dominates)
short [bool] (o): Only show index, name, mtime and value/_hexvalue_ in reply. This reduces communication overhead when the parameter is known.
hex [bool] (o): Return value in hexadecimal representation using the hexvalue attribute.

*) If neither name (dominant) or index is specified, a complete parameter listing of all parameters is performed. The listing is enclosed in a <parameters></parameters> element with individual child <parameter/> elements.

Reply: <parameter/> with attributes:

index [uint]: Parameter index
name [string]: Parameter path
comment* [string]: Comment string
datasize* [uint]: see data types
typ* [enum]: see data types
anz* [uint]: see dimension
cnum* [uint]: see dimension
rnum* [uint]: see dimension
orientation* [enum]: see dimension
dir* [bool]: see atomized variables
flags* [uint]: Bit coded value
- 0x01: readable
- 0x02: writeable
- 0x100: Parameter is part of another parameter (atomized)
persistent* [bool]: Parameter is persistent
mtime (time): Modification time. A modifiation time of 0.0 indicates that the parameter is unmodified.
value** [value list]: List of (comma separated) values
hexvalue** [binary]: Hexadecimal value representation

*) These attributes are not transmitted when short attribute is set.

**) hexvalue is returned when hex attribute is set.

See also: pdserv_parameter()

Example:

<rp index="5"/>
<parameter index="5" name="/Event/State" datasize="4" typ="TUINT_LIST"
    anz="5" cnum="5" rnum="1" orientation="VECTOR" dir="1" flags="7"
    mtime="1565276030.213719" value="56746,1,0,0,0"/>

xsad

Subscribe a signals to the data stream sent by the server.

Command attributes:

coding [enum] (o): If set to Base64, value is base64 coded
event [bool] (o): Transmit value only when it changes
sync [bool] (o): If true, data transmission is synchronized by resetting all decimation counters.
channels [uint list] (o): List of channels to transmit
reduction [uint] (o): Decimation counter; Value is sent every n'th count
blocksize [uint] (o): Number of values to group before transmission
precision [uint] (o): Number of decimal points to use (when not using base64)
group [uint] (o): Data group to send channel in. A data group can only contain a channel once, although a channel can exist in more than one data group. This is used to group similar task/decimation rates together.

Note: When channels is not specified, only sync takes affect. If a channel exists in a group, it is overwritten with the new attribute values.

Reply: none (but see the unsolicited `<data>` message)

Example:

<xsad channels="2,4" group="3" reduction="5" blocksize="10"/>

xsod

Removes signal subscription from a data transmission group.

Command attributes:

channels [uint list] (o): List of channels to remove. If this attribute is not specified, all signal subscriptions are cancelled.
group [uint] (o): Group id to remove channels from

Reply: none

Example:

<xsod channels="4" group="3"/>

xsap

Version: after v3.0

Monitor parameters, transmitting the value on change.

Command attributes:

hex [bool] (o): Values are transmitted in hexadecimal represenation
monitor [bool] (o): If true, transmit all parameter changes. This state can only be reset using <xsop monitor="0"/>.
parameters [uint list] (o): List of parameters to monitor

Reply: none (but see the unsolicited `<data>` message)

Example:

<xsap parameters="5,6"/>

xsop

Version: after v3.0

Cancel monitoring of parameter changes.

Command attributes:

monitor [bool] (o): If false, stop monitoring all parameters. This does not clear the list of parameters to monitor!
parameters [uint list] (o): List of parameters to remove from monitor

Example:

<xsop parameters="6"/>

echo

Unused

read_kanaele

Alias: rk rc

Read the value and all attributes of a signal.

Command attributes:

name [string] (m*): Signal path
index [uint] (m*): Signal index (name dominates)
short [bool] (o): Only show index, name, time and value/_hexvalue_ in reply. This reduces communication overhead when the signal is known.
hex [bool] (o): Return value in hexadecimal representation using the hexvalue attribute.

*) If neither name (dominant) or index is specified, a complete signal listing of all signals is performed. The listing is enclosed in a <channels></channels> element with individual child <channel/> elements. Polling signal values is expensive, thus the value is not transmitted when a complete listing is performed,

Reply: <channel/> with attributes:

index [uint]: Parameter index
name [string]: Parameter path
comment* [string]: Comment string
datasize* [uint]: see data types
typ* [enum]: see data types
anz* [uint]: see dimension
cnum* [uint]: see dimension
rnum* [uint]: see dimension
orientation* [enum]: see dimension
dir* [bool]: see atomized variables
bufsize [uint]: Parameter is persistent
task [uint]: Task id signal belongs to
HZ [double]: Signal calculation value (may be slower than task!)
time [time]: Sample time
value** [value list]: List of (comma separated) values
hexvalue** [binary]: Hexadecimal value representation

*) These attributes are not transmitted when short attribute is set.

**) hexvalue is returned when hex attribute is set.

Example:

<rk index="29"/>
<channel index="29" name="/osc/derivative" comment="Derivative of [cos,sin]"
    datasize="8" typ="TDBL_LIST" anz="2" cnum="2" rnum="1" orientation="VECTOR"
    text="Derivative of [cos,sin]" dir="1" bufsize="1000" task="0" HZ="100"
    time="1565280630.398467" value="11.04316657549318,-5.031003686345138"/>

read_param_values

Alias: rpv

Return a list of all parameter values, sequencially sorted by index.

Reply: <param_values/> with attributes:

value [value list]: Semicolon separated list of parameter values.

See also: pdserv_signal()

Example:

<read_param_values/>
<param_values value="1.2;10;1;0;20;56746,1,0,0,0"/>

auth

Version: since v3.0

Feature test: login

Perform SASL authentication dialog.

Command attributes:

clientdata [base64 string]: Data from client
mech [string]: Chosen mechanism
logout [bool]: Close session

Reply: <saslauth/> with attributes:

mechlist [string]: List of supported mechanisms.
serverdata [base64 string]: Server SASL reply.
success [bool]: If supplied, this is the final step with login indicating success or failure.

1) The first step is to obtain a list of supported mechanisms.

<auth/>
<saslauth mechlist="DIGEST-MD5 CRAM-MD5 NTLM PLAIN LOGIN ANONYMOUS"/>

Note: When login is mandatory, the <saslauth mechlist="..."/> element is sent automatically upon connection without prior request.

The list of mechanisms does not change. This first step is required only once upon initial connection by the client.

This step is not required if the mechanism is known apriori.

2) The client chooses a mechanism. Depending on the mechanism, the client library will start the conversation with some data or not. Supplying a mech** attribute initiates the server side of the conversation (client-send-first). Depending on the mechanism, the server may require more data or even signal success or failure.

<auth mech="PLAIN" clientdata="lkjsdlfjkl"/>
<saslauth serverdata="lkjasdflkj" success="1"/>

The mech attribute may only be sent on initiation. Certain mechanisms may require more than one interaction. Subsequent commands will only have clientdata, until the server replies with a success attribute.

list

Version: since v2.0

Feature test: list

Return a list of parameters, signals and directories.

Command attributes:

path [string] (m): Directory path to list
noderived [bool] (o): Omit the dir attribute from <parameter/> and <channel/> elements when only atomized versions of the parent variable exist beneath it.

Reply: <listing></listing> element with children:

<dir/>: Attribute path of a directory
<channel/>: See read_kanaele
<parameter/>: See read_parameter

Example:

<list path="/"/>
<listing>
 <dir path="/Event"/>
 <channel index="32" name="/SawTooth" datasize="1" typ="TUCHAR"
     bufsize="100" task="0" HZ="10"/>
 <dir path="/Taskinfo"/>
 <dir path="/osc"/>
</listing>

starttls

Version: since v3.0

Feature test: tls

Start tls data transmission.

Command attributes: none

Reply: <tls/>\r\n

Upon reception of this command, the server switches over to TLS transmission. However, the client must continue reception in plain text until the reply is received, including \r\n!

The client may switch its command channel to TLS after sending this command, but reception must continue in unencrypted until the reply (including \r\n!) is received before switching reception to TLS as well.

Example:

<starttls/>
<tls>\r\n

remote_host

Version: since v2.0

Feature test: polite

Command to set miscellaneous states.

Command attributes:

name [string] (o): Remote host name, may be used in TLS conversation
applicationname [string]: Name of application logging in
access [bool] (o): Request permission to change parameters. This is another ugly feature of MSR, as the client decides whether it is allowed to write paramters or not.
polite [bool] (o): Do not produce any unsolicited message, such as <pu>, messages, etc. Only commands that produce an immediate reply will send data from server to client.

Example:

<remote_host name="myhostname" applicationname="GUI" access="1"/>

read_statistics

Alias: read_statics (no, this is not a spelling mistake) rs

Get information about other MSR clients connected to the server.

Reply: <clients></clients> element with <client/> child elements with attributes:

name [string]: Text as specified using the name attribute in `<remote_host/>`.
appname [string]: Text as specified using the applicationname attribute in `<remote_host/>`.
countin: Received bytes
countout: Transmitted bytes
connectedtime: Time when client connected

Example:

<read_statistics/>
<clients>
    <client name="lansim (192.168.22.56:52276)"
        apname="Persistent Manager, Version: 0.3.1"
        countin="19908501" countout="27337577"
        connectedtime="1282151176.659208"/>
    <client .../>
</clients>

message_history

Feature test: history

Arguments:

seq (uint) (o): request a message with a specific sequence number

Request the server to send past messages. Without attributes, the latest message including all active messages are sent enclosed in a <message_history></message_history> element in increasing order of sequence numbers.

With seq argument, the specific message with that sequence number is returned if it exists.

Reply:

No attributes: a list of all active messages as well as the latest message. The list is enclosed in <message_history></message_history>
When seq is specified and invalid: Nothing
When seq is specified and valid: The requested message

Notes:

Typically this command is used to discover all past messages. To achieve this, first send

<message_history/>

and process all messages returned, taking note of the sequence numbers in the replies.

Then use

<message_history seq="XX" id="nn"/>

to retrieve individual messages, counting down from the highest sequence number. If a message does not exist, nothing is returned. Therefor the id tag should be used in this context to detect an empty reply.

Note: The sequence number is a uint32_t data type and can wrap (though very unlikely, but possible). Counting down from 0 will yield sequence number 4294967295 (2^32-1).

See also: pdserv_event()

Example of retrieving active messages:

<message_history/>
<message_history>
 <crit_error name="/Limit" index="3" seq="150" prio="0"
     time="1565386296.209379" text="Event message 3"/>
 <crit_error name="/Limit" index="1" seq="4196" prio="0"
     time="1565389793.155808" text="Event message 2"/>
 <reset name="/Limit" index="0" seq="4220" time="1565389796.195809"/>
</message_history>

The last message has sequence number 4220 (/Limit[0]) but it is inactive. Messages /Limit[1] and /Limit[3] still persist.

To regenerate the list, retrieve messages with sequence numbers counting down from the highest sequence number (4220): 4219, 4218... (possibly, but not necessarily skipping 4196):

<message_history seq="4219" id="mh"/>
<crit_error name="/Limit" index="0" seq="4219" prio="0"
    time="1565389795.655814" text="Event message 1" id="mh"/>
<ack id="mh"/>

Example of retrieving a specific message with sequence number 3220:

<message_history seq="3220" id="mh"/>

Message 3220 does not exist any more becuase it was purged by the server (in this example it keeps 1000 messages - see configuration file). The list is now complete as far as possible.

However, even though the history has been exhausted, message /Limit[3] with sequence number 150 is still known to be active.

Unsolicited stream

ack

This element is sent just after the reply to any command with an id attribute is processed. This feature can be used to synchronize communication with the server.

Note: If id is specified in [<starttls/>] command, the <ack> is already encrypted!

Element attributes:

id [string]: Same as id attribute of command
time [time] (after v3.0): System time

Example:

connected

This element is sent as a greeting upon connection to the server. It is used to greet the client with a information about the server, running process and a list of capabilities.

Element attributes:

name (string): Always MSR
host (string): Host name (useful for TLS)
features (enum):
- pushparameters:
- binparameters:
- eventchannels:
- statistics:
- pmtime:
- aic:
- messages:
- polite: (since v2) See remote_host
- list: (since v2) See list
- login: (after v3.0) See auth
- history: (after v3.0) See message_history
- xsap: (after v3.0) See xsap/xsop
- tls: (since v3) See tls
recievebufsize (uint): Size of input buffer. An non-terminated command exceeding this limit will cause network disconnection. (yes, recievebufsize is a spelling mistake!)
app (string): (since v2) Application name
appversion (string): (since v2) Application version
endian (enum): (after v3.0)
- big
- little
login (string): (after v3.0) If mandatory, client must log in

Example:

<connected name="MSR" host="lappi" app="PdServ Test" appversion="1.234"
    version="393226"
   features="pushparameters,binparameters,eventchannels,statistics,pmtime,
             aic,messages,polite,list,login,history,xsap,tls"
   endian="little" recievebufsize="8192"/>

data

A <data></data> element is a container enclosing child elements of values for signals that were subscribed using xsad.

Element attributes:

time [time]: Time when <data> element is transmitted.
group [int]: Value of xsad:group, if not zero
level: unused

Child elements:

<F/> signal data of fixed decimation rate subscription as specified in xsad:reduction.

Attributes:
- c [uint]: Signal index
- d [data]: Array of signal values, grouped together as specified in xsad:blocksize
<E>: Same as <F/>, but for event based subscribtion (xsad:event) instead of stream based.
<time/>

Attributes:
- d [uint64 array]: uint64_t array of size xasd:blocksize of time values when the sample was taken. Time is in nano-seconds since epoch. Coding is base64
 
 This time is more accurate than data:time, since it contains the actual timestamp when a sample was taken and not one calculated backwards from data:time.

Example of event-based subscription:

<xsad channels="29" event="1" group="9" id="1234"/>
<ack id="1234" time="1565613895.117706"/>
<data group="9" level="0" time="1565613895.122557">
 <time d="XWcUZ5ItuhU="/>
 <E c="29" d="-9.844291201868986,-5.137457097023935"/>
</data>

pu

Notification to all clients of a parameter change. Clients interested in its value must then request it.

parameter

A client can subscribe to monitor parameter value changes using xsap. Upon change, the value is sent automatically by the server using a <parameter/> element upon value change.

Example:

<xsap parameters="5" id="pmon"/>
<ack id="pmon" time="1565627421.396449"/>
<parameter index="5" name="/Event/State" datasize="4" typ="TUINT_LIST"
    anz="5" cnum="5" rnum="1" orientation="VECTOR" flags="7"
    mtime="1565627429.852331"
    hexvalue="0100000000000000000000000000000001000000" pm="1"/>
<parameter index="5" name="/Event/State" mtime="1565627438.417570"
    hexvalue="0000000000000000000000000000000001000000" pm="1"/>

Note

This this is very similar to read_parameter reply, except for:

the pm attribute, indicating that this is a monitored parameter
no atomized versions of the parameter is sent
only a hexdec value is sent
parameters that are known to the client (it was sent to the client in a long version already) produce a short reply (see sort attribute in read_parameter.

Event: crit_error, error, warn, info, reset

A message is generated by the process when some event occurs. They have the following categories as tag name:

crit_error
error
warn
info
reset

<reset/> is a special event element signalling that the event has been cancelled.

Attributes are:

name [path]: Path that identifies the event
index [int]: Event element:
- -1: Scalar event
- 0..N-1: element index when event path is a vector
seq [uint32]: Event sequence number, linearly increasing, rolling over at 2^32-1

prio [enum]: Priority

prio	Priority	Tag name
0	Emergency	crit_error
1	Alert	crit_error
2	Critical	crit_error
3	Error	error
4	Warning	warn
5	Notice	info
6	Info	info
7	Debug	info

time [time]: Time when event occurred
text [string]: Descriptive text message

See also: pdserv_event()

Example:

<warn name="/Limit" index="1" seq="7" prio="4" time="1565689720.991359"
    text="Event message 2"/>
<reset name="/Limit" index="1" seq="8" time="1565689724.251307"/>

Table of Contents

Introduction

Command stream from client to server

Dimension

Binary value representation

hexdec-format

Base64 encoding

Atomized variables

Commands

ping

write_parameter

read_parameter

xsad

xsod

xsap

xsop

echo

read_kanaele

read_param_values

auth

list

starttls

remote_host

read_statistics

message_history

Unsolicited stream

ack

connected

data

pu

parameter

Event: crit_error, error, warn, info, reset