GEOPM AGENT TUTORIAL
====================

This directory contains a tutorial on extending the behavior of the
GEOPM runtime by creating an Agent.  The Agent is responsible for
reading signals from the platform, adjusting controls on the platform,
deciding how to aggregate data up the tree to parent Controller nodes,
and enhancing the trace and report with additional statistics.
Policies are given to the root of the tree from the resource manager
or user configuration file and passed down the tree to the leaf
Agents, where they are enacted through platform controls.  Platform
signals are collected at the leaves and used to generate samples to be
passed up the root.  It is important to note that there may not be a
direct mapping from a policy to a specific control, nor from a
specific platform signal to a sample.

This example uses the signals and controls provided by ExampleIOGroup
from the previous tutorial found in $GEOPM_ROOT/tutorial/iogroup.  The
ExampleAgent calculates the system utilization from the user, system,
and idle time signals and prints the result to standard output or
standard error.  The policies in this example are the bounds on the
idle percentage that determine whether standard out or standard error
is used.


0. Agent Interface
------------------

Agents extend the Agent base class found in src/Agent.hpp.  The pure
virtual methods in this interface must be implemented by every Agent;
however, some methods may have empty or degenerate implementations.
Some methods of the Agent interact with the platform directly when the
Agent is acting at the leaf of the tree (level 0).  Other methods are
only used at higher levels to send data up and down the tree.

In addition to the interface methods, it may be useful to implement
several static methods for use during plugin registration: plugin_name(),
make_plugin(), policy_names(), and sample_names().


1. Initialization
-----------------

init():

  Pushes all signals and controls that the Agent will use onto
  PlatformIO.  This instructs PlatformIO and the underlying IOGroups
  to read the specified signals and write the specified controls as a
  batch.  The Agent will not call read_batch()/write_batch() itself,
  and will only interact with these signals and controls through
  sample() and adjust().

  The inputs to this method describe the location of this Agent in the
  tree; although this example does not need this information, other
  Agents might have different behavior depending on their role in the
  tree.

  Care must be taken to ensure all IOGroup plugins that the agent depends on
  are loaded prior to calling init().  See Section 3 - "Set up registration on
  plugin load" in the IOGroup tutorial [README](../iogroup/README.md) for more
  information.

2. Policies
-----------

Policies are sent from the root of the tree down to the leaves.  The
Agents at each level in between distribute their policies to children
at each level. Finally, the policies are used to decide how to adjust
platform controls at the leaf level.  The policies for the
ExampleAgent are the high and low thresholds for idle percent that will
trigger messages to be printed to the screen.  Note that NAN is a valid
setting for a policy and indicates that the Agent should use a
reasonable default value for that policy.

split_policy():

  Copies the policy from the parent into the vector for each child.
  A more sophisticated Agent could use samples coming up the tree as
  feedback to be able to assign slightly different policies to each
  child.  Note that the output vector is owned by the Controller
  and should not be resized by the Agent.

adjust_platform():

  The ExampleAgent stages a message containing the idle percent to one
  of the controls provided by the ExampleIOGroup.  If the idle percent
  is in bounds then the message is staged for the "STDOUT" control, if
  it is out of bounds the message is staged for the "STDERR" control.

do_write_batch():

  If the current idle percent is invalid (NaN), this method returns
  false to indicate to the runtime that nothing should be printed.
  Otherwise it returns true and the value staged by adjust_platform()
  will be written to standard out or standard error.

3. Samples
----------

Samples are collected at the leaf level and sent up the tree to the
root.  The Agents at each level aggregate the samples from their
children to be sent up to their parent.  Samples can come directly
from platform signals, or they may be generated by the Agent.  The
ExampleAgent generates system idle percent to be sent up
the tree.

sample_platform():

  Collects time spent in each mode using the ExampleIOGroup signals
  USER_TIME, NICE_TIME, SYSTEM_TIME, and IDLE_TIME.  The utilization
  is calculated as the percentage of each time compared to the
  total.  Note that the output vector is owned by the Controller and
  should not be resized by the Agent.

aggregate_sample():

  The vector passed in contains a vector of samples from each child.
  For each sample being passed up to the next level in the tree,
  this method takes the average of the samples across children to
  create a single value.


4. Tracing and reporting
------------------------

Agents implement several methods that allow them to include additional
information in reports and traces.

trace_names():

  Sets the names of the columns to be added to the trace.  It is the
  responsibility of the Agent to make sure the number and order of
  these names match the values returned by trace_values().

trace_values():

  Fills in the output vector with the latest set of values to be
  inserted into the trace.  In this example, both the calculated
  samples that were sent up the tree as well as the measured signal
  values are added to the trace.

report_header():

  Adds key-value pairs of Agent-specific details or statistics to the
  top of the report.  In this example, the Agent adds the wait time
  used for the control loop.

report_host():

  Adds key-value pairs of details for a single compute node.  In this
  example, the Agent adds the highest and lowest idle % seen on the node.

report_region():

  The ExampleAgent does not do any per-region analysis, so no region
  details are added to the report.  Agents that do track region
  information can use this method to extend the region sections of
  the report.


5. Control loop cadence
-----------------------

The Agent is responsible for deciding the cadence of the main loop
of the Controller that walks up and down the tree and interacts with the
platform.  It controls the length of time the Controller will wait
after each cycle by implementing the wait() method.  In this example,
wait() uses the time of the previous call to make sure the Controller
cycles are close to 1 second, even if the work from one cycle takes
slightly more or less time than the previous one.

The cadence of tree communication can also be gated using the
do_send_policy() and do_send_sample() methods.  The do_send_policy()
method will determine whether policies at the current level are sent
down the tree in a given iteration of the control loop.  The
do_send_sample() method will likewise determine whether samples are
sent up the tree.  In this example, policies and samples are always
propagated on every iteration, so both methods return true.

6. Set up registration on plugin load
-------------------------------------

In order to be available as the Agent type in the Controller, an Agent
must be registered with the Agent factory.  The factory is a singleton
and can be accessed with the agent_factory() method.  The
register_plugin() method of the factory takes three arguments: the
name of the plugin ("example"), a pointer to a function that returns a
unique_ptr to a new object of the plugin type, and a dictionary
containing the policy names and sample used in the Agent's interaction
with the resource manager.  In this example,
ExampleAgent::plugin_name() provides the first argument, and
ExampleAgent::make_plugin is used as the second.  The dictionary is
constructed using a helper method from the Agent base class,
make_dictionary(), and uses policy_names() and sample_names() to
provide the values.

ExampleAgent is registered at the time the plugin is loaded by GEOPM
in the example_agent_load() method at the top of the file; the
constructor attribute indicates that this method will run at plugin
load time.  GEOPM will automatically try to load any plugins it finds
in the plugin path (discussed in the man page for geopm(7) under the
description of GEOPM_PLUGIN_PATH).  Do not link any of the GEOPM
libraries into the plugin shared object; this will cause a circular
link dependency.


7. Run with GEOPM
-----------------

Specify the ExampleAgent as the Agent for the Controller by setting
GEOPM_AGENT=example.  The name of the Agent should match the name
returned by plugin_name() used for registration.  This Agent requires
values for two policies listed in policy_names(); these are provided
in example_policy.json.  Pass this configuration file to launcher with
the --geopm-policy option or by setting GEOPM_POLICY=example_config.json.
Note that to be recognized as an agent plugin, the shared library
filename must begin with "libgeopmagent_" and end in
".so.1.0.0".  Be sure that both the ExampleIO
plugin and the ExampleAgent plugin are in GEOPM_PLUGIN_PATH.

An example run script is provided in agent_tutorial.sh.  It uses the
geopmbench application.  Before running, build and install GEOPM and
make sure that both plugins are built using the "tutorial_build_*.sh"
script in each folder.  During the run, the idle percent will be
printed to either standard out or standard error, depending on whether
it falls within the bounds set in example_policy.json.  The run script
redirects standard out/error to separate files.

After the run completes, inspect the report and trace files generated.
The report should show that the agent used was "example" and you should see
the agent-specific details in the header and node sections of the report.
The trace should have additional columns for the utilization percents and
the signals read by the agent as returned by trace_values().
