All the mail mirrored from lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v4] doc: guidelines for library statistics
@ 2015-06-16 13:35 Cristian Dumitrescu
  2015-06-16 13:40 ` Mcnamara, John
  0 siblings, 1 reply; 5+ messages in thread
From: Cristian Dumitrescu @ 2015-06-16 13:35 UTC (permalink / raw)
  To: dev

v4 changes
-more fixes for bullets

v3 changes
-fixed bullets for correct doc generation

v2 changes
-small text changes
-reordered sections to have guidelines at the top and motivation at the end
-broke lines at 80 characters

Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
---
 doc/guides/guidelines/index.rst      |    1 +
 doc/guides/guidelines/statistics.rst |  104 ++++++++++++++++++++++++++++++++++
 2 files changed, 105 insertions(+), 0 deletions(-)
 create mode 100644 doc/guides/guidelines/statistics.rst

diff --git a/doc/guides/guidelines/index.rst b/doc/guides/guidelines/index.rst
index b2b0a92..c01f958 100644
--- a/doc/guides/guidelines/index.rst
+++ b/doc/guides/guidelines/index.rst
@@ -6,3 +6,4 @@ Guidelines
     :numbered:
 
     coding_style
+    statistics
diff --git a/doc/guides/guidelines/statistics.rst b/doc/guides/guidelines/statistics.rst
new file mode 100644
index 0000000..bc91723
--- /dev/null
+++ b/doc/guides/guidelines/statistics.rst
@@ -0,0 +1,104 @@
+Library Statistics
+==================
+
+Description
+-----------
+
+This document describes the guidelines for DPDK library-level statistics counter
+support. This includes guidelines for turning library statistics on and off and
+requirements for preventing ABI changes when implementing statistics.
+
+
+Mechanism to allow the application to turn library statistics on and off
+------------------------------------------------------------------------
+
+Each library that maintains statistics counters should provide a single build
+time flag that decides whether the statistics counter collection is enabled or
+not. This flag should be exposed as a variable within the DPDK configuration
+file. When this flag is set, all the counters supported by current library are
+collected for all the instances of every object type provided by the library.
+When this flag is cleared, none of the counters supported by the current library
+are collected for any instance of any object type provided by the library:
+
+.. code-block:: console
+
+	# DPDK file config/common_linuxapp, config/common_bsdapp, etc.
+	CONFIG_RTE_<LIBRARY_NAME>_STATS_COLLECT=y/n
+
+The default value for this DPDK configuration file variable (either "yes" or
+"no") is decided by each library.
+
+
+Prevention of ABI changes due to library statistics support
+-----------------------------------------------------------
+
+The layout of data structures and prototype of functions that are part of the
+library API should not be affected by whether the collection of statistics
+counters is turned on or off for the current library. In practical terms, this
+means that space should always be allocated in the API data structures for
+statistics counters and the statistics related API functions are always built
+into the code, regardless of whether the statistics counter collection is turned
+on or off for the current library.
+
+When the collection of statistics counters for the current library is turned
+off, the counters retrieved through the statistics related API functions should
+have a default value of zero.
+
+
+Motivation to allow the application to turn library statistics on and off
+-------------------------------------------------------------------------
+
+It is highly recommended that each library provides statistics counters to allow
+an application to monitor the library-level run-time events. Typical counters
+are: number of packets received/dropped/transmitted, number of buffers
+allocated/freed, number of occurrences for specific events, etc.
+
+However, the resources consumed for library-level statistics counter collection
+have to be spent out of the application budget and the counters collected by
+some libraries might not be relevant to the current application. In order to
+avoid any unwanted waste of resources and/or performance impacts, the
+application should decide at build time whether the collection of library-level
+statistics counters should be turned on or off for each library individually.
+
+Library-level statistics counters can be relevant or not for specific
+applications:
+
+* For Application A, counters maintained by Library X are always relevant and
+  the application needs to use them to implement certain features, such as traffic
+  accounting, logging, application-level statistics, etc. In this case,
+  the application requires that collection of statistics counters for Library X is
+  always turned on.
+
+* For Application B, counters maintained by Library X are only useful during the
+  application debug stage and are not relevant once debug phase is over. In this
+  case, the application may decide to turn on the collection of Library X
+  statistics counters during the debug phase and at a later stage turn them off.
+
+* For Application C, counters maintained by Library X are not relevant at all.
+  It might be that the application maintains its own set of statistics counters
+  that monitor a different set of run-time events (e.g. number of connection
+  requests, number of active users, etc). It might also be that the application
+  uses multiple libraries (Library X, Library Y, etc) and it is interested in the
+  statistics counters of Library Y, but not in those of Library X. In this case,
+  the application may decide to turn the collection of statistics counters off for
+  Library X and on for Library Y.
+
+The statistics collection consumes a certain amount of CPU resources (cycles,
+cache bandwidth, memory bandwidth, etc) that depends on:
+
+* Number of libraries used by the current application that have statistics
+  counters collection turned on.
+
+* Number of statistics counters maintained by each library per object type
+  instance (e.g. per port, table, pipeline, thread, etc).
+
+* Number of instances created for each object type supported by each library.
+
+* Complexity of the statistics logic collection for each counter: when only
+  some occurrences of a specific event are valid, additional logic is typically
+  needed to decide whether the current occurrence of the event should be counted
+  or not. For example, in the event of packet reception, when only TCP packets
+  with destination port within a certain range should be recorded, conditional
+  branches are usually required. When processing a burst of packets that have been
+  validated for header integrity, counting the number of bits set in a bitmask
+  might be needed.
-- 
1.7.4.1

^ permalink raw reply related	[flat|nested] 5+ messages in thread
* Re: [PATCH v4] doc: guidelines for library statistics
@ 2015-06-18 14:38 Morten Brørup
  2015-06-18 15:12 ` Thomas Monjalon
  0 siblings, 1 reply; 5+ messages in thread
From: Morten Brørup @ 2015-06-18 14:38 UTC (permalink / raw)
  To: dev

The suggested solution with only one single flag per library prevents implementing statistics with varying granularity for different purposes. E.g. a library could have one set of statistics counters for ordinary SNMP purposes, and another set of statistics counters for debugging/optimization purposes.

 

Multiple flags per library should be possible. A hierarchy of flags per library is probably not required.

 

And since the compile time flag CONFIG_RTE_<LIBRARY_NAME>[_<STATS_COLLECTION_NAME>]_STATS_COLLECT already tells the application if the statistics are present or not, the application should simply use this flag to determine if statistics are accessible or not.

 

 

 

Med venlig hilsen / kind regards

 

Morten Brørup

CTO

 

 

 

SmartShare Systems A/S

Tonsbakken 16-18

DK-2740 Skovlunde

Denmark

 

Office      +45 70 20 00 93

Direct      +45 89 93 50 22

Mobile      +45 25 40 82 12

 

mb@smartsharesystems.com <mailto:mb@smartsharesystems.com> 

www.smartsharesystems.com <http://www.smartsharesystems.com/> 

 

^ permalink raw reply	[flat|nested] 5+ messages in thread
end of thread, other threads:[~2015-06-23 13:44 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-06-16 13:35 [PATCH v4] doc: guidelines for library statistics Cristian Dumitrescu
2015-06-16 13:40 ` Mcnamara, John
2015-06-23 13:43   ` Thomas Monjalon
  -- strict thread matches above, loose matches on Subject: below --
2015-06-18 14:38 Morten Brørup
2015-06-18 15:12 ` Thomas Monjalon

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.