Further improvements to PStats documentation

Author: rdb

optimization/basic-performance-diagnostics.rst

@@ -6,9 +6,10 @@ Basic Performance Diagnostics
 Introductory Performance Diagnostics
 ------------------------------------
 
-In Panda3D, the "big gun" of performance analysis is called pstats. This program
-gives you real-time diagnostic analysis of a running Panda3D virtual world
-broken down into hundreds of different categories.
+In Panda3D, the "big gun" of performance analysis is called
+:doc:`PStats <pstats/index>`. This program gives you real-time diagnostic
+analysis of a running Panda3D virtual world broken down into hundreds of
+different categories.
 
 But sometimes, when you've just encountered a problem, you don't want that much
 information. Sometimes, you just want a simple question answered, like "how many

optimization/pstats/basic-profiling.rst

@@ -125,6 +125,10 @@ the profiling machine:
 .. code-block:: text
 
    pstats-host profiling-machine-ip-or-hostname
+   pstats-port 5185
+
+By default, the port number used by PStats is 5185. It can be changed by using
+the ``pstats-port`` variable, as shown above.
 
 Session Files
 -------------
@@ -137,17 +141,20 @@ Use the "Save Session" menu item to store the recorded data in a session file.
 At any point, you can launch the PStats server (without a connected client) and
 use "Open Session" to review the recorded data.
 
-If you close the PStats Server by accident without saving the session file to
-disk, you can start PStats and use the "Open Last Session" menu option to
-restore this data.
+The PStats Server also automatically stores the last recorded session to a file
+in the Panda3D cache directory called ``last-session.pstats``. If you close the
+PStats Server by accident without saving the session file to disk, you can start
+PStats and use the "Open Last Session" menu option to restore this data.
 
 Exporting to JSON
 -----------------
 
 To export the timing information to a format that can be read by other
 applications, the "Export to JSON" menu option can be used. The format of this
-file is the Trace Event Format. This can be read by a variety of tools,
-including the Chrome Tracing tool and the online Perfetto application.
+file is the `Trace Event Format <https://docs.google.com/document/d/1CvAClvFfyA5R-PhYUmn5OOQtYMH4h6I0nSsKchNAySU/preview>`__.
+This can be read by a variety of tools, including the
+`Chrome Tracing tool <about:tracing>`__ and the online
+`Perfetto <https://ui.perfetto.dev/>`__ application.
 
 It is also possible to use this feature if no graphical PStats server is
 available. To do this, use the ``text-stats`` utility like so::

optimization/pstats/customization.rst

@@ -17,6 +17,24 @@ This will cause the server to remember the current layout of the graph windows.
 The next time you start a new session and connect a client, all of the
 previously saved graph windows will be reopened.
 
+Guide Bars
+----------
+
+The running Panda client suggests its target frame rate, as well as the initial
+vertical scale of the strip chart (that is, the height of the colored bars).
+You can change the scale freely by clicking within the graph itself and dragging
+the mouse up or down as necessary. One of the horizontal guide bars is drawn in
+a lighter shade of gray; this one represents the actual target frame rate
+suggested by the client. The other, darker, guide bars are drawn automatically
+at harmonic subdivisions of the target frame rate. You can change the target
+frame rate with the Config.prc variable pstats-target-frame-rate on the client.
+
+You can also create any number of user-defined guide bars by dragging them into
+the graph from the gray space immediately above or below the graph. These are
+drawn in a dashed blue line. It is sometimes useful to place one of these to
+mark a performance level so it may be compared to future values (or to alternate
+configurations).
+
 Collector Colors
 ----------------
 

optimization/pstats/graph-types.rst

@@ -5,7 +5,8 @@ Graph Types
 
 The PStats server offers a range of different graphs, giving different views of
 the data being sent from the client. The graph windows can be opened from the
-Graphs pull-down menu.
+Graphs pull-down menu, but they can also be opened by right-clicking a
+particular collector in a chart.
 
 .. contents::
    :local:
@@ -25,58 +26,43 @@ represents the total amount of time spent on each frame; within the frame, the
 time is further divided into the primary subdivisions represented by different
 color bands (and labeled on the left). These subdivisions are called
 "collectors" in the PStats terminology, since they represent time collected by
-different tasks.
-
-Normally, the three primary collectors are App, Cull, and Draw, the three stages
-of the graphics pipeline. Atop these three colored collectors is the label
-"Frame", which represents any remaining time spent in the frame that was not
-specifically allocated to one of the three child collectors (normally, there
-should not be significant time reported here).
-
-The frame time in milliseconds, averaged over the past three seconds, is drawn
-above the upper right corner of the graph. The labels on the guide bars on the
-right are also shown in milliseconds; if you prefer to think about a target
-frame rate rather than an elapsed time in milliseconds, you may find it useful
-to select "Hz" from the Units pulldown menu, which changes the time units
-accordingly.
-
-The running Panda client suggests its target frame rate, as well as the initial
-vertical scale of the graph (that is, the height of the colored bars). You can
-change the scale freely by clicking within the graph itself and dragging the
-mouse up or down as necessary. One of the horizontal guide bars is drawn in a
-lighter shade of gray; this one represents the actual target frame rate
-suggested by the client. The other, darker, guide bars are drawn automatically
-at harmonic subdivisions of the target frame rate. You can change the target
-frame rate with the Config.prc variable pstats-target-frame-rate on the client.
-
-You can also create any number of user-defined guide bars by dragging them into
-the graph from the gray space immediately above or below the graph. These are
-drawn in a dashed blue line. It is sometimes useful to place one of these to
-mark a performance level so it may be compared to future values (or to alternate
-configurations).
-
-The primary collectors labeled on the left might themselves be further
-subdivided, if the data is provided by the client. For instance, App is often
-divided into Show Code, Animation, and Collisions, where Show Code is the time
+different tasks. The top-most label indicates the collector that is currently
+being viewed, and the labels below it indicate its subdivisions.
+
+Normally, the primary collector is called "Frame", representing the total amount
+of time spent rendering a particular frame. This is subdivided into App, Cull,
+and Draw, the three stages of the graphics pipeline, a Wait collector for time
+spent waiting on other threads or VSync, and a further \* collector for
+operations that may occur across multiple stages. Any remaining time not
+specifically allocated to one of those child collectors is assigned to the
+parent "Frame" collector (normally, there should not be significant time
+reported here).
+
+All of these categories contain further subdivisions, which themselves may be
+subdivided further, if this data is provided by the client. For instance, App is
+often divided into Tasks, Animation, and Collisions, where Tasks is the time
 spent executing any Python code, Animation is the time used to compute any
 animated characters, and Collisions is the time spent in the collision
-traverser(s).
+traverser(s), etc.
 
 To see any of these further breakdowns, double-click on the corresponding
 colored label (or on the colored band within the graph itself). This narrows the
-focus of the strip chart from the overall frame to just the selected collector,
-which has two advantages. Firstly, it may be easier to observe the behavior of
-one particular collector when it is drawn alone (as opposed to being stacked on
-top of some other color bars), and the time in the upper-right corner will now
-reflect just the total time spent within just this collector. Secondly, if there
-are further breakdowns to this collector, they will now be shown as further
-colored bars. As in the Frame chart, the topmost label is the name of the parent
-collector, and any time shown in this color represents time allocated to the
-parent collector that is not accounted for by any of the child collectors.
-
-You can further drill down by double-clicking on any of the new labels; or
-double-click on the top label, or the white part of the graph, to return back up
-to the previous level. Right-clicking a label will provide further options.
+focus of the strip chart from the overall frame to just the selected collector.
+Not only does it make it easier to observe the behavior of that particular
+collector since it is drawn alone (as opposed to being stacked on top of some
+other color bars), but if there are further breakdowns to this collector, they
+will now be shown as further colored bars. As in the Frame chart, the topmost
+label is the name of the currently focused collector, and any time shown in this
+color represents time allocated to the current collector that is not accounted
+for by any of the child collectors. To return to the parent level, simply
+double-click this top-most collector.
+
+The time spent in the currently focused collector, averaged over the past three
+seconds, is drawn above the upper right corner of the graph. By default, this is
+shown in milliseconds, which is a better metric than a target frame rate, but
+the unit can be changed from the Units pulldown menu if desirable. Some
+collectors will additionally show a number indicating how often they were
+started in the latest frame.
 
 Value-based Strip Charts
 ------------------------
@@ -106,7 +92,7 @@ The way the bars are stacked indicates how the collectors are nested. Let's say
 that Panda3D performs a Cull pass for display region A and B separately. The
 Strip Chart view would just tell you the total Cull time in the frame, which
 doesn't tell you which scene you need to optimize. The Flame Graph view on the
-other hand, will show two separate Cull bars, one stacked above the bar for
+other hand will show two separate Cull bars, one stacked above the bar for
 display region A, and the other stacked above the bar for display region B.
 
 You can double-click on any bar to focus in to that particular collector and
@@ -139,11 +125,16 @@ clock, the GPU and CPU threads may not be perfectly aligned.
 There are several ways to navigate through the timeline. By double-clicking a
 particular bar, the view will zoom to fit that bar. You can also use the WASD
 keys to navigate, or the scroll wheel of the mouse while holding the control key
-on the keyboard.
+on the keyboard. If the timeline takes up so much vertical space that it runs
+off the edge of the chart, you can use the scroll wheel of the mouse *without*
+holding the control key to bring everything into view.
 
 Please note that PStats discards data older than 60 seconds by default. To be
 able to see the entire timeline, you need to change the ``pstats-history``
-configuration variable.
+configuration variable (eg. you could set it to ``inf`` to never discard data).
+Furthermore, it is possible to see dropped frames if the frame rate is too high
+or if the send queue is full. If you wish to see all frames, increase the
+``pstats-max-rate`` and ``pstats-max-queue-size`` variables.
 
 The Piano Roll
 --------------
@@ -164,7 +155,9 @@ reads from left to right.)
 Unlike a strip chart, a piano roll chart does not show trends; the chart shows
 only the current frame's data. The horizontal axis shows time within the frame,
 and the individual collectors are stacked up in an arbitrary ordering along the
-vertical axis.
+vertical axis. It is possible that there are so many collectors that they run
+off the edge of the window; in this case, use the scroll wheel on a mouse to
+scroll through the label stack on the left side.
 
 The time spent within the frame is drawn from left to right; at any given time,
 the collector(s) that are active will be drawn with a horizontal bar. You can

optimization/pstats/internals.rst

@@ -10,8 +10,9 @@ The PStats Client
 -----------------
 
 The client code is in panda/src/pstatclient, and is available to run in every
-Panda client unless it is compiled out. (It will be compiled out if OPTIMIZE is
-set to level 4, unless DO_PSTATS is also explicitly set to non-empty.)
+Panda client unless it is compiled out. (It will be compiled out when building
+for Release in CMake or when passing ``--optimize 4`` to makepanda, unless
+DO_PSTATS is also explicitly set to non-empty.)
 
 The client code is designed for minimal runtime overhead when it is compiled in
 but not enabled (that is, when the client is not in contact with a PStats
@@ -20,39 +21,51 @@ PStats server). It is also designed for zero runtime overhead when it is
 compiled out.
 
 There is one global :class:`.PStatClient` class object, which manages all of the
-communications on the client side. Each PStatCollector is simply an index into
-an array stored within the PStatClient object, although the interface is
-intended to hide this detail from the programmer.
+communications on the client side. Each :class:`.PStatCollector` is simply an
+index into an array stored within the :class:`.PStatClient` object, although the
+interface is intended to hide this detail from the programmer.
 
-Initially, before the PStatClient has established a connection, calls to start()
-and stop() simply return immediately.
+Initially, before the :class:`.PStatClient` has established a connection, calls
+to :meth:`~.PStatCollector.start()` and :meth:`~.PStatCollector.stop()` simply
+return immediately.
 
 When you call :meth:`.PStatClient.connect()`, the client attempts to contact the
-PStatServer via a TCP connection to the hostname and port named in the pstats-
-host and pstats-port Config.prc variables, respectively. (The default hostname
-and port are localhost and 5185.) You can also pass in a specific hostname
-and/or port to the connect() call. Upon successful connection and handshake with
-the server, the PStatClient sends a list of the available collectors, along with
-their names, colors, and hierarchical relationships, on the TCP channel.
-
-Once connected, each call to start() and stop() adds a collector number and
-timestamp to an array maintained by the PStatClient. At the end of each frame,
-the PStatClient boils this array into a datagram for shipping to the server.
-Each start() and stop() event requires 6 bytes; if the resulting datagram will
-fit within a UDP packet (1K bytes, or about 84 start/stop pairs), it is sent
-via UDP; otherwise, it is sent on the TCP channel. (Some fraction of the
-packets that are eligible for UDP, from 0% to 100%, may be sent via TCP
-instead; you can specify this with the pstats-tcp-ratio Config.prc variable.)
+PStatServer via a TCP connection to the hostname and port named in the
+pstats-host and pstats-port Config.prc variables, respectively. (The default
+hostname and port are localhost and 5185.) You can also pass in a specific
+hostname and/or port to the :meth:`~.PStatClient.connect()` call. Upon
+successful connection and handshake with the server, the :class:`.PStatClient`
+sends a list of the available collectors, along with their names, colors, and
+hierarchical relationships, on the TCP channel.
+
+Once connected, each call to :meth:`~.PStatCollector.start()` and
+:meth:`~.PStatCollector.stop()` adds a collector number and timestamp to an
+array maintained by the PStatClient. At the end of each frame, the PStatClient
+boils this array into a datagram for shipping to the server.
+Each :meth:`~.PStatCollector.start()` and :meth:`~.PStatCollector.stop()` event
+requires 6 bytes; if the resulting datagram will fit within a UDP packet (1K
+bytes, or about 84 start/stop pairs), it is sent via UDP; otherwise, it is sent
+on the TCP channel. (Some fraction of the packets that are eligible for UDP,
+from 0% to 100%, may be sent via TCP instead; you can specify this with the
+``pstats-tcp-ratio`` Config.prc variable.)
 
 Also, to prevent flooding the network and/or overwhelming the PStats server,
 only so many frames of data will be sent per second. This parameter is
-controlled by the pstats-max-rate Config.prc variable and is set to 30 by
+controlled by the ``pstats-max-rate`` Config.prc variable and is set to 30 by
 default. (If the packets are larger than 1K, the max transmission rate is also
 automatically reduced further in proportion.) If the frame rate is higher than
 this limit, some frames will simply not be transmitted. The server is designed
 to cope with missing frames and will assume missing frames are similar to their
 neighbors.
 
+Finally, to prevent an excessive backlog building up if there is too much data
+for the transmission to handle, Panda3D will only queue up a certain number of
+frames of data at a time. This is determined by the value of the
+``pstats-max-queue-size`` variable. If the backlog of frames to send is greater
+than this value, subsequent frames are dropped. Note that each thread sends its
+own frame, so you need to make sure this value is at least as large to
+accommodate all the threads sending data at once.
+
 The server does all the work of analyzing the data after that. The client's next
 job is simply to clear its array and prepare itself for the next frame.
 
@@ -64,7 +77,9 @@ server code is in pandatool/src/gtk-stats and pandatool/src/win-stats, for Unix
 and Windows, respectively. (There is also an OS-independent text-stats
 subdirectory, which builds a trivial PStats server that presents a scrolling-
 text interface. This is mainly useful as a proof of technology rather than as a
-usable tool.)
+usable tool, but it does have an option to output the data in JSON format so
+that it can be analyzed by a third-party application, if the PStats server is
+not available for this platform.)
 
 The GUI-specific code is the part that manages the interaction with the user via
 the creation of windows and the handling of mouse input, etc.; most of the real
@@ -74,24 +89,25 @@ directory.
 The PStatServer owns all of the connections, and uses network sockets to
 communicate with the clients. It listens on the specified port for new
 connections, using the pstats-port Config.prc variable to determine the port
-number (this is the same variable that specifies the port to the client).
+number (this is the same variable that specifies the port to the client),
+although this can be overridden by using the ``-p`` option on the command-line.
 Usually you can leave this at its default value of 5185, but there may be some
 cases in which that port is already in use on a particular machine (for
 instance, maybe someone else is running another PStats server on another display
 of the same machine).
 
 Once a connection is received, it creates a PStatMonitor class (this class is
 specialized for each of the different GUI variants) that handles all the data
-for this particular connection. In the case of the windows pstats.exe program,
-each new monitor instance is represented by a new toplevel window. Multiple
-monitors can be active at once.
+for this particular connection. A PStatMonitor is also created when a session
+is loaded from a file.
 
 The work of digesting the data from the client is performed by the PStatView
 class, which analyzes the pattern of start and stop timestamps, along with the
 relationship data of the various collectors, and boils it down into a list of
 the amount of time spent in each collector per frame.
 
-Finally, a PStatStripChart or PStatPianoRoll class object defines the actual
-graph output of colored lines and bars; the generic versions of these include
-virtual functions to do the actual drawing (the GUI specializations of these
-redefine these methods to make the appropriate calls).
+Finally, a PStatStripChart, PStatFlameGraph, PStatTimeline or PStatPianoRoll
+class object defines the actual graph output of colored lines and bars; the
+generic versions of these include virtual functions to do the actual drawing
+(the GUI specializations of these redefine these methods to make the appropriate
+calls).