Analyze System Health charts to assess trigger performance

In this procedure, you will create a simple trigger that logs the processing time of SMTP responses. You will introduce a deliberate error into the trigger configuration to ensure that a trigger exception occurs.

1. Click the System Settings icon, and then click Triggers.
2. Click Create.
3. In the Name field, type Track Processing Time.
4. Click Enable debug log.
5. Click the Events field, and then add the following events to the trigger configuration:
   • SMTP_REQUEST
   • SMTP_RESPONSE
   • SMPP_RESPONSE
6. Copy and paste the following code into the right pane:

   var proto;
   switch(event) {
       case 'SMTP_REQUEST':
       case 'SMTP_RESPONSE':
           proto = SMTP;
           break;
       case 'SMPP_RESPONSE':
           proto = SMPP;
           break;           
   }      
   
   if (!proto || !proto.processingTime) {
       debug('Processing Time = ' + proto.processingTime + " on " + event);
   }
   

   Copy
7. Click Show Advanced Options, and then select Assign to all devices.
8. Click Save.
   A confirmation message appears that states that the trigger script contains errors. Ignore the message for the purposes of this walkthrough.
9. Click Save Trigger.

The System Health dashboard contains charts that pertain to the health and performance of ExtraHop system components and services. In this procedure, you will consult trigger performance charts on the System Health dashboard to check the impact of the trigger you created in the previous section.

1. In the upper right corner of the window, click the System Settings icon and then select System Health.
2. Scroll down to the Triggers region of the dashboard and locate the Trigger Details chart.
   This chart lists the most active triggers on your system along with the cycles, executes, and exceptions associated with those triggers.
   
   
3. Locate your Track Processing Time trigger and look at the following information:
   1. Compare the values in the Trigger Executes and Trigger Exceptions columns. This information reveals that half of the time the trigger runs, an exception occurs. Because this trigger does not run very often, the impact is not critical. However, if the trigger were modified to run on a popular event such as HTTP, the impact could be extreme.
   2. Compare the value in the Trigger Cycles column with the same value for other triggers running on your system. This information reveals that the average number of cycles consumed by the trigger is relatively low compared to other running triggers. High cycle consumption can indicate that a trigger script is not efficient and might be prone to stall, causing triggers in the queue to back up and drop from the queue.
4. Scroll down the System Health dashboard and locate the Trigger Exceptions by Trigger chart. The chart displays the Track Processing Time trigger you created, similar to the following figure:
   

   This chart displays which triggers have exceptions and the number of exceptions generated in the specified time range. Timestamps can help you locate exception error messages in the debug log.

   Tip:	To highlight a single line in the chart, click the name of the trigger, and then select Hold Focus from the drop-down menu.

5. Scroll up on the System Health dashboard and locate the Trigger Load by Trigger chart. This chart lists the percentage of cycles on the ExtraHop system that are being consumed by each trigger.
   

In this procedure, you will view exceptions in the trigger debug log that identify where the problem occurs in the trigger script, and then you will resolve the error.

1. In the upper right corner of the window, click the System Settings icon and then select Triggers.
2. Click Track Processing Time to open the trigger.
3. Click the Debug Log tab.
   In this walkthrough, the debug log displays output similar to the following figure:
   
   
4. Scroll through the log and look for entries flagged as Uncaught Error. Each error message includes the timestamp when the error occurred, the line number in the script that resulted in the error, and a description of the error.
   You should see the following error message in the log:

   Line 12: Uncaught Error: Action is not valid on event SMTP_REQUEST.

   Copy

   Tip:	In addition to exception errors, the debug log also displays uncaught syntax errors, such as an unexpected curly brace, or a type error, such as an invalid value.

5. Click the Editor tab, and then locate line 12 in the script to identify the action that is invalid on SMTP requests. In the following figure, line 12 shows that the action is to access the processingTime property on events:
   

   This information combined with the information from debug log error messages shows that accessing the processingTime property is invalid on SMTP request events.

6. Remove the unsupported SMTP event from the script and the trigger configuration by completing the following steps:
   1. Delete the following line from the trigger script:

      case 'SMTP_REQUEST':

      Copy
   2. Click the Configuration tab.
   3. Delete SMTP_REQUEST from the Events field.
7. Click Save and Close.
   The trigger is saved without displaying a validation error.
8. In the upper right corner of the window, click the System Settings icon and then select System Health.
9. Wait 5-10 minutes, and then scroll to the Trigger Exceptions chart that should look similar to the following figure:
   

In this section, you will create a custom trigger performance dashboard and add several charts discussed in this walkthrough.

1. Click Dashboards at the top of the page.
2. Click the command menu in the upper-right corner, and select New Dashboard.
3. In the Title field, type Trigger Performance.
4. Click Create.
5. Click the empty chart widget in your newly created dashboard to launch the Metric Explorer.
6. Click Add Source.
7. Click the Any Record Type drop-down menu and select Appliances.
8. From the list, select the name of the ExtraHop system you want.
9. In the Metrics field, type Trigger, and then select Capture System Health - Trigger Exceptions by Trigger from the list.
10. Click Save to return to your dashboard.
    The chart should look similar to the following figure:
    
    
11. Drag a new chart widget to the region and configure the chart by completing the following steps:
    1. Select the same ExtraHop system you specified for the previous chart.
    2. In the Metrics field, type Triggers, and then select Capture System Health - Trigger Cycles.
    3. In the Details section, click None, and then select Trigger.
    4. Click Save.
12. Drag a new chart widget to the region and configure the chart by completing the following steps:
    1. Select the same ExtraHop system you specified for the previous chart.
    2. In the Metrics field, type triggers, and then select Capture System Health - Trigger Executes.
    3. In the Details section, click None, and then select Trigger.
    4. Click Save.
13. Drag a new chart widget to the region and configure the chart by completing the following steps:
    1. Select the same ExtraHop system you specified for the previous chart.
    2. In the Metrics field, type Triggers, and then select Capture System Health - Trigger Drops.
    3. Click Save.
14. Click Exit Layout Mode from the upper-right corner.
    The dashboard should look similar to the following figure: