crowdstrike: add support for DataProtectionDetectionSummaryEvent events

[navnit-elastic]

Proposed commit message

crowdstrike: add support for DataProtectionDetectionSummaryEvent events

This change enhances ingest pipeline for falcon and fdr data streams
to parse a new DataProtectionDetectionSummaryEvent events.
The test samples are taken from a live CrowdStrike instance.

Note

The test samples have been modified to test pipeline scenarios. This requires a wider variety of logs, but not all types could be generated at this time.

Schema documentation: https://falcon.us-2.crowdstrike.com/documentation/page/d88d9ed6/streaming-api-event-dictionary#l7b00fac

DataProtectionDetectionSummaryEvent

Event Description: Falcon generates a new DataProtectionDetectionSummaryEvent for every data protection detection. Based on the detection type (rule-based or anomaly-based) some of the fields listed below would not be present.

Platform: Windows

Fields

Field	Description
AgentId	Falcon sensor Agent ID
CompositeId	Data-protection-based detection ID in Falcon
Name	Name of the data protection generated detection
Description	Description of the data protection generated detection
MitreAttack	An array containing info about associated MITRE tactics and techniques.PatternId: The specific pattern ID associated with the tactic and technique.Tactic: The name of the tactic associated with the behavior.TacticId: The ID of the tactic associated with the behavior.Technique: The name of the technique/sub-technique associated with the behavior.TechniqueId: The ID of the technique/sub-technique associated with the behavior.
Tactic	Description of the tactic, as defined by the MITRE ATT&CK matrix.Note: Deprecated. See MitreAttack for this info.
TacticId	Identifier of the tactic, as defined by the MITRE ATT&CK matrix (e.g., TA0001).Note: Deprecated. See MitreAttack for this info.
Technique	Name of the technique or sub-technique associated with the behavior.Note: Deprecated. See MitreAttack for this info.
TechniqueId	ID of the technique or sub-technique associated with the behavior.Note: Deprecated. See MitreAttack for this info.
Severity	Severity of the detection:10 - informational (rule-based only)20–30 - low40–50 - medium60–70 - high80–90 - critical (rule-based only)
SeverityName	The severity level of the detection as a string (e.g., Informational / Low / Medium / High / Critical)
FalconHostLink	URL to view the incident in Falcon console
Destination	Destination of the data. Describes the channel of egress and a specific set of metadata depending on the egress channel value.
UserName	The name of the user that performed the egress
UserTitle	Title of the user that performed the egress
UserDepartment	Department name of the user that performed the egress
UserSid	Active Directory User SID of the user that initiated egress
Hostname	Hostname on which the egress was initiated
PatternId	The numerical ID of the pattern associated with the action taken on the detection
DataVolume	File size of the egressed data
DetectionType	Type of the detection:• Anomaly-based• Rule-based
EgressEventId	A unique identifier for every egress event (Rule-based detection only)
Policy	ID and Name of the policy that matched the egress event (Rule-based detection only)
MatchedClassification	Data protection classification applied to the egress according to precedence (includes ID and Name). (Rule-based detection only)
RelatedClassifications	List of related data protection classifications based on the egress. (Rule-based detection only)
UserNotified	Indicates if the user was notified.Values: True / False (Rule-based detection only)
ResponseAction	Response action as set on the egress rule (Rule-based detection only)
RuleId	Matched rule ID of the egress (Rule-based detection only)
IsClipboard	Indicates if it is a clipboard egress.Values: True / False (Rule-based detection only)
Labels	ID and Name of the data labels added to the file included in the egress (Rule-based detection only)
ContentPatterns	ID and Name of the content patterns matching the file content included in the egress (Rule-based detection only)
OriginWebLocations	Web location and user details related to the egress (e.g., name, cloud username, and web location) (Rule-based detection only)
ContentSha	Content 256SHA of the file included in the egress (Rule-based detection only)
Filename	Filename included in the egress (Rule-based detection only)
FileType	Metadata about the file included in the egress (e.g., categoryID, categoryName, path, and type) (Rule-based detection only)
EventTimestamp	Timestamp of egress attempt (Rule-based detection only)
EgressSessionId	ID of the egress session event generated to evaluate for anomaly-based detections (Anomaly-based detection only)
SessionStartTimestamp	Start timestamp of the egress session (Anomaly-based detection only)
SessionEndTimestamp	End timestamp of the egress session (Anomaly-based detection only)
UserMapped	Indicates whether a known-user account is mapped to a real user (True/False) (Anomaly-based detection only)
FilesEgressedCount	Number of files included in the egress session (Anomaly-based detection only)
FileCategoryCounts	Name, ID, and Count of the file categories included in the egress session (Anomaly-based detection only)
ContentPatternCounts	Name, ID, and Count of the content patterns matching the files in the egress session (Anomaly-based detection only)
AnodeIndicators	Metadata of anomaly indicators (e.g., name, destination, entity type, file count/data volume increase from baseline) (Anomaly-based detection only)

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

• [ ]

How to test this PR locally

Pipeline Tests for FDR:

--- Test results for package: crowdstrike - START ---
╭─────────────┬─────────────┬───────────┬───────────────────────────────────────────────────────────────────────────┬────────┬──────────────╮
│ PACKAGE     │ DATA STREAM │ TEST TYPE │ TEST NAME                                                                 │ RESULT │ TIME ELAPSED │
├─────────────┼─────────────┼───────────┼───────────────────────────────────────────────────────────────────────────┼────────┼──────────────┤
│ crowdstrike │ fdr         │ pipeline  │ (ingest pipeline warnings test-data.log)                                  │ PASS   │ 481.800724ms │
│ crowdstrike │ fdr         │ pipeline  │ (ingest pipeline warnings test-fdr-data-protection-detection-summary.log) │ PASS   │ 413.784101ms │
│ crowdstrike │ fdr         │ pipeline  │ (ingest pipeline warnings test-fdr-epp-detection-summary.log)             │ PASS   │ 423.815258ms │
│ crowdstrike │ fdr         │ pipeline  │ (ingest pipeline warnings test-fdr-lengthy-field-delete.log)              │ PASS   │ 423.764575ms │
│ crowdstrike │ fdr         │ pipeline  │ (ingest pipeline warnings test-fdr-lengthy-field-index.log)               │ PASS   │ 420.291825ms │
│ crowdstrike │ fdr         │ pipeline  │ (ingest pipeline warnings test-fdr.log)                                   │ PASS   │  459.85565ms │
│ crowdstrike │ fdr         │ pipeline  │ (ingest pipeline warnings test-fdrv2-notmanaged.log)                      │ PASS   │  540.12158ms │
│ crowdstrike │ fdr         │ pipeline  │ (ingest pipeline warnings test-linux.log)                                 │ PASS   │  418.28868ms │
│ crowdstrike │ fdr         │ pipeline  │ (ingest pipeline warnings test-macos.log)                                 │ PASS   │ 454.163052ms │
│ crowdstrike │ fdr         │ pipeline  │ (ingest pipeline warnings test-tags-formats.log)                          │ PASS   │ 431.229053ms │
│ crowdstrike │ fdr         │ pipeline  │ (ingest pipeline warnings test-windows.log)                               │ PASS   │ 451.897229ms │
│ crowdstrike │ fdr         │ pipeline  │ test-data.log                                                             │ PASS   │ 258.415868ms │
│ crowdstrike │ fdr         │ pipeline  │ test-fdr-data-protection-detection-summary.log                            │ PASS   │ 205.131493ms │
│ crowdstrike │ fdr         │ pipeline  │ test-fdr-epp-detection-summary.log                                        │ PASS   │ 596.877799ms │
│ crowdstrike │ fdr         │ pipeline  │ test-fdr-lengthy-field-delete.log                                         │ PASS   │ 164.601283ms │
│ crowdstrike │ fdr         │ pipeline  │ test-fdr-lengthy-field-index.log                                          │ PASS   │ 183.671326ms │
│ crowdstrike │ fdr         │ pipeline  │ test-fdr.log                                                              │ PASS   │   3.4338112s │
│ crowdstrike │ fdr         │ pipeline  │ test-fdrv2-notmanaged.log                                                 │ PASS   │  229.66601ms │
│ crowdstrike │ fdr         │ pipeline  │ test-linux.log                                                            │ PASS   │ 296.438905ms │
│ crowdstrike │ fdr         │ pipeline  │ test-macos.log                                                            │ PASS   │ 593.096238ms │
│ crowdstrike │ fdr         │ pipeline  │ test-tags-formats.log                                                     │ PASS   │  163.62555ms │
│ crowdstrike │ fdr         │ pipeline  │ test-windows.log                                                          │ PASS   │ 2.869325867s │
╰─────────────┴─────────────┴───────────┴───────────────────────────────────────────────────────────────────────────┴────────┴──────────────╯
--- Test results for package: crowdstrike - END   ---
Done

Pipeline Test for Falcon:

--- Test results for package: crowdstrike - START ---
╭─────────────┬─────────────┬───────────┬──────────────────────────────────────────────────────────────────────────────┬────────┬──────────────╮
│ PACKAGE     │ DATA STREAM │ TEST TYPE │ TEST NAME                                                                    │ RESULT │ TIME ELAPSED │
├─────────────┼─────────────┼───────────┼──────────────────────────────────────────────────────────────────────────────┼────────┼──────────────┤
│ crowdstrike │ falcon      │ pipeline  │ (ingest pipeline warnings test-event-stream.log)                             │ PASS   │ 477.947283ms │
│ crowdstrike │ falcon      │ pipeline  │ (ingest pipeline warnings test-falcon-audit-events.log)                      │ PASS   │ 460.165382ms │
│ crowdstrike │ falcon      │ pipeline  │ (ingest pipeline warnings test-falcon-auth-activity.log)                     │ PASS   │ 440.931839ms │
│ crowdstrike │ falcon      │ pipeline  │ (ingest pipeline warnings test-falcon-cspmioa-streaming.log)                 │ PASS   │ 488.591236ms │
│ crowdstrike │ falcon      │ pipeline  │ (ingest pipeline warnings test-falcon-cspmsearch-streaming.log)              │ PASS   │ 430.170911ms │
│ crowdstrike │ falcon      │ pipeline  │ (ingest pipeline warnings test-falcon-data-protection-detection-summary.log) │ PASS   │ 480.396331ms │
│ crowdstrike │ falcon      │ pipeline  │ (ingest pipeline warnings test-falcon-detection-summary.log)                 │ PASS   │ 464.402403ms │
│ crowdstrike │ falcon      │ pipeline  │ (ingest pipeline warnings test-falcon-epp-detection-summary.log)             │ PASS   │ 467.236019ms │
│ crowdstrike │ falcon      │ pipeline  │ (ingest pipeline warnings test-falcon-events.log)                            │ PASS   │ 464.043804ms │
│ crowdstrike │ falcon      │ pipeline  │ (ingest pipeline warnings test-falcon-firewall.log)                          │ PASS   │ 472.725904ms │
│ crowdstrike │ falcon      │ pipeline  │ (ingest pipeline warnings test-falcon-identity-protection-incident.log)      │ PASS   │  438.95106ms │
│ crowdstrike │ falcon      │ pipeline  │ (ingest pipeline warnings test-falcon-incident-summary.log)                  │ PASS   │  430.40317ms │
│ crowdstrike │ falcon      │ pipeline  │ (ingest pipeline warnings test-falcon-ipd-summary.log)                       │ PASS   │  422.42387ms │
│ crowdstrike │ falcon      │ pipeline  │ (ingest pipeline warnings test-falcon-mobile-detection-summary.log)          │ PASS   │ 471.428698ms │
│ crowdstrike │ falcon      │ pipeline  │ (ingest pipeline warnings test-falcon-recon-notification.log)                │ PASS   │ 426.277095ms │
│ crowdstrike │ falcon      │ pipeline  │ (ingest pipeline warnings test-falcon-remote-response.log)                   │ PASS   │ 524.861896ms │
│ crowdstrike │ falcon      │ pipeline  │ (ingest pipeline warnings test-falcon-sample.log)                            │ PASS   │ 311.724417ms │
│ crowdstrike │ falcon      │ pipeline  │ (ingest pipeline warnings test-falcon-tags-list.log)                         │ PASS   │ 328.150766ms │
│ crowdstrike │ falcon      │ pipeline  │ (ingest pipeline warnings test-falcon-tags.log)                              │ PASS   │ 304.795276ms │
│ crowdstrike │ falcon      │ pipeline  │ (ingest pipeline warnings test-falcon-user-activity.log)                     │ PASS   │ 339.134353ms │
│ crowdstrike │ falcon      │ pipeline  │ (ingest pipeline warnings test-falcon-xdr-detection-summary.log)             │ PASS   │ 310.633625ms │
│ crowdstrike │ falcon      │ pipeline  │ test-event-stream.log                                                        │ PASS   │ 295.973254ms │
│ crowdstrike │ falcon      │ pipeline  │ test-falcon-audit-events.log                                                 │ PASS   │ 237.906635ms │
│ crowdstrike │ falcon      │ pipeline  │ test-falcon-auth-activity.log                                                │ PASS   │ 161.361214ms │
│ crowdstrike │ falcon      │ pipeline  │ test-falcon-cspmioa-streaming.log                                            │ PASS   │ 162.775867ms │
│ crowdstrike │ falcon      │ pipeline  │ test-falcon-cspmsearch-streaming.log                                         │ PASS   │ 148.688993ms │
│ crowdstrike │ falcon      │ pipeline  │ test-falcon-data-protection-detection-summary.log                            │ PASS   │ 184.539358ms │
│ crowdstrike │ falcon      │ pipeline  │ test-falcon-detection-summary.log                                            │ PASS   │ 176.223635ms │
│ crowdstrike │ falcon      │ pipeline  │ test-falcon-epp-detection-summary.log                                        │ PASS   │ 668.556289ms │
│ crowdstrike │ falcon      │ pipeline  │ test-falcon-events.log                                                       │ PASS   │ 167.907611ms │
│ crowdstrike │ falcon      │ pipeline  │ test-falcon-firewall.log                                                     │ PASS   │  169.07128ms │
│ crowdstrike │ falcon      │ pipeline  │ test-falcon-identity-protection-incident.log                                 │ PASS   │ 136.102318ms │
│ crowdstrike │ falcon      │ pipeline  │ test-falcon-incident-summary.log                                             │ PASS   │ 146.504388ms │
│ crowdstrike │ falcon      │ pipeline  │ test-falcon-ipd-summary.log                                                  │ PASS   │ 162.799593ms │
│ crowdstrike │ falcon      │ pipeline  │ test-falcon-mobile-detection-summary.log                                     │ PASS   │ 156.542712ms │
│ crowdstrike │ falcon      │ pipeline  │ test-falcon-recon-notification.log                                           │ PASS   │ 125.228673ms │
│ crowdstrike │ falcon      │ pipeline  │ test-falcon-remote-response.log                                              │ PASS   │ 180.252138ms │
│ crowdstrike │ falcon      │ pipeline  │ test-falcon-sample.log                                                       │ PASS   │ 157.517314ms │
│ crowdstrike │ falcon      │ pipeline  │ test-falcon-tags-list.log                                                    │ PASS   │  97.934393ms │
│ crowdstrike │ falcon      │ pipeline  │ test-falcon-tags.log                                                         │ PASS   │ 108.215587ms │
│ crowdstrike │ falcon      │ pipeline  │ test-falcon-user-activity.log                                                │ PASS   │ 122.954359ms │
│ crowdstrike │ falcon      │ pipeline  │ test-falcon-xdr-detection-summary.log                                        │ PASS   │ 114.424759ms │
╰─────────────┴─────────────┴───────────┴──────────────────────────────────────────────────────────────────────────────┴────────┴──────────────╯
--- Test results for package: crowdstrike - END   ---
Done

Related issues

Screenshots

[elastic-vault-github-plugin-prod]

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: c0ccf6a

cc @navnit-elastic

[elasticmachine]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[efd6]

LGTM, but I'd like to get @w0rk3r to take a look as well.