Skip to main content
My preferencesSign out
Proofpoint, Inc.

Essentials Threat API

The information and data accessible via this API contain Proofpoint proprietary, confidential, and/or trade secret information. Sharing or providing the information to another party without Proofpoint's express written consent is prohibited. Please review the updated Terms of Use for Proofpoint APIs. The API's Terms of Use can be found online at API Terms of Use | Proofpoint US.

Overview

Security Information and Event Management (SIEM) solutions are used by many organizations to identify and correlate various security events occurring in their point products. Examples of SIEM products include HP's ArcSight, IBM's QRadar, and Splunk. 

The API allows integration with these solutions by giving administrators the ability to periodically download detailed information about several types of threat events in a SIEM-compatible, vendor-neutral format. Currently, the following event types are exposed:

  1. Blocked or permitted clicks to threats recognized by URL Defense
  2. Blocked or delivered messages that contain threats recognized by URL Defense or Attachment Defense

API Features

General Service Notes

  1. All timestamps in the returned events are in UTC.
  2. The API is returns data in JSON format.
  3. A maximum of one hour of data can be requested in a single transaction.
  4. No paging support is available; all the applicable events in the requested time period will be returned in the log.
  5. The results provided by this API may not be in any logical order.
  6. Requests to the service may be throttled to prevent abuse.
  7. If results cannot be obtained within a timeout period, the service will return an error.
  8. The maximum time into the past that can be queried is 3 days with a maximum fetch time of 1 hour.

Security

Each request must use:

  • SSL.
  • Integration Keys to authenticate to the API.
  • the HTTP Basic Authorization method.
  • the HTTP GET method.

Standard Responses

Requests to the endpoints can produce a response with a variety of HTTP status codes. The following table describes the scenarios in which these codes can be produced.

Code Message Scenarios

200

Success

The request was successful. If records matching are found, they will be returned in the response body.

400

Bad Request

The request is missing a mandatory "request" parameter, a parameter contains data which is incorrectly formatted, or the API doesn't have enough information to determine the identity of the customer.

401

Unauthorized

There is no authorization information included in the request, the authorization information is incorrect, or the user is not authorized.

403

Forbidden

The user is authenticated for the service but is not authorized to access data for the given customer.

429

Too Many Requests

The user has made too many requests over the past 24 hours and has been throttled.

500

Internal Server Error

The service has encountered an unexpected situation and is unable to give a better response to the request.

Throttle Limits

These endpoints provide methods to fetch information about click and message events for a given time period. The number of queries connected to this resource are limited by a simple, rolling 24-hour throttle. Once exceeded, the API will start returning 429 HTTP status codes until 24 hours past the oldest request has elapsed. Requests to the clicks/permitted API and requests to other APIs are throttled into different pools. This allows more frequent queries to the clicks/permitted API.

Endpoint

Max Number of Requests

clicks/permitted

1800 per 24 hours

all

issues

clicks/blocked

messages/delivered

messages/blocked

1800 per 24 hours

 

Available Endpoints

All endpoints are available on one of two hosts, depending on region:

/v2/siem/clicks/blocked

Fetch events for clicks to malicious URLs blocked in the specified time period

/v2/siem/clicks/permitted

Fetch events for clicks to malicious URLs permitted in the specified time period

/v2/siem/messages/blocked

Fetch events for messages blocked in the specified time period which contained a known threat

/v2/siem/messages/delivered

Fetch events for messages delivered in the specified time period which contained a known threat

/v2/siem/issues

Fetch events for clicks to malicious URLs permitted and messages delivered containing a known threat within the specified time period. (It is a combination of /v2/siem/clicks/permitted and /v2/siem/messages/delivered)

/v2/siem/all

Fetch events for all clicks and messages relating to known threats within the specified time period

Required Parameters

One of the following three query parameters describing the desired time range for the data must be supplied with each request:

interval

A string containing an ISO8601-formatted interval. If this interval overlaps with previous requests for data, records from the previous request may be duplicated. The minimum interval is thirty seconds. The maximum interval is one hour.

Examples:

  • 2016-05-01T12:00:00Z/2016-05-01T13:00:00Z - an hour interval, beginning at noon UTC on 05-01-2016
  • PT30M/2016-05-01T12:30:00Z - the thirty minutes beginning at noon UTC on 05-01-2016 and ending at 12:30pm UTC
  • 2016-05-01T05:00:00-0700/PT30M - the same interval as above, but using -0700 as the time zone
sinceSeconds 

An integer representing a time window in seconds from the current API server time. The start of the window is the current API server time, rounded to the nearest minute, less the number of seconds provided. The end of the window is the current API server time rounded to the nearest minute. If JSON output is selected, the end time is included in the returned result.

sinceTime

A string containing an ISO8601 date. It represents the start of the data retrieval period. The end of the period is determined by current API server time rounded to the nearest minute. If JSON output is selected, the end time is included in the returned result.

Optional Parameters

One or more of these parameters may also be provided:

customerData

A boolean specifying whether customer threat data is returned. All customers underneath the partner will be included. If no value is specified, defaults to False. Either customerData or ownData MUST be set to True.

ownData

A boolean specifying whether the organization's own threat data is returned. If no value is specified, defaults to True. Either customerData or ownData MUST be set to True.

threatType

A string specifying which threat type will be returned in the data. If no value is specified, all threat types are returned. The following values are accepted:

  • url
  • attachment
  • messageText
threatStatus

A string specifying which threat statuses will be returned in the data. If no value is specified, active and cleared threats are returned. The following values are accepted:

  • active
  • cleared
  • falsePositive

Example Commands in Curl

The following commands assume that principal and secret are defined environment variables. They correspond to the service principal and secret that was created on the API Keys page.

curl "https://us-siem.proofpointessentials.com/v2/siem/issues?format=json&sinceTime=2016-05-01T12:00:00Z" --user "$PRINCIPAL:$SECRET" -s

Retrieves events from noon on 05/01/2016 to the present. Returned events are limited to just permitted clicks and delivered messages with known threats.

curl "https://us-siem.proofpointessentials.com/v2/siem/all?sinceSeconds=3600" --user "$PRINCIPAL:$SECRET" -s

Retrieves events to the present, starting 3600 seconds before the query time. All events are returned.

curl "https://us-siem.proofpointessentials.com/v2/siem/clicks/permitted?interval=PT30M/2016-05-01T12:30:00Z" --user "$PRINCIPAL:$SECRET" -s

Retrieves events from the thirty minutes beginning at noon UTC on 05-01-2016 and ending at 12:30pm UTC. Only permitted clicks are returned.

curl "https://us-siem.proofpointessentials.com/v2/siem/all?interval=PT30M/2016-05-01T12:30:00Z&threatStatus=falsePositive&threatStatus=active&threatStatus=cleared" --user "$PRINCIPAL:$SECRET" -s

Retrieves events from the thirty minutes beginning at noon UTC on 05-01-2016 and ending at 12:30pm UTC. All events are returned. False positives are included in the output.

Note About Event Time

Event Time in SIEM is the time the event was ingested into the SIEM. The time range used in the query parameters controls which events the SIEM API returns based on the time that the event was created, not the time the event occurred. The time an event is created is always after either of these two times:

  • the time that the message was sent or the time click occurred (messageTime or clickTime)

  • the time that the threat referenced by the message or click was recognized and condemned by Proofpoint (threatsInfoMap/threatTime)

In other words, a request using the sinceSeconds=3600 parameter will retrieve all events which have been created in the last hour. It is possible that the events returned from that interval reference messages or clicks which were first observed more than one hour ago—perhaps even several days ago.

Results

Structure

Fields

Type

Restrictions

Description

queryEndTime

DateTime

ISO8601 Timestamp

The time at which the period queried for data ended

messagesDelivered

Array

Message Events

An array containing all messages with threats which were delivered by PPS

messagesBlocked

Array

Message Events

An array containing all messages with threats which were quarantined by PPS

clicksPermitted

Array

Click Events

An array containing all clicks to URL threats which were permitted

clicksBlocked

Array

Click Events

An array containing all clicks to URL threats which were blocked

Message Events

Field

Content

Restrictions

Description

ccAddresses

Array of strings

 

A list of email addresses contained within the CC: header, excluding friendly names.

stackName

String

 

The name of the Essentials stack which processed the message.

customerEid

String

 

The customers entity ID.

customerName

String

 

The customer's name, as configured in Essentials.

parentEid

String

 

The parent's EID.

parentName

String

 

The parent's name, as configured in Essentials.

completelyRewritten

String

  • true

  • false

  • na

The rewrite status of the message. If value is 'true', all instances of URL threats within the message were successfully rewritten. If the value is 'false', at least one instance of the a threat URL was not rewritten. If the value is 'na', the message did not contain any URL-based threats.

fromAddress

String

 

The email address contained in the From: header, excluding friendly name.

GUID

String

 

The ID of the message within PPS. It can be used to identify the message in PPS and is guaranteed to be unique.

headerFrom

String

 

The full content of the From: header, including any friendly name.

headerReplyTo

String

 

 If present, the full content of the Reply-To: header, including any friendly names.

impostorScore

int

0-100

The impostor score of the message. Higher scores indicate higher certainty.

malwareScore

int

0-100

The malware score of the message. Higher scores indicate higher certainty.

messageDetailsUrl

String

 

A permalink to the messages' details page.

messageID

String

 

Message-ID extracted from the headers of the email message. It can be used to look up the associated message in PPS and is not unique.

messageParts

Array of JSON Structures

 

An array of structures which contain details about parts of the message, including both message bodies and attachments.

messageParts/contentType

String

 

The true, detected Content-Type of the messagePart. This may differ from the oContentType value.

messageParts/disposition

String

  • inline

  • attached

If the value is "inline," the messagePart is a message body. If the value is "attached," the messagePart is an attachment.

messageParts/filename

String

 

The filename of the messagePart.

messageParts/md5

String

 

The MD5 hash of the messagePart contents.

messageParts/oContentType

String

 

The declared Content-Type of the messagePart.

messageParts/sandboxStatus

String

  • unsupported

  • threat

  • clean

  • prefilter

  • uploaded

  • inprogress

  • uploaddisabled

The verdict returned by the sandbox during the scanning process. If the value is "unsupported", the messagePart is not supported by Attachment Defense and was not scanned. If the value is "clean", the sandbox returned a clean verdict. If the value is "threat", the sandbox returned a malicious verdict. If the value is "prefilter", the messagePart contained no active content, and was therefore not sent to the sandboxing service. If the value is "uploaded," the message was uploaded by PPS to the sandboxing service, but did not yet have a verdict at the time the message was processed. If the value is "inprogress," the attachment had been uploaded and was awaiting scanning at the time the message was processed. If the verdict is "uploaddisabled," the attachment was eligible for scanning, but was not uploaded because of PPS policy.

messageParts/sha256

String

 

The SHA256 hash of the messagePart contents.

messageSize

Integer

 

The size in bytes of the message, including headers and attachments.

messageTime

DateTime

ISO8601 Timestamp

When the message was delivered to the user or quarantined by PPS

phishScore

int

0-100

The phish score of the message. Higher scores indicate higher certainty.

quarantineRule

String

 

The name of the rule which quarantined the message. This appears only for messagesBlocked events.

recipient

String

Email Address

An array containing the email addresses of the SMTP (envelope) recipients

replyToAddress

String

 

The email address contained in the Reply-To: header, excluding friendly name.

sender

String

Email Address

The email address of the SMTP (envelope) sender. The user-part is hashed. The domain-part is cleartext.

senderIP

String

IP Address

The IP address of the sender.

spamScore

int

0-100

The spam score of the message. Higher scores indicate higher certainty.

subject

String

 

The subject line of the message, if available.

threatsInfoMap

Array of JSON Structures

 

An array of structures which contain details about detected threats within the message. There may be more than one threat per message.

threatsInfoMap/detectionType

String

Nullable

New field detectionType and ThreatInfo added to response

threatsInfoMap/classification

String

  • Malware

  • Phish

  • Spam

  • Impostor (for BEC/Message Text threats)

  • TOAD (Telephone-Oriented Attack Delivery)

The category of threat found in the message

threatsInfoMap/threat

String

  • SHA256

  • URL

  • email address

The artifact which was condemned by Proofpoint. The malicious URL, hash of the attachment threat, or email address of the impostor sender.

threatsInfoMap/threatId

String

 

The unique identifier associated with this threat. It can be used to query the forensics and campaign endpoints.

threatsInfoMap/threatStatus

String

  • active

  • falsepositive

  • cleared

The current state of the threat.

threatsInfoMap/threatTime

DateTime

ISO8601 Timestamp

Proofpoint assigned the threatStatus at this time.

threatsInfoMap/threatType

String

  • Attachment

  • URL

  • Message

Whether the threat was an attachment, URL, or message type.

threatsInfoMap/actors

Array of JSON Structures

 

An array of structures which contain details about the actors associated with a threat

toAddresses

Array of strings

 

A list of email addresses contained within the To: header, excluding friendly names.

xmailer

String

 

The content of the X-Mailer: header, if present.

Click Events

Field 

Content 

Restrictions 

Description 

classification

String

  • Malware

  • Phish

  • Spam

The threat category of the malicious URL

clickIP

String

IP Address

The external IP address of the user who clicked on the link. If the user is behind a firewall performing network address translation, the IP address of the firewall will be shown.

clickTime

DateTime

ISO8601 Timestamp

The time the user clicked on the URL

GUID

String

 

The ID of the message within PPS. It can be used to identify the message in PPS and is guaranteed to be unique.

id

String

 

The unique id of the click.

recipient

String

Email Address

The email address of the recipient

sender

String

Email Address

The email address of the sender. The user-part is hashed. The domain-part is cleartext.

senderIP

String

IP Address

The IP address of the sender

threatID

String

 

The unique identifier associated with this threat. It can be used to query the forensics and campaign endpoints. 

threatTime

DateTime

ISO8601 Timestamp

Proofpoint identified the URL as a threat at this time.

threatStatus

String

  • active

  • falsepositive

  • cleared

The current state of the threat.

url

String

URL

The malicious URL which was clicked

userAgent

String

 

The User-Agent header from the clicker's HTTP request

Example Output
{
    "clicksPermitted": [{
            "classification": "MALWARE",
            "clickIP": "192.0.2.1",
            "clickTime": "2016-06-24T19:17:44.000Z",
            "GUID": "b27dbea0-87d5-463b-b93c-4e8b708289ce",
            "id": "8c8b4895-a277-449f-r797-547e3c89b25a",
            "messageID": "8c6cfedd-3050-4d65-8c09-c5f65c38da81",
            "recipient": "bruce.wayne@pharmtech.zz",
            "sender": "9facbf452def2d7efc5b5c48cdb837fa@badguy.zz",
            "senderIP": "192.0.2.255",
            "threatID": "61f7622167144dba5e3ae4480eeee78b23d66f7dfed970cfc3d086cc0dabdf50",
            "threatTime": "2016-06-24T19:17:46.000Z",
            "threatStatus": "active",
            "url": "http://badguy.zz/",
            "userAgent": "Mozilla/5.0(WindowsNT6.1;WOW64;rv:27.0)Gecko/20100101Firefox/27.0"
        }
    ],
    "messagesBlocked": [{
            "GUID": "c26dbea0-80d5-463b-b93c-4e8b708219ce",
            "QID": "r2FNwRHF004109",
            "ccAddresses": ["bruce.wayne@university-of-education.zz"],
            "completelyRewritten": "true",
            "fromAddress": "badguy@evil.zz",
            "headerCC": "\"Bruce Wayne\" <bruce.wayne@university-of-education.zz>",
            "headerFrom": "\"A. Badguy\" <badguy@evil.zz>",
            "headerReplyTo": null,
            "headerTo": "\"Clark Kent\" <clark.kent@pharmtech.zz>; \"Diana Prince\" <diana.prince@pharmtech.zz>",
            "impostorScore": 0,
            "malwareScore": 100,
            "messageID": "20160624211145.62086.mail@evil.zz",
            "messageParts": [{
                    "contentType": "text/plain",
                    "disposition": "inline",
                    "filename": "text.txt",
                    "md5": "008c5926ca861023c1d2a36653fd88e2",
                    "oContentType": "text/plain",
                    "sandboxStatus": "unsupported",
                    "sha256": "85738f8f9a7f1b04b5329c590ebcb9e425925c6d0984089c43a022de4f19c281"
                }, {
                    "contentType": "application/pdf",
                    "disposition": "attached",
                    "filename": "Invoice for Pharmtech.pdf",
                    "md5": "5873c7d37608e0d49bcaa6f32b6c731f",
                    "oContentType": "application/pdf",
                    "sandboxStatus": "threat",
                    "sha256": "2fab740f143fc1aa4c1cd0146d334c5593b1428f6d062b2c406e5efe8abe95ca"
                }
            ],
            "messageTime": "2016-06-24T21:18:38.000Z",
            "modulesRun": ["pdr", "sandbox", "spam", "urldefense"],
            "phishScore": 46,
            "policyRoutes": ["default_inbound", "executives"],
            "quarantineFolder": "Attachment Defense",
            "quarantineRule": "module.sandbox.threat",
            "recipient": ["clark.kent@pharmtech.zz", "diana.prince@pharmtech.zz"],
            "replyToAddress": null,
            "sender": "e99d7ed5580193f36a51f597bc2c0210@evil.zz",
            "senderIP": "192.0.2.255",
            "spamScore": 4,
            "subject": "Please find a totally safe invoice attached.",
            "threatsInfoMap": [{
                    "classification": "MALWARE",
                    "threat": "2fab740f143fc1aa4c1cd0146d334c5593b1428f6d062b2c406e5efe8abe95ca",
                    "threatId": "2fab740f143fc1aa4c1cd0146d334c5593b1428f6d062b2c406e5efe8abe95ca",
                    "threatStatus": "active",
                    "threatTime": "2016-06-24T21:18:38.000Z",
                    "threatType": "ATTACHMENT",
                }, {
                    "classification": "MALWARE",
                    "threat": "badsite.zz",
                    "threatId": "3ba97fc852c66a7ba761450edfdfb9f4ffab74715b591294f78b5e37a76481aa",
                    "threatTime": "2016-06-24T21:18:07.000Z",
                    "threatType": "URL",
                }
            ],
            "toAddresses": ["clark.kent@pharmtech.zz", "diana.prince@pharmtech.zz"]
        }, "xmailer": "Spambot v2.5"],
    "queryEndTime": "2016-06-24T21:36:00Z"
}