Get SIEM Logs

Description

This API endpoint can be used to get MTA logs. Logs are available up to 7-days from current date, and use tokens to bookmark progress in logs' downloaded.

Pre-requisites

The data served by this endpoint is only generated when the requested log type is enabled for your account in the Enhanced Logging section of the Account Settings page in the Administration Console. To enable this feature:

  1. Log in to the Administration Console.
  2. Navigate to the Administration | Account | Account Settings menu.
  3. Locate and select the Enhanced Logging section.
  4. Enable the log type(s) you would like to get using this endpoint.
  5. Select Save to apply the change.

In order to use this endpoint the logged in user must be a Mimecast administrator with at least the Gateway | Tracking | Read permission.

Notes

  • Unlike the majority of our endpoints this function returns an application/octet-stream stream of log data.
  • Log data is written on every Mimecast server where the application for each log type runs and is made available for you every 30 minutes throughout the day.
  • Log data is stored by Mimecast for 7 days only, however once downloaded you can keep the data for as long as you require.
  • Only one log file is returned per request and Mimecast uses a token based system for you to request the next available log.
  • Tokens are returned in the "mc-siem-token" response header.
  • The data type and date of each log downloaded is indicated in the "Content-Disposition" response header.

URI

To use this endpoint you send a POST request to:

  • /api/audit/get-siem-logs

Request Headers

The following request headers must be included in your request:

Field Description
Authorization Please see the Authorization guide for more information on building the Authorization header.
x-mc-req-id

A randomly generated GUID, for example,

8578FCFC-A305-4D9A-99CB-F4D5ECEFE297
x-mc-app-id The Application ID provided with your Registered API Application.
x-mc-date

The current date and time in the following format, for example,

Tue, 24 Nov 2015 12:50:11 UTC

Request Body

{
  "data":[
    {
      "type": "String",
      "token": "String",
      "fileFormat": "String",
      "compress": true
    }
  ]
}
data
Field Type Required Description
type String Required The type of log file to request. Currently the only type available is "MTA." However Mimecast may add new log types in the future.
token String Optional Used to request the next available log file. The token value is found in the "mc-siem-token" response header of the previous request. If not included the earliest available log will be returned by the API.
fileFormat String Optional Used to determine output format.  Must be one of - json for JSON output and key_value for Pipe Delimited.  Default setting is key_value.   
compress Boolean Optional Used to determine if response will be a text-based output, or a .zip file stream containing multiple log files. Default value is false.

Response

Response with data when compress option is false

When there are log files available this endpoint returns an application/octet-stream stream of data, with the following example response headers:

Content-Type: application/octet-stream
Content-Disposition: attachment; filename="delivery_20160705162902600.log"
Pragma: private, no-cache
Cache-control: private, must-revalidate, no-store
mc-siem-token: eNo9jUELgjAYQP_Ldx60xTTyVi1a...
Transfer-Encoding: chunked
Date: Mon, 11 Jul 2016 09:55:50 +0100
Connection: Keep-Alive

Where:

Field Description
Content-Disposition Describes the type of data returned. The format of the "filename" key follows the type_dateTime pattern.
mc-siem-token The token value that should be used to request the next available log file.
Response with data when compress option is true

When there are log files available this endpoint returns an application/octet-stream stream of a ZIP archive file.

Content-Type: application/octet-stream
Content-Disposition: attachment; filename="8578FCFC-A305-4D9A-99CB-F4D5ECEFE297.zip"
Pragma: private, no-cache
Cache-control: private, must-revalidate, no-store
mc-siem-token: eNo9jUELgjAYQP_Ldx60xTTyVi1a...
Transfer-Encoding: chunked
Date: Mon, 11 Jul 2016 09:55:50 +0100
Connection: Keep-Alive

Where:

Field Description
Content-Disposition Describes the type of data returned. The format of the "filename" uses the GUID presented in request header x-mc-req-id.
mc-siem-token The token value that should be used to request the next available log file.
Response when no new logs are available

When all logs have been downloaded the response will conform to the API's standard JSON payload with the isLastToken boolean value found in the "meta" section. For example:

200
OK
Response:
{
     "meta": {
          "isLastToken":Boolean,
          "status":Number
     },
     "data":[],
     "fail":[]
}

Sample Code

Sample code is provided to demonstrate how to use the API and is not representative of a production application. To use the sample code; complete the required variables as described, populate the desired values in the request body, and execute in your favorite IDE.  Please see the Global Base URL's page to find the correct base URL to use for your account.

POST {base_url}/api/audit/get-siem-logs
Authorization: MC {accesskKey}:{Base64 encoded signed Data To Sign}
x-mc-date: {dateTime}
x-mc-req-id: {unique id}
x-mc-app-id: {applicationId}
Content-Type: application/json
Accept: application/json

{
  "data":[
    {
      "type": "MTA",
      "token": "value of previous mc-siem-token response header",
      "fileFormat": "JSON",
      "compress": true
    }
  ]
}
import base64
import hashlib
import hmac
import uuid
import datetime
import requests

# Setup required variables
base_url = "https://xx-api.mimecast.com"
uri = "/api/audit/get-siem-logs"
url = base_url + uri
access_key = "YOUR ACCESS KEY"
secret_key = "YOUR SECRET KEY"
app_id = "YOUR APPLICATION ID"
app_key = "YOUR APPLICATION KEY"

# Generate request header values
request_id = str(uuid.uuid4())
hdr_date = datetime.datetime.utcnow().strftime("%a, %d %b %Y %H:%M:%S") + " UTC"

# DataToSign is used in hmac_sha1
dataToSign = ':'.join([hdr_date, request_id, uri, app_key])

# Create the HMAC SHA1 of the Base64 decoded secret key for the Authorization header
hmac_sha1 = hmac.new(base64.b64decode(secret_key), dataToSign.encode(), digestmod=hashlib.sha1).digest()

# Use the HMAC SHA1 value to sign the hdrDate + ":" requestId + ":" + URI + ":" + appkey
sig = base64.b64encode(hmac_sha1).rstrip()

# Create request headers
headers = {
    'Authorization': 'MC ' + access_key + ':' + sig.decode(),
    'x-mc-app-id': app_id,
    'x-mc-date': hdr_date,
    'x-mc-req-id': request_id,
    'Content-Type': 'application/json'
}

payload = {
    "data":[
        {
            'type': 'MTA',
            'token': 'value of previous mc-siem-token response header',
            'fileFormat': 'JSON',
            'compress': true
        }
    ]
}

r = requests.post(url=url, headers=headers, data=str(payload))

print(r.text)
static void Main(string[] args)
        {
            //Setup required variables
            string baseUrl = "https://xx-api.mimecast.com";
            string uri = "/api/audit/get-siem-logs";
            string accessKey = "YOUR ACCESS KEY";
            string secretKey = "YOUR SECRET KEY";
            string appId = "YOUR APPLICATION ID";
            string appKey = "YOUR APPLICATION KEY";

            //Generate request header values
            string hdrDate = System.DateTime.Now.ToUniversalTime().ToString("R");
            string requestId = System.Guid.NewGuid().ToString();

            //Create the HMAC SHA1 of the Base64 decoded secret key for the Authorization header
            System.Security.Cryptography.HMAC h = new System.Security.Cryptography.HMACSHA1(System.Convert.FromBase64String(secretKey));

            //Use the HMAC SHA1 value to sign the hdrDate + ":" requestId + ":" + URI + ":" + appkey
            byte[] hash = h.ComputeHash(System.Text.Encoding.Default.GetBytes(hdrDate + ":" + requestId + ":" + uri + ":" + appKey));

            //Build the signature to be included in the Authorization header in your request
            string signature = "MC " + accessKey + ":" + System.Convert.ToBase64String(hash);

            //Build Request
            System.Net.HttpWebRequest request = (System.Net.HttpWebRequest)System.Net.WebRequest.Create(baseUrl + uri);
            request.Method = "POST";
            request.ContentType = "application/json";

            //Add Headers
            request.Headers[System.Net.HttpRequestHeader.Authorization] = signature;
            request.Headers.Add("x-mc-date", hdrDate);
            request.Headers.Add("x-mc-req-id", requestId);
            request.Headers.Add("x-mc-app-id", appId);

            //Add request body
            //Create and write data to stream
            string postData = @"{
                 ""data"":[
                        {
                            ""type"": ""MTA"",
                            ""token"": ""value of previous mc-siem-token response header"",
                            ""fileFormat"": """JSON""",
                            ""compress"": true
                        }
                    ]
                }";

            byte[] payload = System.Text.Encoding.UTF8.GetBytes(postData);

            System.IO.Stream stream = request.GetRequestStream();
            stream.Write(payload, 0, payload.Length);
            stream.Close();

            //Send Request
            System.Net.HttpWebResponse response = (System.Net.HttpWebResponse)request.GetResponse();

            //Output response to console
            System.IO.StreamReader reader = new System.IO.StreamReader(response.GetResponseStream());
            string responseBody = "";
            string temp = null;
            while ((temp = reader.ReadLine()) != null)
            {
                responseBody += temp;
            };
            System.Console.WriteLine(responseBody);
            System.Console.ReadLine();
        }
#Setup required variables
$baseUrl = "https://xx-api.mimecast.com"
$uri = "/api/audit/get-siem-logs"
$url = $baseUrl + $uri
$accessKey = "YOUR ACCESS KEY"
$secretKey = "YOUR SECRET KEY"
$appId = "YOUR APPLICATION ID"
$appKey = "YOUR APPLICATION KEY"

#Generate request header values
$hdrDate = (Get-Date).ToUniversalTime().ToString("ddd, dd MMM yyyy HH:mm:ss UTC")
$requestId = [guid]::NewGuid().guid

#Create the HMAC SHA1 of the Base64 decoded secret key for the Authorization header
$sha = New-Object System.Security.Cryptography.HMACSHA1
$sha.key = [Convert]::FromBase64String($secretKey)
$sig = $sha.ComputeHash([Text.Encoding]::UTF8.GetBytes($hdrDate + ":" + $requestId + ":" + $uri + ":" + $appKey))
$sig = [Convert]::ToBase64String($sig)

#Create Headers
$headers = @{"Authorization" = "MC " + $accessKey + ":" + $sig;
                "x-mc-date" = $hdrDate;
                "x-mc-app-id" = $appId;
                "x-mc-req-id" = $requestId;
                "Content-Type" = "application/json"}

#Create post body
$postBody = "{
                 ""data"":[
                        {
                            ""type"": ""MTA"",
                            ""token"": ""value of previous mc-siem-token response header""
                            ""fileFormat"": """JSON""",
                            ""compress"": true
                        }
                    ]
                }"

#Send Request
$response = Invoke-RestMethod -Method Post -Headers $headers -Body $postBody -Uri $url

#Print the response
$response
public static void main(String[] args) throws java.io.IOException, java.security.NoSuchAlgorithmException, java.security.InvalidKeyException {

        //set up variables for request
        String baseUrl = "https://xx-api.mimecast.com";
        String uri = "/api/audit/get-siem-logs";
        String url = "https://" + baseUrl + uri;
        String accessKey = "YOUR ACCESS KEY";
        String secretKey = "YOUR SECRET KEY";
        String appId = "YOUR APPLICATION ID";
        String appKey = "YOUR APPLICATION KEY";

        //create URL object
        java.net.URL obj = new java.net.URL(url);

        // set guid for x-mc-req-id header
        String guid = java.util.UUID.randomUUID().toString();

        // set date for x-mc-date header
        java.text.SimpleDateFormat sdf = new java.text.SimpleDateFormat("EEE, d MMM yyyy HH:mm:ss z");
        sdf.setTimeZone(java.util.TimeZone.getTimeZone("UTC"));
        String date = sdf.format(new java.util.Date());

        //create signature for the Authorization header
        String dataToSign = date + ":" + guid + ":" + uri + ":" + appKey;
        String hmacSHA1 = "HmacSHA1";
        javax.crypto.spec.SecretKeySpec signingKey = new javax.crypto.spec.SecretKeySpec(org.apache.commons.codec.binary.Base64.decodeBase64(secretKey.getBytes()), hmacSHA1);
        javax.crypto.Mac mac = javax.crypto.Mac.getInstance(hmacSHA1);
        mac.init(signingKey);
        String sig = new String(org.apache.commons.codec.binary.Base64.encodeBase64(mac.doFinal(dataToSign.getBytes())));

        // create request object
        javax.net.ssl.HttpsURLConnection con = (javax.net.ssl.HttpsURLConnection) obj.openConnection();

        //set request type to POST
        con.setRequestMethod("POST");
        con.setDoOutput(true);

        //add reuqest headers
        con.setRequestProperty("Authorization", "MC " + accessKey + ":" + sig);
        con.setRequestProperty("x-mc-req-id", guid);
        con.setRequestProperty("x-mc-app-id", appId);
        con.setRequestProperty("x-mc-date", date);
        con.setRequestProperty("Content-Type", "application/json");
        con.setRequestProperty("Accept", "application/json");

        //Add post body to the request
        String postBody = "{\n" +
                "    \"data\":[\n" +
                "        {\n" +
                "            \"type\": \"MTA\",\n" +
                "            \"token\": \"value of previous mc-siem-token response header",\n" +
                "            \"fileFormat\": \"JSON\",\n" +
                "            \"compress\": true\n" +
                "        }\n" +
                "    ]\n" +
                "}";
        java.io.OutputStream os = con.getOutputStream();
        os.write(postBody.getBytes("UTF-8"));
        os.close();

        //process response
        java.io.BufferedReader in = new java.io.BufferedReader(
                new java.io.InputStreamReader(con.getInputStream()));
        String inputLine;
        StringBuffer response = new StringBuffer();

        while ((inputLine = in.readLine()) != null) {
            response.append(inputLine);
        }
        in.close();

        //return result
        java.lang.System.out.println(response.toString());
    }
Back to Top