RiskIQ Illuminate

Reputation Scoring

The RiskIQ Illuminate platform provides dynamic reputation scoring on IPs and hostnames based on real-world activity, indicators, and behaviors.

This library provides access to reptuation scores through the object analyzer (recommended), the command line interface, or the low-level request wrappers.

Review the Getting Started guide for details on setting up your credentials and development environment.

Hostname and IP Reputation Analysis

>>> from passivetotal import analyzer
>>> analyzer.init()
>>> reputation = analyzer.Hostname('2020-windows.com').reputation
>>> print(reputation)
72 (SUSPICIOUS)
>>> print(analyzer.IPAddress('123.213.1.23').reputation.score)
88
>>> analyzer.IPAddress('123.213.1.23').reputation.rules
[{'name': 'Third Party Blocklist (vo)',
  'description': 'Threat Type: SERVICE SCANNER',
  'severity': 5,
  'link': None},
 {'name': 'Open ports observed',
  'description': 'The number of open ports may indicate maliciousness',
  'severity': 3,
 'link': None}]

The Hostname and IPAddress analyzer objects provide a reputation property that returns an instance of a ReputationScore object. That object can be treated directly like a string or an integer, or you can access the properties directly.

class passivetotal.analyzer.hostname.Hostname(hostname)

Represents a hostname such as api.passivetotal.org.

Instances are stored as class members so subsequent instantiations for the same hostname return the same object.

Because of this, storing instances in variables is optional, which can be especially useful in interactive sessions such as Jupyter notebooks.

property reputation

RiskIQ Illuminate Reputation profile for a hostname or IP.

Return type

passivetotal.analyzer.illuminate.reputation.ReputationScore

class passivetotal.analyzer.ip.IPAddress(ip)

Represents an IPv4 address such as 8.8.8.8

Instances are stored as class members so subsequent instantiations for the same IP return the same object.

Because of this, storing instances in variables is optional, which can be especially useful in interactive sessions such as Jupyter notebooks.

property reputation

RiskIQ Illuminate Reputation profile for a hostname or IP.

Return type

passivetotal.analyzer.illuminate.reputation.ReputationScore

Reputation Score CLI

The pt-client command line script provides quick access to reputation profiles on one or more hosts in several formats. To get started, view the help for the illuminate command:

(venv) % pt-client illuminate --help
usage: passivetotal illuminate [-h] [--reputation] [--format {json,csv,text}] [--brief] query [query ...]

positional arguments:
query                 One or more hostnames or IPs

optional arguments:
-h, --help            show this help message and exit
--reputation          Get hostname or IP reputation from RiskIQ Illuminate.
--format {json,csv,text}
                        Format of the output from the query
--brief               Create a brief output; for reputation, prints score and classification only

Use --reputation and pass a space-separated list of hostnames or IPs as the query parameter.

The default format is “json” - for interactive use, try --format=text:

(venv) % pt-client illuminate --reputation --format=text 2020-windows.com
2020-windows.com  72 (SUSPICIOUS)
   Registrant email provider (severity 3)
      Domain is registered with an email provider that is
      more likely to register malicious domains
   Registrar (severity 3)
      Domains registered with this registrar are more likely
      to be malicious

The --brief option produces a more compact output with one result per line, which is also useful with the --format=csv parameter to prepare a compact dataset for import into another product.

Pass multiple hostnames or IPs at the end of the command (separated by spaces) to analyze multiple hosts at one time.

Reputation Request Wrapper

Use the low-level Illuminate request wrapper for direct queries to the API.

class passivetotal.libs.illuminate.IlluminateRequest(*args, **kwargs)

Client to interface with the RiskIQ Illuminate calls from the PassiveTotal API.

get_reputation(**kwargs)

Get RiskIQ Illuminate score for a domain or IP address.

Reference: https://api.riskiq.net/api/reputation/

Parameters

query – Domain or IP address to search

Returns

Dict of results

Reputation Score Reference

class passivetotal.analyzer.illuminate.ReputationScore(api_response, query=None)

RiskIQ Illuminate Reputation profile for a hostname or an IP.

property as_dict

Representation as a dictionary object.

property classification

Reputation classification as a string.

Typical values include GOOD, SUSPICIOUS, MALICIOUS, or UNKNOWN.

property rules

List of rules that informed the reputation score.

Returns a list of dictionaries.

property score

Reputation score as an integer ranging from 0-100.

Higher values indicate a greater likelihood of maliciousness.

to_dataframe(explode_rules=False, drop_links=False)

Render this object as a Pandas DataFrame.

Parameters
  • explode_rules – Whether to create a row for each rule using pandas.DataFrame.explode (optional, defaults to False)

  • drop_links – Whether to include links when present in exploded rules (optional, defaults to False)

Return type

pandas.DataFrame

Attack Surface Intelligence

RiskIQ Illuminate offers Attack Surface Intelligence (ASI) that delivers prioritized insights on an organization’s attack surface, including impact assets (observations).

ASI is available to licensed users of the RiskIQ Illuminate API.

The analyzer module provides an easy-to-use overlay to interact with the Attack Surface API endpoints and quickly obtain a list of impacted hosts.

Review the Getting Started guide for details on setting up your credentials and development environment.

Your Attack Surface

An essential use case for the RiskIQ Illuminate ASI API is to understand your own organization’s attack surface.

>>> from passivetotal import analyzer
>>> analyzer.init()
>>> my_asi = analyzer.AttackSurface()
>>> my_asi
<AttackSurface #99901 "RiskIQ, Inc.">
>>> my_asi.name
'RiskIQ, Inc.'
>>> my_asi.high_priority_observation_count
0

For a complete reference of the properties available in an AttackSurface object, see passivetotal.analyzer.illuminate.asi.AttackSurface

Attack Surfaces contain a list of insights organized by priority. Insights are included in the response even if there are no impacted assets, but it is easy to filter the list to focus on only the insights with “observations”.

>>> for insight in my_asi.medium_priority_insights:
        print(insight.name, insight.observation_count)
ASI: CVE-2021-123 Potential vulnerability 0
...
ASI: Multiple vulnerabilities in System X 1
>>> for insight in my_asi.medium_priority_insights.only_active_insights:
        print(insight.name)
ASI: Multiple vulnerabilities in System Q

Insight lists are of type passivetotal.analyzer.illuminate.asi.AttackSurfaceInsights and contain a list of passivetotal.analyzer.illuminate.asi.AttackSurfaceInsight objects. Each of these objects provide properties to filter, sort, and render a list of insights. See the API reference below or click the class references here to see other options.

You can obtain the entire list of insights across all three priority levels (high, medium, and low) at once. Use the passivetotal.analyzer.illuminate.asi.AttackSurface.all_insights or passivetotal.analyzer.illuminate.asi.AttackSurface.all_active_insights properties to get the complete list.

Attack Surface Observations

Attack Surface Observations are assets related to a given insight within the context of a specific Attack Surface. Assets typically include IPs and hostnames, and include the “first seen” and “last seen” dates that describe when RiskIQ detected an indication the asset was potentially impacted.

Observations are available in the observations property of an AttackSurfaceInsight.

>>> insights = my_asi.medium_priority_insights.only_active_insights
>>> first_insight = insights[0]
>>> for observation in first_insight.observations:
        print(observation.type)
        print(observation.name)
HOST
subdomain.passivetotal.org

Notice we filtered the list to only include active insights. If you skip this step, be prepared to catch AnalyzerAPIError exceptions thrown by the API when there are no observations available for a given insight.

The list of observations is returned as a passivetotal.analyzer.illuminate.asi.AttackSurfaceObservations object that includes a list of passivetotal.analyzer.illuminate.asi.AttackSurfaceObservation objects. Like nearly all analyzer objects, these objects can be easily rendered as a Python dictionary for integration with other systems using the as_dict property.

>>> insights = my_asi.medium_priority_insights.only_active_insights
>>> first_insight = insights[0]
>>> observations = first_insight.observations.as_dict
>>> observations
{'records': [{'type': 'HOST', 'name': 'subdomain.passivetotal.org', 'firstseen': '2021-02-03 04:05:06', 'lastseen': '2021-06-07 08:09:10'}]}

Third-Party Attack Surfaces

The Third-Party ASI module of Illuminate provides access to the Attack Surfaces of other organizations (aka “vendors”). Your API credentials must be specifically licensed to access third-party Attack Surfaces.

To obtain a list of Attack Surfaces, use the load() method of the passivetotal.analyzer.illuminate.asi.AttackSurfaces class.

>>> vendor_attack_surfaces = analyzer.illuminate.AttackSurfaces.load()
>>> for vendor_asi in vendor_attack_surfaces:
        print(vendor_asi.name, vendor_asi.high_priority_observation_count)
Example Vendor, Inc 13
SaaS Provider 0
Solutions Systems 9

Or, if you already know the RiskIQ ID of the third-party vendor you want to load, pass it as a parameter to the top-level analyzer.AttackSurface() method we used to load our own attack surface.

>>> vendor_asi = analyzer.AttackSurface('12345') # load by ID
>>> print(vendor_asi.name)
Example Vendor, Inc.

You can also load an attack surface by name, if you use a string that is precise enough to find exactly one vendor.

>>> vendor_asi = analyzer.AttackSurface('ample') # load by name match
>>> print(vendor_asi.name)
Example Vendor, Inc.

This will load the entire list of attack surfaces before searching - use it sparingly and primarily in interactive use. Automated processes should load vendor attack surfaces by ID whenever possible.

The passivetotal.analyzer.illuminate.asi.AttackSurface objects returned in this list provide the same functionality as the objects described above that represent your own attack surface. Use the same techniques to enumerate the insights and observations (assets) for a vendor ASI.

>>> vendor_attack_surfaces = analyzer.illuminate.AttackSurfaces.load()
>>> for vendor_asi in vendor_attack_surfaces:
        if vendor_asi.high_priority_observation_count > 0:
            print(vendor_asi.name)
            for insight in vendor_asi.high_priority_insights.only_active_insights:
                print('--- {0.name}'.format(insight))
Example Vendor, Inc
--- ASI: CVE 123
--- ASI: CVE 445
SaaS Provider
--- [Potential] Expired items
Solutions Systems
--- Deprecated Technologies

Examples & Notebooks

Jupyter Notebook

ASI Reference

passivetotal.analyzer.illuminate.asi.AttackSurfaces

Collection of RiskIQ Illuminate Attack Surfaces in a list-like object containing AttackSurface objects.

passivetotal.analyzer.illuminate.asi.AttackSurface

RiskIQ Illuminate Attack Surface Intelligence.

passivetotal.analyzer.illuminate.asi.AttackSurfaceInsights

Collection of insights associated with an attack surface in a list-like object containing AttackSurfaceInsight objects.

passivetotal.analyzer.illuminate.asi.AttackSurfaceInsight

An insight associated with an attack surface.

passivetotal.analyzer.illuminate.asi.AttackSurfaceObservations

Collection of observations (assets) associated with an attack surface insight in a list-like object containing AttackSurfaceObservation objects.

passivetotal.analyzer.illuminate.asi.AttackSurfaceObservation

An attack surface asset (typically identified by an IP address or hostname) that is impacted by an observation.

class passivetotal.analyzer.illuminate.asi.AttackSurfaces(pagesize=400)

Collection of RiskIQ Illuminate Attack Surfaces in a list-like object containing AttackSurface objects.

Primarily used for enumerating a set of third-party vendors.

Public Data Attributes:

Inherited from RecordList

all

All the records as a list.

as_dict

Return the recordlist as a list of dictionary objects.

length

Inherited from AsDictionary

as_dict

Return the recordlist as a list of dictionary objects.

pretty

Pretty printed version of this object's dictionary representation.

Inherited from PagedRecordList

totalrecords

Total number of available records as reported by the API.

has_more_records

Whether more records are available.

Inherited from ForPandas

as_df

Get this object as a Pandas DataFrame.

Public Methods:

__init__([pagesize])

__getitem__(key)

load([pagesize])

Get a list of all third-party (vendor) attack surfaces authorized for this API account.

Inherited from RecordList

__init__([pagesize])

__iter__()

__getitem__(key)

__len__()

parse(api_response)

Implementations must accept an API response and populate themselves with a list of the correct record types.

filter(**kwargs)

Shortcut for filter_and.

filter_fn(fn)

Return only records where a function returns true.

filter_and(**kwargs)

Return only records that match all key/value arguments.

filter_or(**kwargs)

Return only records that match any key/value arguments.

filter_in(**kwargs)

Return only records where a field contains one or more values.

filter_substring(**kwargs)

Return only records where a case-insensitive match on the field returns true.

filter_substring_in(**kwargs)

Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.

sorted_by(field[, reverse])

Return a sorted list.

filter_dateseen_after(date_string)

Filter only results where the firstseen date property is after a specified date.

filter_dateseen_before(date_string)

Filter only results where the lastseen date property is before a specified date.

filter_dateseen_between(start_date_string, ...)

Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.

Inherited from PagedRecordList

load_next_page()

Load the next page of results from the API.

load_all_pages()

Load all pages of results from the API.

Inherited from ForPandas

to_dataframe(**kwargs)

Render this object as a Pandas DataFrame.


property all

All the records as a list.

property as_df

Get this object as a Pandas DataFrame.

Use to_dataframe() instead if you need to control how the dataframe is built.

Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype: pandas.DataFrame

property as_dict

Return the recordlist as a list of dictionary objects.

filter(**kwargs)

Shortcut for filter_and.

filter_and(**kwargs)

Return only records that match all key/value arguments.

filter_dateseen_after(date_string: str)

Filter only results where the firstseen date property is after a specified date.

The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’

filter_dateseen_before(date_string: str)

Filter only results where the lastseen date property is before a specified date.

The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’

filter_dateseen_between(start_date_string: str, end_date_string: str)

Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.

Date strings should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’

filter_fn(fn)

Return only records where a function returns true.

filter_in(**kwargs)

Return only records where a field contains one or more values.

Usage:

filter_in(fieldname=[‘value1’,’value2’]) or filter_in(fieldname=’value1,value2)

filter_or(**kwargs)

Return only records that match any key/value arguments.

filter_substring(**kwargs)

Return only records where a case-insensitive match on the field returns true.

filter_substring_in(**kwargs)

Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.

Usage:

filter_substring_in(fieldname=[‘substring’,’othersub’]) or filter_substring_in(fieldname=’substring,othersub’)

property has_more_records

Whether more records are available.

Return type

bool

static load(pagesize=400)

Get a list of all third-party (vendor) attack surfaces authorized for this API account.

Parameters

pagesize – Size of pages to retrieve from the API (optional).

Return type

passivetotal.analyzer.illuminate.asi.AttackSurfaces.

load_all_pages()

Load all pages of results from the API.

load_next_page()

Load the next page of results from the API.

Throws AnalyzerError when has_more_records is False.

parse(api_response)

Implementations must accept an API response and populate themselves with a list of the correct record types.

property pretty

Pretty printed version of this object’s dictionary representation.

sorted_by(field, reverse=False)

Return a sorted list.

Parameters
  • field – name of the attribute to sort on

  • reverse – whether to sort in reverse order.

to_dataframe(**kwargs)

Render this object as a Pandas DataFrame.

Implementations may add additional keywords to customize building the data structure.

Default implementation tries to iterate through self and calls to_dataframe on each record with the same parameters passed to this method. If that fails (usually because self isn’t iterable), it uses the as_dict param of self.

Return type

pandas.DataFrame

property totalrecords

Total number of available records as reported by the API.

class passivetotal.analyzer.illuminate.asi.AttackSurface(id=None, api_response=None)

RiskIQ Illuminate Attack Surface Intelligence.

Public Data Attributes:

all_insights

All insights across all levels (high, medium, and low).

all_active_insights

All insights with active observations across all levels.

id

is_own

Whether this attack surface represents the org associated with the API credentials.

is_third_party

Whether this is a third-party attack surface.

name

high_priority_observation_count

Number of high-priority insights with impacted assets (observations).

medium_priority_observation_count

Number of medium-priority insights with impacted assets (observations).

low_priority_observation_count

Number of low-priority insights with impacted assets (observations).

high_priority_insights

Get high priority insights.

medium_priority_insights

Get medium priority insights.

low_priority_insights

Get low priority insights.

cves

Get a list of CVEs associated with this attack surface.

components

List of components (detections) vulnerable to this CVE in this attack surface.

Inherited from AsDictionary

as_dict

Return a dictionary representation of the object.

pretty

Pretty printed version of this object's dictionary representation.

Inherited from ForPandas

as_df

Get this object as a Pandas DataFrame.

Public Methods:

__new__(cls[, id, api_response])

__str__()

Return str(self).

__repr__()

Return repr(self).

load([id])

find([id_or_name])

Find one attack surface.

get_insights(level)

Get insights at a level (high, medium or low).

get_observation_count(level)

Get number of insights with impacted assets (observations) at a given level.

get_cves([pagesize])

Get a list of CVEs impacting assets in this attack surface.

get_components([pagesize])

Get a list of vulnerable components (detections) in this attack surface.

to_dataframe()

Render this object as a Pandas DataFrame.

Inherited from Record

match_all(**kwargs)

Whether attributes of this record match all the key/value arguments.

match_any(**kwargs)

Whether attributes of this record match any of the key/value arguments.

Inherited from ForPandas

to_dataframe()

Render this object as a Pandas DataFrame.


property all_active_insights

All insights with active observations across all levels.

Return type

passivetotal.analyzer.illuminate.asi.AttackSurfaceInsights

property all_insights

All insights across all levels (high, medium, and low).

Return type

passivetotal.analyzer.illuminate.asi.AttackSurfaceInsights

property as_df

Get this object as a Pandas DataFrame.

Use to_dataframe() instead if you need to control how the dataframe is built.

Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype: pandas.DataFrame

property as_dict

Return a dictionary representation of the object.

property components

List of components (detections) vulnerable to this CVE in this attack surface.

Return type

passivetotal.analyzer.illuminate.vuln.AttackSurfaceComponents

property cves

Get a list of CVEs associated with this attack surface.

Return type

passivetotal.analyzer.illuminate.vuln.AttackSurfaceCVEs

static find(id_or_name=None)

Find one attack surface.

Call with no parameters to find your Attack Surface. Pass a number to load a specific Attack Surface by ID, or pass a string to load the entire list of Attack Surfaces and search them by case-insensitive substring.

Raises AnalyzerError if no attack surfaces are found or if more than one Attack Surface name matches the parameter.

Returns

passivetotal.illuminate.asi.AttackSurface

get_components(pagesize=400)

Get a list of vulnerable components (detections) in this attack surface.

Parameters

pagesize – Size of pages to retrieve from the API.

Return type

passivetotal.analyzer.illuminate.vuln.AttackSurfaceComponents

get_cves(pagesize=400)

Get a list of CVEs impacting assets in this attack surface.

Parameters

pagesize – Size of pages to retrieve from the API.

Return type

passivetotal.analyzer.illuminate.vuln.AttackSurfaceCVEs

get_insights(level)

Get insights at a level (high, medium or low).

Parameters

level – Priority level (high, medium, or low).

Returns

AttackSurfaceInsights

get_observation_count(level)

Get number of insights with impacted assets (observations) at a given level.

Parameters

level – Priority level (high, medium, or low).

Return type

int Count of insights with observations

property high_priority_insights

Get high priority insights.

Return type

passivetotal.analyzer.illuminate.asi.AttackSurfaceInsights

property high_priority_observation_count

Number of high-priority insights with impacted assets (observations).

property is_own

Whether this attack surface represents the org associated with the API credentials.

property is_third_party

Whether this is a third-party attack surface.

property low_priority_insights

Get low priority insights.

Return type

passivetotal.analyzer.illuminate.asi.AttackSurfaceInsights

property low_priority_observation_count

Number of low-priority insights with impacted assets (observations).

match_all(**kwargs)

Whether attributes of this record match all the key/value arguments.

match_any(**kwargs)

Whether attributes of this record match any of the key/value arguments.

property medium_priority_insights

Get medium priority insights.

Return type

passivetotal.analyzer.illuminate.asi.AttackSurfaceInsights

property medium_priority_observation_count

Number of medium-priority insights with impacted assets (observations).

property pretty

Pretty printed version of this object’s dictionary representation.

to_dataframe()

Render this object as a Pandas DataFrame.

Return type

pandas.DataFrame

class passivetotal.analyzer.illuminate.asi.AttackSurfaceInsights(attack_surface=None, level=None, api_response=None)

Collection of insights associated with an attack surface in a list-like object containing AttackSurfaceInsight objects.

__init__(attack_surface=None, level=None, api_response=None)

Public Data Attributes:

attack_surface

Attack surface these insights are associated with.

level

active_insight_count

only_active_insights

Filter to only active insights (insights with active observations).

total_insight_count

total_observations

Inherited from RecordList

all

All the records as a list.

as_dict

Return the recordlist as a list of dictionary objects.

length

Inherited from AsDictionary

as_dict

Return the recordlist as a list of dictionary objects.

pretty

Pretty printed version of this object's dictionary representation.

Inherited from ForPandas

as_df

Get this object as a Pandas DataFrame.

Public Methods:

__init__([attack_surface, level, api_response])

load(attack_surface, level)

Inherited from RecordList

__init__([attack_surface, level, api_response])

__iter__()

__getitem__(key)

__len__()

parse(api_response)

Implementations must accept an API response and populate themselves with a list of the correct record types.

filter(**kwargs)

Shortcut for filter_and.

filter_fn(fn)

Return only records where a function returns true.

filter_and(**kwargs)

Return only records that match all key/value arguments.

filter_or(**kwargs)

Return only records that match any key/value arguments.

filter_in(**kwargs)

Return only records where a field contains one or more values.

filter_substring(**kwargs)

Return only records where a case-insensitive match on the field returns true.

filter_substring_in(**kwargs)

Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.

sorted_by(field[, reverse])

Return a sorted list.

filter_dateseen_after(date_string)

Filter only results where the firstseen date property is after a specified date.

filter_dateseen_before(date_string)

Filter only results where the lastseen date property is before a specified date.

filter_dateseen_between(start_date_string, ...)

Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.

Inherited from ForPandas

to_dataframe(**kwargs)

Render this object as a Pandas DataFrame.


property all

All the records as a list.

property as_df

Get this object as a Pandas DataFrame.

Use to_dataframe() instead if you need to control how the dataframe is built.

Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype: pandas.DataFrame

property as_dict

Return the recordlist as a list of dictionary objects.

property attack_surface

Attack surface these insights are associated with.

Return type

passivetotal.analyzer.illuminate.asi.AttackSurface

filter(**kwargs)

Shortcut for filter_and.

filter_and(**kwargs)

Return only records that match all key/value arguments.

filter_dateseen_after(date_string: str)

Filter only results where the firstseen date property is after a specified date.

The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’

filter_dateseen_before(date_string: str)

Filter only results where the lastseen date property is before a specified date.

The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’

filter_dateseen_between(start_date_string: str, end_date_string: str)

Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.

Date strings should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’

filter_fn(fn)

Return only records where a function returns true.

filter_in(**kwargs)

Return only records where a field contains one or more values.

Usage:

filter_in(fieldname=[‘value1’,’value2’]) or filter_in(fieldname=’value1,value2)

filter_or(**kwargs)

Return only records that match any key/value arguments.

filter_substring(**kwargs)

Return only records where a case-insensitive match on the field returns true.

filter_substring_in(**kwargs)

Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.

Usage:

filter_substring_in(fieldname=[‘substring’,’othersub’]) or filter_substring_in(fieldname=’substring,othersub’)

property only_active_insights

Filter to only active insights (insights with active observations).

Return type

AttackSurfaceInsights

parse(api_response)

Implementations must accept an API response and populate themselves with a list of the correct record types.

property pretty

Pretty printed version of this object’s dictionary representation.

sorted_by(field, reverse=False)

Return a sorted list.

Parameters
  • field – name of the attribute to sort on

  • reverse – whether to sort in reverse order.

to_dataframe(**kwargs)

Render this object as a Pandas DataFrame.

Implementations may add additional keywords to customize building the data structure.

Default implementation tries to iterate through self and calls to_dataframe on each record with the same parameters passed to this method. If that fails (usually because self isn’t iterable), it uses the as_dict param of self.

Return type

pandas.DataFrame

class passivetotal.analyzer.illuminate.asi.AttackSurfaceInsight(attack_surface, level, api_response)

An insight associated with an attack surface.

Public Data Attributes:

id

Unique ID for the insight.

level

Priority level of this insight.

attack_surface

Attack surface this insight is associated with.

name

Insight short name.

description

Insight long description.

link

API link to get observations.

observation_count

Number of observations (assets) impacted by this insight.

has_observations

Whether this insight has any active observations (assets).

observations

List of impacted assets.

Inherited from AsDictionary

as_dict

Return a dictionary representation of the object.

pretty

Pretty printed version of this object's dictionary representation.

Inherited from ForPandas

as_df

Get this object as a Pandas DataFrame.

Public Methods:

__init__(attack_surface, level, api_response)

__str__()

Return str(self).

to_dataframe()

Render this insight as a Pandas DataFrame.

get_observations([pagesize])

Get a list of impacted assets (observations).

Inherited from Record

match_all(**kwargs)

Whether attributes of this record match all the key/value arguments.

match_any(**kwargs)

Whether attributes of this record match any of the key/value arguments.

Inherited from ForPandas

to_dataframe()

Render this insight as a Pandas DataFrame.


property as_df

Get this object as a Pandas DataFrame.

Use to_dataframe() instead if you need to control how the dataframe is built.

Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype: pandas.DataFrame

property as_dict

Return a dictionary representation of the object.

property attack_surface

Attack surface this insight is associated with.

Return type

passivetotal.analyzer.illuminate.asi.AttackSurface

property description

Insight long description.

get_observations(pagesize=400)

Get a list of impacted assets (observations).

Parameters

pagesize – Size of pages to retrieve from the API.

Return type

AttackSurfaceObservations

property has_observations

Whether this insight has any active observations (assets).

property id

Unique ID for the insight.

property level

Priority level of this insight.

API link to get observations.

match_all(**kwargs)

Whether attributes of this record match all the key/value arguments.

match_any(**kwargs)

Whether attributes of this record match any of the key/value arguments.

property name

Insight short name.

property observation_count

Number of observations (assets) impacted by this insight.

property observations

List of impacted assets.

Return type

AttackSurfaceObservations

property pretty

Pretty printed version of this object’s dictionary representation.

to_dataframe()

Render this insight as a Pandas DataFrame.

Return type

pandas.DataFrame

class passivetotal.analyzer.illuminate.asi.AttackSurfaceObservations(insight, group_by, segment_by, pagesize=400)

Collection of observations (assets) associated with an attack surface insight in a list-like object containing AttackSurfaceObservation objects.

Public Data Attributes:

asset_types

List of unique asset types in this observation list.

hostnames

List of unique hostnames in this observation list.

ips

List of unique IP addresses in this observation list.

Inherited from RecordList

all

All the records as a list.

as_dict

Return the recordlist as a list of dictionary objects.

length

Inherited from AsDictionary

as_dict

Return the recordlist as a list of dictionary objects.

pretty

Pretty printed version of this object's dictionary representation.

Inherited from PagedRecordList

totalrecords

Total number of available records as reported by the API.

has_more_records

Whether more records are available.

Inherited from ForPandas

as_df

Get this object as a Pandas DataFrame.

Public Methods:

__init__(insight, group_by, segment_by[, ...])

Inherited from RecordList

__init__(insight, group_by, segment_by[, ...])

__iter__()

__getitem__(key)

__len__()

parse(api_response)

Implementations must accept an API response and populate themselves with a list of the correct record types.

filter(**kwargs)

Shortcut for filter_and.

filter_fn(fn)

Return only records where a function returns true.

filter_and(**kwargs)

Return only records that match all key/value arguments.

filter_or(**kwargs)

Return only records that match any key/value arguments.

filter_in(**kwargs)

Return only records where a field contains one or more values.

filter_substring(**kwargs)

Return only records where a case-insensitive match on the field returns true.

filter_substring_in(**kwargs)

Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.

sorted_by(field[, reverse])

Return a sorted list.

filter_dateseen_after(date_string)

Filter only results where the firstseen date property is after a specified date.

filter_dateseen_before(date_string)

Filter only results where the lastseen date property is before a specified date.

filter_dateseen_between(start_date_string, ...)

Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.

Inherited from PagedRecordList

load_next_page()

Load the next page of results from the API.

load_all_pages()

Load all pages of results from the API.

Inherited from ForPandas

to_dataframe(**kwargs)

Render this object as a Pandas DataFrame.


property all

All the records as a list.

property as_df

Get this object as a Pandas DataFrame.

Use to_dataframe() instead if you need to control how the dataframe is built.

Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype: pandas.DataFrame

property as_dict

Return the recordlist as a list of dictionary objects.

property asset_types

List of unique asset types in this observation list.

filter(**kwargs)

Shortcut for filter_and.

filter_and(**kwargs)

Return only records that match all key/value arguments.

filter_dateseen_after(date_string: str)

Filter only results where the firstseen date property is after a specified date.

The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’

filter_dateseen_before(date_string: str)

Filter only results where the lastseen date property is before a specified date.

The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’

filter_dateseen_between(start_date_string: str, end_date_string: str)

Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.

Date strings should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’

filter_fn(fn)

Return only records where a function returns true.

filter_in(**kwargs)

Return only records where a field contains one or more values.

Usage:

filter_in(fieldname=[‘value1’,’value2’]) or filter_in(fieldname=’value1,value2)

filter_or(**kwargs)

Return only records that match any key/value arguments.

filter_substring(**kwargs)

Return only records where a case-insensitive match on the field returns true.

filter_substring_in(**kwargs)

Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.

Usage:

filter_substring_in(fieldname=[‘substring’,’othersub’]) or filter_substring_in(fieldname=’substring,othersub’)

property has_more_records

Whether more records are available.

Return type

bool

property hostnames

List of unique hostnames in this observation list.

Return type

passivetotal.analyzer.Hostname

property ips

List of unique IP addresses in this observation list.

Return type

passivetotal.analyzer.IPAddress

load_all_pages()

Load all pages of results from the API.

load_next_page()

Load the next page of results from the API.

Throws AnalyzerError when has_more_records is False.

parse(api_response)

Implementations must accept an API response and populate themselves with a list of the correct record types.

property pretty

Pretty printed version of this object’s dictionary representation.

sorted_by(field, reverse=False)

Return a sorted list.

Parameters
  • field – name of the attribute to sort on

  • reverse – whether to sort in reverse order.

to_dataframe(**kwargs)

Render this object as a Pandas DataFrame.

Implementations may add additional keywords to customize building the data structure.

Default implementation tries to iterate through self and calls to_dataframe on each record with the same parameters passed to this method. If that fails (usually because self isn’t iterable), it uses the as_dict param of self.

Return type

pandas.DataFrame

property totalrecords

Total number of available records as reported by the API.

class passivetotal.analyzer.illuminate.asi.AttackSurfaceObservation(insight, api_response)

An attack surface asset (typically identified by an IP address or hostname) that is impacted by an observation.

Public Data Attributes:

type

Type of impacted asset (i.e.

name

Name of impacted asset (i.e.

insight

Attack surface insight associated with this observation.

hostname

Shortcut property to access an analyzer.Hostname object from this observation, if the observation type is HOST, else returns None.

ip

Shortcut property to access an analyzer.IPAddress object from this observation, if the observation type is IP_ADDRESS, else returns None.

Inherited from AsDictionary

as_dict

Return a dictionary representation of the object.

pretty

Pretty printed version of this object's dictionary representation.

Inherited from FirstLastSeen

firstseen

Date & time the record was first seen.

firstseen_date

Date record was first seen.

firstseen_raw

Raw firstseen value returned by the API.

lastseen

Date & time the record was most recently observed.

lastseen_date

Date the record was most recently observed.

lastseen_raw

Raw lastseen value returned by the API.

duration

Length of time record was observed, in days.

Inherited from ForPandas

as_df

Get this object as a Pandas DataFrame.

Public Methods:

__init__(insight, api_response)

__str__()

Return str(self).

__repr__()

Return repr(self).

to_dataframe()

Render this object as a Pandas dataframe.

Inherited from Record

match_all(**kwargs)

Whether attributes of this record match all the key/value arguments.

match_any(**kwargs)

Whether attributes of this record match any of the key/value arguments.

Inherited from ForPandas

to_dataframe()

Render this object as a Pandas dataframe.


property as_df

Get this object as a Pandas DataFrame.

Use to_dataframe() instead if you need to control how the dataframe is built.

Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype: pandas.DataFrame

property as_dict

Return a dictionary representation of the object.

property duration

Length of time record was observed, in days.

Calculates the timedelta between firstseen and lastseen.

Return type

int

property firstseen

Date & time the record was first seen.

Return type

datetime

property firstseen_date

Date record was first seen.

Return type

date

property firstseen_raw

Raw firstseen value returned by the API.

property hostname

Shortcut property to access an analyzer.Hostname object from this observation, if the observation type is HOST, else returns None.

Return type

passivetotal.analyzer.Hostname

property insight

Attack surface insight associated with this observation.

Return type

AttackSurfaceInsight

property ip

Shortcut property to access an analyzer.IPAddress object from this observation, if the observation type is IP_ADDRESS, else returns None.

Return type

passivetotal.analyzer.IPAddress

property lastseen

Date & time the record was most recently observed.

Return type

datetime

property lastseen_date

Date the record was most recently observed.

Return type

date

property lastseen_raw

Raw lastseen value returned by the API.

match_all(**kwargs)

Whether attributes of this record match all the key/value arguments.

match_any(**kwargs)

Whether attributes of this record match any of the key/value arguments.

property name

Name of impacted asset (i.e. the IP address or hostname).

property pretty

Pretty printed version of this object’s dictionary representation.

to_dataframe()

Render this object as a Pandas dataframe.

Return type

pandas.DataFrame

property type

Type of impacted asset (i.e. HOST or IP_ADDRESS).

Intelligence Profiles (CTI)

The RiskIQ Illuminate platform offers a Cyber Threat Intelligence (CTI) module that delivers insights into adversary threat infrastructure organized around a set of purpose-built intelligence profiles, each with a curated set of indicators.

The analyzer module provides the optimal interface to query the CTI dataset, but low-level request wrappers are also available. See below for the API reference docs.

Review the Getting Started guide for details on setting up your credentials and development environment.

Important: this module must be specifically enabled for your API credentials.

Intel Profiles

For most use cases, start with the list of RiskIQ Illuminate intel profiles. These are returned as a passivetotal.analyzer.illuminate.cti.IntelProfiles type that can be iterated and filtered like a regular list.

>>> from passivetotal import analyzer
>>> analyzer.init()
>>> intel_profiles = analyzer.illuminate.IntelProfiles.load()
>>> for profile in intel_profiles:
...     print(profile.id, profile.title, profile.indicatorcount_riskiq)

Each record in the list is of type passivetotal.analyzer.illuminate.cti.IntelProfile

Intel Profiles are identified with a unique string in the id parameter. Once you know the profile you want to focus on, you can instantiate it directly using that id.

>>> profile = analyzer.illuminate.IntelProfile('apt33')
>>> print(profile.pretty)
{ 'id': 'apt33',
  'indicatorcount_osint': 33333,
  'indicatorcount_riskiq': 55555,
  'tags_raw': [ {'countryCode': None, 'label': 'Malicious'},
                {'countryCode': 'us', 'label': 'Target: USA'}],
  'title': 'APT33'}

Indicator Lists

The RiskIQ research team curates lists of indicators associated with each intel profile, some sourced from open-source intelligence, and others surfaced directly from RiskIQ proprietary datasets.

There are several ways to obtain the list of indicators associated with a specific intel profile. Each method will return an object of type passivetotal.analyzer.illuminate.cti.IntelProfileIndicator that can be iterated like a standard list and also offers several built-in methods and properties to filter, sort, and render the list.

Obtain the list of indicators directly as a property of an passivetotal.analyzer.illuminate.cti.IntelProfile:

>>> profile = analyzer.illuminate.IntelProfile('apt33')
>>> for indicator in profile.indicators:
...    print(indicator.pretty)

Each indicator is of type passivetotal.analyzer.illuminate.cti.IntelProfileIndicator

For more granular control, call the get_indicators() method and set additional parameters supported by the API to narrow the list:

>>> profile = analyzer.illuminate.IntelProfile('apt33')
>>> indicators = profile.get_indicators(sources='riskiq')
>>> len(indicators)
55555

Or, skip the intel profile entirely and go straight to the indicator list.

>>> ioc_list = analyzer.illuminate.IntelProfileIndicatorList('apt33')
>>> ioc_list.load_all_pages()
>>> len(ioc_list)
55555

The underlying API calls require pagination, but the analyzer module handles that automatically when you access either the indicators property or call get_indicators() directly on the profile objects. Here, we are using the load_all_pages() method to populate the list directly.

If you only need the values of the indicators, you can quickly access them as a plain Python list with the values property:

>>> profile = analyzer.illuminate.IntelProfile('apt33')
>>> indicators = profile.get_indicators(sources='riskiq')
>>> indicators.values
['.....']
>>> indicators.only_riskiq.values
['.....']
>>> indicators.filter_in(type='domain,ip').values
['.....']

Search By Indicator

The CTI API provides a mechanism to search for intel profiles by the value of an indicator. You can query the API directly for any of the indicator types stored in the RiskIQ dataset, or you can access the list as a property of analyzer.Hostname or analyzer.IPAddress objects.

To search the API directly, use the passivetotal.analyzer.illuminate.cti.IntelProfiles.find_by_indicator() static method:

>>> results = analyzer.illuminate.IntelProfiles.find_by_indicator('threat_actor@gmail.com')
>>> len(results)
1
>>> print(results[0].title)
APT123
>>> for ioc in results[0].indicators:
...    print(ioc)

Or, if you are working with a hostname or IP address, access the intel_profiles property to obtain the profile list. Be sure to test the length of the list before accessing properties to avoid runtime exceptions.

>>> if len(analyzer.IPAddress('123.123.123.123').intel_profiles) > 0:
...     print(intel_profiles.pretty)

Examples & Notebooks

Jupyter Notebook

CTI Reference

passivetotal.analyzer.illuminate.cti.IntelProfiles

List of RiskIQ Intel Profiles from the Illuminate CTI module.

passivetotal.analyzer.illuminate.cti.IntelProfile

RiskIQ Intel Profile on a specific actor group.

passivetotal.analyzer.illuminate.cti.IntelProfileIndicatorList

passivetotal.analyzer.illuminate.cti.IntelProfileIndicator

An indicator associated with an intel profile.

class passivetotal.analyzer.illuminate.cti.IntelProfiles(api_response=None, query=None)

List of RiskIQ Intel Profiles from the Illuminate CTI module.

Public Data Attributes:

totalrecords

Total number of profiles available in this record list.

Inherited from RecordList

all

All the records as a list.

as_dict

Return the recordlist as a list of dictionary objects.

length

Inherited from AsDictionary

as_dict

Return the recordlist as a list of dictionary objects.

pretty

Pretty printed version of this object's dictionary representation.

Inherited from ForPandas

as_df

Get this object as a Pandas DataFrame.

Public Methods:

__getitem__(key)

load([query, profile_type])

Get a list of all RiskIQ Intel Profiles.

find_by_indicator(query, **kwargs)

Search profiles by indicator.

parse(api_response)

Parse an API response.

to_dataframe([ignore_index])

Render this object as a Pandas dataframe.

Inherited from RecordList

__init__([api_response, query])

__iter__()

__getitem__(key)

__len__()

parse(api_response)

Parse an API response.

filter(**kwargs)

Shortcut for filter_and.

filter_fn(fn)

Return only records where a function returns true.

filter_and(**kwargs)

Return only records that match all key/value arguments.

filter_or(**kwargs)

Return only records that match any key/value arguments.

filter_in(**kwargs)

Return only records where a field contains one or more values.

filter_substring(**kwargs)

Return only records where a case-insensitive match on the field returns true.

filter_substring_in(**kwargs)

Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.

sorted_by(field[, reverse])

Return a sorted list.

filter_dateseen_after(date_string)

Filter only results where the firstseen date property is after a specified date.

filter_dateseen_before(date_string)

Filter only results where the lastseen date property is before a specified date.

filter_dateseen_between(start_date_string, ...)

Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.

Inherited from ForPandas

to_dataframe([ignore_index])

Render this object as a Pandas dataframe.


property all

All the records as a list.

property as_df

Get this object as a Pandas DataFrame.

Use to_dataframe() instead if you need to control how the dataframe is built.

Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype: pandas.DataFrame

property as_dict

Return the recordlist as a list of dictionary objects.

filter(**kwargs)

Shortcut for filter_and.

filter_and(**kwargs)

Return only records that match all key/value arguments.

filter_dateseen_after(date_string: str)

Filter only results where the firstseen date property is after a specified date.

The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’

filter_dateseen_before(date_string: str)

Filter only results where the lastseen date property is before a specified date.

The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’

filter_dateseen_between(start_date_string: str, end_date_string: str)

Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.

Date strings should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’

filter_fn(fn)

Return only records where a function returns true.

filter_in(**kwargs)

Return only records where a field contains one or more values.

Usage:

filter_in(fieldname=[‘value1’,’value2’]) or filter_in(fieldname=’value1,value2)

filter_or(**kwargs)

Return only records that match any key/value arguments.

filter_substring(**kwargs)

Return only records where a case-insensitive match on the field returns true.

filter_substring_in(**kwargs)

Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.

Usage:

filter_substring_in(fieldname=[‘substring’,’othersub’]) or filter_substring_in(fieldname=’substring,othersub’)

static find_by_indicator(query, **kwargs)

Search profiles by indicator.

Parameters
  • query – Indicator value as a string

  • types – Types of indicators (optional)

  • categories – Categories of indicators (optional)

  • sources – Sources of indicators [riskiq, osint] (optional)

static load(query=None, profile_type=None)

Get a list of all RiskIQ Intel Profiles.

Parameters
  • query – Submit a query param to the API to limit results to only matching providers (optional)

  • profile_type – Submit a type param to the API to limit results to only certain profile types (optional)

parse(api_response)

Parse an API response.

property pretty

Pretty printed version of this object’s dictionary representation.

sorted_by(field, reverse=False)

Return a sorted list.

Parameters
  • field – name of the attribute to sort on

  • reverse – whether to sort in reverse order.

to_dataframe(ignore_index=False, **kwargs)

Render this object as a Pandas dataframe.

property totalrecords

Total number of profiles available in this record list.

class passivetotal.analyzer.illuminate.cti.IntelProfile(id=None, api_response=None)

RiskIQ Intel Profile on a specific actor group.

Public Data Attributes:

aliases

List of alternative names for this actor group.

id

RiskIQ identifier for this actor group.

indicatorcount_osint

Count of available indicators from open source intelligence reports.

indicatorcount_riskiq

Count of available indicators sourced from RiskIQ primary research.

indicators

Unfiltered indicator list associated with this intel profile.

tags

List of profile tags associated with this actor group.

tags_raw

List of profile tags as returned by the API.

title

RiskIQ title for this actor profile.

Inherited from AsDictionary

as_dict

Return a dictionary representation of the object.

pretty

Pretty printed version of this object's dictionary representation.

Inherited from ForPandas

as_df

Get this object as a Pandas DataFrame.

Public Methods:

__new__(cls[, id, api_response])

__str__()

Return str(self).

__repr__()

Return repr(self).

load(profile_id)

Load an intel profile by the RiskIQ-assigned identifier string.

get_indicators([all_pages, types, ...])

Get a list of indicators associated with this intel profile.

to_dataframe()

Render this profile as a Pandas DataFrame.

Inherited from Record

match_all(**kwargs)

Whether attributes of this record match all the key/value arguments.

match_any(**kwargs)

Whether attributes of this record match any of the key/value arguments.

Inherited from ForPandas

to_dataframe()

Render this profile as a Pandas DataFrame.


class ProfileTag(label, country)
property country

Alias for field number 1

property label

Alias for field number 0

property aliases

List of alternative names for this actor group.

property as_df

Get this object as a Pandas DataFrame.

Use to_dataframe() instead if you need to control how the dataframe is built.

Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype: pandas.DataFrame

property as_dict

Return a dictionary representation of the object.

get_indicators(all_pages=True, types=None, categories=None, sources=None, pagesize=400)

Get a list of indicators associated with this intel profile.

Loads all pages of indicators by default. Results with identical params are cached.

Parameters
  • all_pages – Whether to retrieve all pages (optional, defaults to True)

  • types – Types of indicators to search for (optional).

  • categories – Categories of indicators to filter on (optional).

  • sources – Sources of indicators [osint, riskiq] (optional).

  • pagesize – Size of pages to return from the API (defaults to INDICATOR_PAGE_SIZE).

Return type

passivetotal.analyzer.illuminate.cti.IntelProfileIndicatorList

property id

RiskIQ identifier for this actor group.

property indicatorcount_osint

Count of available indicators from open source intelligence reports.

property indicatorcount_riskiq

Count of available indicators sourced from RiskIQ primary research.

property indicators

Unfiltered indicator list associated with this intel profile.

Calls passivetotal.analyzer.illuminate.IntelProfile.get_indicators() with default parameters. Use that method directly for more granular control.

Return type

passivetotal.analyzer.illuminate.cti.IntelProfileIndicatorList

static load(profile_id)

Load an intel profile by the RiskIQ-assigned identifier string.

Return type

passivetotal.analyzer.illuminate.cti.IntelProfile

match_all(**kwargs)

Whether attributes of this record match all the key/value arguments.

match_any(**kwargs)

Whether attributes of this record match any of the key/value arguments.

property pretty

Pretty printed version of this object’s dictionary representation.

property tags

List of profile tags associated with this actor group.

Return type

passivetotal.analyzer.illuminate.cti.IntelProfile.ProfileTag

property tags_raw

List of profile tags as returned by the API.

property title

RiskIQ title for this actor profile.

to_dataframe()

Render this profile as a Pandas DataFrame.

Return type

pandas.DataFrame

class passivetotal.analyzer.illuminate.cti.IntelProfileIndicatorList(profile_id=None, query=None, types=None, categories=None, sources=None, pagesize=400)
__init__(profile_id=None, query=None, types=None, categories=None, sources=None, pagesize=400)

Collection of indicators associated with a RiskIQ Intel Profile as a list-like object containing IntelProfileIndicator objects.

Parameters
  • profile_id – Threat intel profile ID to search for.

  • query – Indicator value to query for.

  • types – Types of indicators to search for (optional).

  • categories – Categories of indicators to filter on (optional).

  • sources – Sources of indicators [osint, riskiq] (optiona).

Public Data Attributes:

only_osint

Filtered list with only indicators from open sources.

only_riskiq

Filtered list with only indicators sourced by RiskIQ.

types

List of indicator types in the list.

values

List of all values in the indicator list.

Inherited from RecordList

all

All the records as a list.

as_dict

Return the recordlist as a list of dictionary objects.

length

Inherited from AsDictionary

as_dict

Return the recordlist as a list of dictionary objects.

pretty

Pretty printed version of this object's dictionary representation.

Inherited from PagedRecordList

totalrecords

Total number of available records as reported by the API.

has_more_records

Whether more records are available.

Inherited from ForPandas

as_df

Get this object as a Pandas DataFrame.

Public Methods:

__init__([profile_id, query, types, ...])

Collection of indicators associated with a RiskIQ Intel Profile as a list-like object containing IntelProfileIndicator objects.

to_dataframe([ignore_index])

Render this object as a Pandas dataframe.

Inherited from RecordList

__init__([profile_id, query, types, ...])

Collection of indicators associated with a RiskIQ Intel Profile as a list-like object containing IntelProfileIndicator objects.

__iter__()

__getitem__(key)

__len__()

parse(api_response)

Implementations must accept an API response and populate themselves with a list of the correct record types.

filter(**kwargs)

Shortcut for filter_and.

filter_fn(fn)

Return only records where a function returns true.

filter_and(**kwargs)

Return only records that match all key/value arguments.

filter_or(**kwargs)

Return only records that match any key/value arguments.

filter_in(**kwargs)

Return only records where a field contains one or more values.

filter_substring(**kwargs)

Return only records where a case-insensitive match on the field returns true.

filter_substring_in(**kwargs)

Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.

sorted_by(field[, reverse])

Return a sorted list.

filter_dateseen_after(date_string)

Filter only results where the firstseen date property is after a specified date.

filter_dateseen_before(date_string)

Filter only results where the lastseen date property is before a specified date.

filter_dateseen_between(start_date_string, ...)

Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.

Inherited from PagedRecordList

load_next_page()

Load the next page of results from the API.

load_all_pages()

Load all pages of results from the API.

Inherited from ForPandas

to_dataframe([ignore_index])

Render this object as a Pandas dataframe.


property all

All the records as a list.

property as_df

Get this object as a Pandas DataFrame.

Use to_dataframe() instead if you need to control how the dataframe is built.

Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype: pandas.DataFrame

property as_dict

Return the recordlist as a list of dictionary objects.

filter(**kwargs)

Shortcut for filter_and.

filter_and(**kwargs)

Return only records that match all key/value arguments.

filter_dateseen_after(date_string: str)

Filter only results where the firstseen date property is after a specified date.

The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’

filter_dateseen_before(date_string: str)

Filter only results where the lastseen date property is before a specified date.

The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’

filter_dateseen_between(start_date_string: str, end_date_string: str)

Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.

Date strings should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’

filter_fn(fn)

Return only records where a function returns true.

filter_in(**kwargs)

Return only records where a field contains one or more values.

Usage:

filter_in(fieldname=[‘value1’,’value2’]) or filter_in(fieldname=’value1,value2)

filter_or(**kwargs)

Return only records that match any key/value arguments.

filter_substring(**kwargs)

Return only records where a case-insensitive match on the field returns true.

filter_substring_in(**kwargs)

Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.

Usage:

filter_substring_in(fieldname=[‘substring’,’othersub’]) or filter_substring_in(fieldname=’substring,othersub’)

property has_more_records

Whether more records are available.

Return type

bool

load_all_pages()

Load all pages of results from the API.

load_next_page()

Load the next page of results from the API.

Throws AnalyzerError when has_more_records is False.

property only_osint

Filtered list with only indicators from open sources.

Return type

passivetotal.analyzer.illuminate.cti.IntelProfileIndicatorList

property only_riskiq

Filtered list with only indicators sourced by RiskIQ.

Return type

passivetotal.analyzer.illuminate.cti.IntelProfileIndicatorList

parse(api_response)

Implementations must accept an API response and populate themselves with a list of the correct record types.

property pretty

Pretty printed version of this object’s dictionary representation.

sorted_by(field, reverse=False)

Return a sorted list.

Parameters
  • field – name of the attribute to sort on

  • reverse – whether to sort in reverse order.

to_dataframe(ignore_index=False, **kwargs)

Render this object as a Pandas dataframe.

property totalrecords

Total number of available records as reported by the API.

property types

List of indicator types in the list.

property values

List of all values in the indicator list.

class passivetotal.analyzer.illuminate.cti.IntelProfileIndicator(api_response)

An indicator associated with an intel profile.

Public Data Attributes:

id

RiskIQ identifier for this indicator.

profile_id

RiskIQ identifier for the intel profile associated with this indicator.

intel_profile

RiskIQ threat intel profile associated with this indicator.

type

Type of indicator.

value

Value of the indicator.

category

Indicator category.

is_osint

Whether this indicator was published in open source intelligence articles.

osint_link

URL for the OSINT source of the indicator, or none if this is not an OSINT indicator.

articleguids

List of RiskIQ OSINT article GUIDs associated with this indicator.

Inherited from AsDictionary

as_dict

Return a dictionary representation of the object.

pretty

Pretty printed version of this object's dictionary representation.

Inherited from FirstLastSeen

firstseen

Date & time the record was first seen.

firstseen_date

Date record was first seen.

firstseen_raw

Raw firstseen value returned by the API.

lastseen

Date & time the record was most recently observed.

lastseen_date

Date the record was most recently observed.

lastseen_raw

Raw lastseen value returned by the API.

duration

Length of time record was observed, in days.

Inherited from ForPandas

as_df

Get this object as a Pandas DataFrame.

Public Methods:

__init__(api_response)

__repr__()

Return repr(self).

__str__()

Return str(self).

to_dataframe()

Render this object as a Pandas DataFrame.

Inherited from Record

match_all(**kwargs)

Whether attributes of this record match all the key/value arguments.

match_any(**kwargs)

Whether attributes of this record match any of the key/value arguments.

Inherited from ForPandas

to_dataframe()

Render this object as a Pandas DataFrame.


property articleguids

List of RiskIQ OSINT article GUIDs associated with this indicator.

property as_df

Get this object as a Pandas DataFrame.

Use to_dataframe() instead if you need to control how the dataframe is built.

Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype: pandas.DataFrame

property as_dict

Return a dictionary representation of the object.

property category

Indicator category.

property duration

Length of time record was observed, in days.

Calculates the timedelta between firstseen and lastseen.

Return type

int

property firstseen

Date & time the record was first seen.

Return type

datetime

property firstseen_date

Date record was first seen.

Return type

date

property firstseen_raw

Raw firstseen value returned by the API.

property id

RiskIQ identifier for this indicator.

property intel_profile

RiskIQ threat intel profile associated with this indicator.

Return type

passivetotal.analyzer.illuminate.cti.IntelProfile

property is_osint

Whether this indicator was published in open source intelligence articles.

property lastseen

Date & time the record was most recently observed.

Return type

datetime

property lastseen_date

Date the record was most recently observed.

Return type

date

property lastseen_raw

Raw lastseen value returned by the API.

match_all(**kwargs)

Whether attributes of this record match all the key/value arguments.

match_any(**kwargs)

Whether attributes of this record match any of the key/value arguments.

URL for the OSINT source of the indicator, or none if this is not an OSINT indicator.

property pretty

Pretty printed version of this object’s dictionary representation.

property profile_id

RiskIQ identifier for the intel profile associated with this indicator.

to_dataframe()

Render this object as a Pandas DataFrame.

Return type

pandas.DataFrame

property type

Type of indicator.

property value

Value of the indicator.

Vulnerability Intelligence

RiskIQ’s Vulnerability Intelligence (Vuln Intel) provides a practical picture of vulnerability risk, focused on a specific Attack Surface (your own or a third-party vendor). It returns a list of “CVEs” (Common Vulnerabilities and Exposures) each identified by a name and offering a list of assets known to be vulnerable to the exploits or weaknesses described in the vuln report.

CVEs for your Attack Surface

In the analyzer module, Vuln Intel is offered primarily through the cves property of an Attack Surface.

>>> cves = analyzer.AttackSurface().cves
>>> for cve in cves:
...    print(cve)

Each record can be printed as a string, but like other analyzer objects, it offers a rich set of properties to display and iterate through the list of CVEs. See the reference for the passivetotal.analyzer.illuminate.vuln.AttackSurfaceCVEs object that represents the list of CVEs, and reference the passivetotal.analyzer.illuminate.vuln.AttackSurfaceCVE object for details on each CVE.

CVEs for Third Parties

Use the same cves property of a third-party vendor attack surface to discover which CVEs they may be vulnerable to. In this example, we load an attack surface for vendr ID “12345”.

>>> cves = analyzer.AttackSurface(12345).cves
>>> for cve in cves:
...    print(cve)

CVE Observations (Assets)

Each AttackSurfaceCVE object offers an observations property that delivers a list of assets (typically IPs or hosts) within a given attack surface that are known to be impacted by the CVE. The list is returned as type passivetotal.analyzer.illuminate.vuln.AttackSurfaceObservations and contains a list of passivetotal.analyzer.illuminate.vuln.AttackSurfaceObservation objects.

>>> cves = analyzer.AttackSurface(12345).cves
>>> scored_cves = cves.filter_fn(lambda c: c.score > 80).sorted_by('score',True)
>>> highest_scored_cve = scored_cves[0]
>>> for observation in highest_scored_cve.observations:
...    print(observation)

In this example, we used the filter_fn method available on most list-like analyzer objects to apply a function similar to the Python filter() method. This helps us find one CVE to focus on, which will then give us a list of observations.

Vulnerability Articles

If you already know the identifier for a CVE article, you can access the complete details of the article, including the description, date published, other scores, and a top-level assessment of how exposed your attack surface is to the CVE.

Each article is of type passivetotal.analyzer.illumiante.vuln.VulnArticle

>>> vuln_article = analyzer.illuminate.VulnArticle.load('CVE-2021-23017')
>>> print(vuln_article.description)
...    'This is a known weakness in...'
>>> print(vuln_article.observation_count)
130
>>> for observation in vuln_article.observations:
...    print(observation.name, observation.firstseen, observation.lastseen)

Here, the observation_count property gives us the number of assets in the primary attack surface associated with our API key that are known to be impacted by this vulnerability. We use the observation property to obtain the list, which contains a list of passivetotal.analyzer.illuminate.vuln.AttackSurfaceCVEObservation objects.

For third-parties, use the impacts property to see how many of your third-party vendors’ attack surfaces are impacted by the vulnerability. Each impacted attack surface provides an observations property to obtain the list of impacted assets.

>>> vuln_article = analyzer.illuminate.VulnArticle.load('CVE-2021-23017')
>>> for vendor in vuln_article.attack_surfaces:
...    print(vendor)
>>> impacted_vendor = focus_article.attack_surfaces.filter_substring(vendor_name='union')[0]
>>> for observation in impacted_vendor.observations:
...    print(observation.name, observation.firstseen, observation.lastseen)

Examples & Notebooks

The example Jupyter Notebook for Attack Surface Intelligence includes a section on how to access CVEs, observations and articles on the primary attack surface and third-party attack surfaces.

Vuln Reference

passivetotal.analyzer.illuminate.vuln.AttackSurfaceCVEs

Collection of CVEs associated with an attack surface as a list-like object containing AttackSurfaceCVE objects.

passivetotal.analyzer.illuminate.vuln.AttackSurfaceCVE

A record of a CVE associated with an asset in an attack surface.

passivetotal.analyzer.illuminate.vuln.AttackSurfaceCVEObservations

Collection of observations (assets) associated with a CVE in a specific attack surface as a list-like object containing AttackSurfaceCVEObservation objects.

passivetotal.analyzer.illuminate.vuln.AttackSurfaceCVEObservation

An observation (asset) vulnerable to a specific CVE in a given attack surface.

passivetotal.analyzer.illuminate.vuln.AttackSurfaceComponents

Collection of vulnerable components (detections) associated with an attack surface as a list-like object containing AttackSurfaceComponent objects.

passivetotal.analyzer.illuminate.vuln.AttackSurfaceComponent

A vulnerable component (detection) observed in an attack surface.

passivetotal.analyzer.illuminate.vuln.VulnArticle

Vulnerabilty report providing details on impacted assets and third-party vendors.

passivetotal.analyzer.illuminate.vuln.VulnArticleImpacts

Collection of Illuminate Attack Surfaces potentially impacted by a vulnerability as a list-like object containing VulnArticleImpact objects.

passivetotal.analyzer.illuminate.vuln.VulnArticleImpact

An impacted third-party attack surface with observations (assets) affected by a given vulnerabilty.

class passivetotal.analyzer.illuminate.vuln.AttackSurfaceCVEs(attack_surface=None, pagesize=400)

Collection of CVEs associated with an attack surface as a list-like object containing AttackSurfaceCVE objects.

Public Data Attributes:

attack_surface

Get the Illuminate Attack Surface associated with this list of CVEs.

Inherited from RecordList

all

All the records as a list.

as_dict

Return the recordlist as a list of dictionary objects.

length

Inherited from AsDictionary

as_dict

Return the recordlist as a list of dictionary objects.

pretty

Pretty printed version of this object's dictionary representation.

Inherited from PagedRecordList

totalrecords

Total number of available records as reported by the API.

has_more_records

Whether more records are available.

Inherited from ForPandas

as_df

Get this object as a Pandas DataFrame.

Public Methods:

__init__([attack_surface, pagesize])

Inherited from RecordList

__init__([attack_surface, pagesize])

__iter__()

__getitem__(key)

__len__()

parse(api_response)

Implementations must accept an API response and populate themselves with a list of the correct record types.

filter(**kwargs)

Shortcut for filter_and.

filter_fn(fn)

Return only records where a function returns true.

filter_and(**kwargs)

Return only records that match all key/value arguments.

filter_or(**kwargs)

Return only records that match any key/value arguments.

filter_in(**kwargs)

Return only records where a field contains one or more values.

filter_substring(**kwargs)

Return only records where a case-insensitive match on the field returns true.

filter_substring_in(**kwargs)

Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.

sorted_by(field[, reverse])

Return a sorted list.

filter_dateseen_after(date_string)

Filter only results where the firstseen date property is after a specified date.

filter_dateseen_before(date_string)

Filter only results where the lastseen date property is before a specified date.

filter_dateseen_between(start_date_string, ...)

Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.

Inherited from PagedRecordList

load_next_page()

Load the next page of results from the API.

load_all_pages()

Load all pages of results from the API.

Inherited from ForPandas

to_dataframe(**kwargs)

Render this object as a Pandas DataFrame.


property all

All the records as a list.

property as_df

Get this object as a Pandas DataFrame.

Use to_dataframe() instead if you need to control how the dataframe is built.

Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype: pandas.DataFrame

property as_dict

Return the recordlist as a list of dictionary objects.

property attack_surface

Get the Illuminate Attack Surface associated with this list of CVEs.

Return type

passivetotal.analyzer.illuminate.asi.AttackSurface

filter(**kwargs)

Shortcut for filter_and.

filter_and(**kwargs)

Return only records that match all key/value arguments.

filter_dateseen_after(date_string: str)

Filter only results where the firstseen date property is after a specified date.

The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’

filter_dateseen_before(date_string: str)

Filter only results where the lastseen date property is before a specified date.

The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’

filter_dateseen_between(start_date_string: str, end_date_string: str)

Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.

Date strings should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’

filter_fn(fn)

Return only records where a function returns true.

filter_in(**kwargs)

Return only records where a field contains one or more values.

Usage:

filter_in(fieldname=[‘value1’,’value2’]) or filter_in(fieldname=’value1,value2)

filter_or(**kwargs)

Return only records that match any key/value arguments.

filter_substring(**kwargs)

Return only records where a case-insensitive match on the field returns true.

filter_substring_in(**kwargs)

Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.

Usage:

filter_substring_in(fieldname=[‘substring’,’othersub’]) or filter_substring_in(fieldname=’substring,othersub’)

property has_more_records

Whether more records are available.

Return type

bool

load_all_pages()

Load all pages of results from the API.

load_next_page()

Load the next page of results from the API.

Throws AnalyzerError when has_more_records is False.

parse(api_response)

Implementations must accept an API response and populate themselves with a list of the correct record types.

property pretty

Pretty printed version of this object’s dictionary representation.

sorted_by(field, reverse=False)

Return a sorted list.

Parameters
  • field – name of the attribute to sort on

  • reverse – whether to sort in reverse order.

to_dataframe(**kwargs)

Render this object as a Pandas DataFrame.

Implementations may add additional keywords to customize building the data structure.

Default implementation tries to iterate through self and calls to_dataframe on each record with the same parameters passed to this method. If that fails (usually because self isn’t iterable), it uses the as_dict param of self.

Return type

pandas.DataFrame

property totalrecords

Total number of available records as reported by the API.

class passivetotal.analyzer.illuminate.vuln.AttackSurfaceCVE(attack_surface, api_response={})

A record of a CVE associated with an asset in an attack surface.

Public Data Attributes:

article

CVE article with complete details on this vulnerability.

attack_surface

Attack surface this CVE is associated with.

description

Description of the CVE, retrieved from the vulnerability article associated with this CVE.

publish_date

Publication date of the vulnerability article associated with this CVE.

cwes

List of CWE IDs for this CVE.

id

CVE identifier (alias for cve_id).

cve_id

CVE identifier.

link

API link to get CVE article.

score

RiskIQ priority score for this CVE.

observation_count

Number of observations (assets) in this attack surface impacted by this vulnerabilty.

observations

List of observations (assets) in this attack surface vulnerable to this CVE.

Inherited from AsDictionary

as_dict

Return a dictionary representation of the object.

pretty

Pretty printed version of this object's dictionary representation.

Inherited from ForPandas

as_df

Get this object as a Pandas DataFrame.

Public Methods:

__init__(attack_surface[, api_response])

__str__()

Return str(self).

__repr__()

Return repr(self).

to_dataframe()

Render this object as a Pandas dataframe.

get_observations([pagesize])

Get a list of observations(assets) vulnerable to this CVE in this attack surface.

Inherited from Record

match_all(**kwargs)

Whether attributes of this record match all the key/value arguments.

match_any(**kwargs)

Whether attributes of this record match any of the key/value arguments.

Inherited from ForPandas

to_dataframe()

Render this object as a Pandas dataframe.


property article

CVE article with complete details on this vulnerability.

Return type

passivetotal.analyzer.illuminate.vuln.VulnArticle

property as_df

Get this object as a Pandas DataFrame.

Use to_dataframe() instead if you need to control how the dataframe is built.

Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype: pandas.DataFrame

property as_dict

Return a dictionary representation of the object.

property attack_surface

Attack surface this CVE is associated with.

Return type

passivetotal.analyzer.illuminate.asi.AttackSurface

property cve_id

CVE identifier.

property cwes

List of CWE IDs for this CVE.

property description

Description of the CVE, retrieved from the vulnerability article associated with this CVE.

get_observations(pagesize=400)

Get a list of observations(assets) vulnerable to this CVE in this attack surface.

Parameters

pagesize – Size of pages to retrieve from the API.

Return type

AttackSurfaceCVEObservations

property id

CVE identifier (alias for cve_id).

API link to get CVE article.

Consider using the article property to access the article directly.

match_all(**kwargs)

Whether attributes of this record match all the key/value arguments.

match_any(**kwargs)

Whether attributes of this record match any of the key/value arguments.

property observation_count

Number of observations (assets) in this attack surface impacted by this vulnerabilty.

property observations

List of observations (assets) in this attack surface vulnerable to this CVE.

Return type

passivetotal.analyzer.illuminate.vuln.AttackSurfaceCVEObservations

property pretty

Pretty printed version of this object’s dictionary representation.

property publish_date

Publication date of the vulnerability article associated with this CVE.

property score

RiskIQ priority score for this CVE.

to_dataframe()

Render this object as a Pandas dataframe.

Return type

pandas.DataFrame

class passivetotal.analyzer.illuminate.vuln.AttackSurfaceCVEObservations(cve=None, pagesize=400)

Collection of observations (assets) associated with a CVE in a specific attack surface as a list-like object containing AttackSurfaceCVEObservation objects.

Public Data Attributes:

cve

Get the CVE associated with this list of observations.

Inherited from RecordList

all

All the records as a list.

as_dict

Return the recordlist as a list of dictionary objects.

length

Inherited from AsDictionary

as_dict

Return the recordlist as a list of dictionary objects.

pretty

Pretty printed version of this object's dictionary representation.

Inherited from PagedRecordList

totalrecords

Total number of available records as reported by the API.

has_more_records

Whether more records are available.

Inherited from ForPandas

as_df

Get this object as a Pandas DataFrame.

Public Methods:

__init__([cve, pagesize])

Inherited from RecordList

__init__([cve, pagesize])

__iter__()

__getitem__(key)

__len__()

parse(api_response)

Implementations must accept an API response and populate themselves with a list of the correct record types.

filter(**kwargs)

Shortcut for filter_and.

filter_fn(fn)

Return only records where a function returns true.

filter_and(**kwargs)

Return only records that match all key/value arguments.

filter_or(**kwargs)

Return only records that match any key/value arguments.

filter_in(**kwargs)

Return only records where a field contains one or more values.

filter_substring(**kwargs)

Return only records where a case-insensitive match on the field returns true.

filter_substring_in(**kwargs)

Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.

sorted_by(field[, reverse])

Return a sorted list.

filter_dateseen_after(date_string)

Filter only results where the firstseen date property is after a specified date.

filter_dateseen_before(date_string)

Filter only results where the lastseen date property is before a specified date.

filter_dateseen_between(start_date_string, ...)

Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.

Inherited from PagedRecordList

load_next_page()

Load the next page of results from the API.

load_all_pages()

Load all pages of results from the API.

Inherited from ForPandas

to_dataframe(**kwargs)

Render this object as a Pandas DataFrame.


property all

All the records as a list.

property as_df

Get this object as a Pandas DataFrame.

Use to_dataframe() instead if you need to control how the dataframe is built.

Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype: pandas.DataFrame

property as_dict

Return the recordlist as a list of dictionary objects.

property cve

Get the CVE associated with this list of observations.

Return type

passivetotal.analyzer.illuminate.vuln.AttackSurfaceCVE

filter(**kwargs)

Shortcut for filter_and.

filter_and(**kwargs)

Return only records that match all key/value arguments.

filter_dateseen_after(date_string: str)

Filter only results where the firstseen date property is after a specified date.

The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’

filter_dateseen_before(date_string: str)

Filter only results where the lastseen date property is before a specified date.

The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’

filter_dateseen_between(start_date_string: str, end_date_string: str)

Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.

Date strings should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’

filter_fn(fn)

Return only records where a function returns true.

filter_in(**kwargs)

Return only records where a field contains one or more values.

Usage:

filter_in(fieldname=[‘value1’,’value2’]) or filter_in(fieldname=’value1,value2)

filter_or(**kwargs)

Return only records that match any key/value arguments.

filter_substring(**kwargs)

Return only records where a case-insensitive match on the field returns true.

filter_substring_in(**kwargs)

Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.

Usage:

filter_substring_in(fieldname=[‘substring’,’othersub’]) or filter_substring_in(fieldname=’substring,othersub’)

property has_more_records

Whether more records are available.

Return type

bool

load_all_pages()

Load all pages of results from the API.

load_next_page()

Load the next page of results from the API.

Throws AnalyzerError when has_more_records is False.

parse(api_response)

Implementations must accept an API response and populate themselves with a list of the correct record types.

property pretty

Pretty printed version of this object’s dictionary representation.

sorted_by(field, reverse=False)

Return a sorted list.

Parameters
  • field – name of the attribute to sort on

  • reverse – whether to sort in reverse order.

to_dataframe(**kwargs)

Render this object as a Pandas DataFrame.

Implementations may add additional keywords to customize building the data structure.

Default implementation tries to iterate through self and calls to_dataframe on each record with the same parameters passed to this method. If that fails (usually because self isn’t iterable), it uses the as_dict param of self.

Return type

pandas.DataFrame

property totalrecords

Total number of available records as reported by the API.

class passivetotal.analyzer.illuminate.vuln.AttackSurfaceCVEObservation(cve, api_response={})

An observation (asset) vulnerable to a specific CVE in a given attack surface.

Public Data Attributes:

attack_surface

Attack surface this observation is associated with.

cve

CVE this observation is vulnerable to, in the context of a specific attack surface.

cve_id

RiskIQ identifier for the CVE this observation is vulnerable to.

type

Type of this observation (asset).

name

Name of this observation (asset).

Inherited from AsDictionary

as_dict

Return a dictionary representation of the object.

pretty

Pretty printed version of this object's dictionary representation.

Inherited from FirstLastSeen

firstseen

Date & time the record was first seen.

firstseen_date

Date record was first seen.

firstseen_raw

Raw firstseen value returned by the API.

lastseen

Date & time the record was most recently observed.

lastseen_date

Date the record was most recently observed.

lastseen_raw

Raw lastseen value returned by the API.

duration

Length of time record was observed, in days.

Inherited from ForPandas

as_df

Get this object as a Pandas DataFrame.

Public Methods:

__init__(cve[, api_response])

__str__()

Return str(self).

__repr__()

Return repr(self).

to_dataframe()

Render this object as a Pandas dataframe.

Inherited from Record

match_all(**kwargs)

Whether attributes of this record match all the key/value arguments.

match_any(**kwargs)

Whether attributes of this record match any of the key/value arguments.

Inherited from ForPandas

to_dataframe()

Render this object as a Pandas dataframe.


property as_df

Get this object as a Pandas DataFrame.

Use to_dataframe() instead if you need to control how the dataframe is built.

Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype: pandas.DataFrame

property as_dict

Return a dictionary representation of the object.

property attack_surface

Attack surface this observation is associated with.

Return type

passivetotal.analyzer.illuminate.asi.AttackSurface

property cve

CVE this observation is vulnerable to, in the context of a specific attack surface.

Return type

passivetotal.analyzer.illuminate.vuln.AttackSurfaceCVE

property cve_id

RiskIQ identifier for the CVE this observation is vulnerable to.

property duration

Length of time record was observed, in days.

Calculates the timedelta between firstseen and lastseen.

Return type

int

property firstseen

Date & time the record was first seen.

Return type

datetime

property firstseen_date

Date record was first seen.

Return type

date

property firstseen_raw

Raw firstseen value returned by the API.

property lastseen

Date & time the record was most recently observed.

Return type

datetime

property lastseen_date

Date the record was most recently observed.

Return type

date

property lastseen_raw

Raw lastseen value returned by the API.

match_all(**kwargs)

Whether attributes of this record match all the key/value arguments.

match_any(**kwargs)

Whether attributes of this record match any of the key/value arguments.

property name

Name of this observation (asset).

property pretty

Pretty printed version of this object’s dictionary representation.

to_dataframe()

Render this object as a Pandas dataframe.

Return type

pandas.DataFrame

property type

Type of this observation (asset).

class passivetotal.analyzer.illuminate.vuln.AttackSurfaceComponents(attack_surface=None, pagesize=400)

Collection of vulnerable components (detections) associated with an attack surface as a list-like object containing AttackSurfaceComponent objects.

Public Data Attributes:

attack_surface

Get the attack surface associated with this list of observations.

Inherited from RecordList

all

All the records as a list.

as_dict

Return the recordlist as a list of dictionary objects.

length

Inherited from AsDictionary

as_dict

Return the recordlist as a list of dictionary objects.

pretty

Pretty printed version of this object's dictionary representation.

Inherited from PagedRecordList

totalrecords

Total number of available records as reported by the API.

has_more_records

Whether more records are available.

Inherited from ForPandas

as_df

Get this object as a Pandas DataFrame.

Public Methods:

__init__([attack_surface, pagesize])

Inherited from RecordList

__init__([attack_surface, pagesize])

__iter__()

__getitem__(key)

__len__()

parse(api_response)

Implementations must accept an API response and populate themselves with a list of the correct record types.

filter(**kwargs)

Shortcut for filter_and.

filter_fn(fn)

Return only records where a function returns true.

filter_and(**kwargs)

Return only records that match all key/value arguments.

filter_or(**kwargs)

Return only records that match any key/value arguments.

filter_in(**kwargs)

Return only records where a field contains one or more values.

filter_substring(**kwargs)

Return only records where a case-insensitive match on the field returns true.

filter_substring_in(**kwargs)

Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.

sorted_by(field[, reverse])

Return a sorted list.

filter_dateseen_after(date_string)

Filter only results where the firstseen date property is after a specified date.

filter_dateseen_before(date_string)

Filter only results where the lastseen date property is before a specified date.

filter_dateseen_between(start_date_string, ...)

Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.

Inherited from PagedRecordList

load_next_page()

Load the next page of results from the API.

load_all_pages()

Load all pages of results from the API.

Inherited from ForPandas

to_dataframe(**kwargs)

Render this object as a Pandas DataFrame.


property all

All the records as a list.

property as_df

Get this object as a Pandas DataFrame.

Use to_dataframe() instead if you need to control how the dataframe is built.

Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype: pandas.DataFrame

property as_dict

Return the recordlist as a list of dictionary objects.

property attack_surface

Get the attack surface associated with this list of observations.

Return type

passivetotal.analyzer.illuminate.asi.AttackSurface

filter(**kwargs)

Shortcut for filter_and.

filter_and(**kwargs)

Return only records that match all key/value arguments.

filter_dateseen_after(date_string: str)

Filter only results where the firstseen date property is after a specified date.

The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’

filter_dateseen_before(date_string: str)

Filter only results where the lastseen date property is before a specified date.

The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’

filter_dateseen_between(start_date_string: str, end_date_string: str)

Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.

Date strings should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’

filter_fn(fn)

Return only records where a function returns true.

filter_in(**kwargs)

Return only records where a field contains one or more values.

Usage:

filter_in(fieldname=[‘value1’,’value2’]) or filter_in(fieldname=’value1,value2)

filter_or(**kwargs)

Return only records that match any key/value arguments.

filter_substring(**kwargs)

Return only records where a case-insensitive match on the field returns true.

filter_substring_in(**kwargs)

Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.

Usage:

filter_substring_in(fieldname=[‘substring’,’othersub’]) or filter_substring_in(fieldname=’substring,othersub’)

property has_more_records

Whether more records are available.

Return type

bool

load_all_pages()

Load all pages of results from the API.

load_next_page()

Load the next page of results from the API.

Throws AnalyzerError when has_more_records is False.

parse(api_response)

Implementations must accept an API response and populate themselves with a list of the correct record types.

property pretty

Pretty printed version of this object’s dictionary representation.

sorted_by(field, reverse=False)

Return a sorted list.

Parameters
  • field – name of the attribute to sort on

  • reverse – whether to sort in reverse order.

to_dataframe(**kwargs)

Render this object as a Pandas DataFrame.

Implementations may add additional keywords to customize building the data structure.

Default implementation tries to iterate through self and calls to_dataframe on each record with the same parameters passed to this method. If that fails (usually because self isn’t iterable), it uses the as_dict param of self.

Return type

pandas.DataFrame

property totalrecords

Total number of available records as reported by the API.

class passivetotal.analyzer.illuminate.vuln.AttackSurfaceComponent(attack_surface, api_response={})

A vulnerable component (detection) observed in an attack surface.

Public Data Attributes:

attack_surface

Attack surface this component is associated with.

type

Type of this component (detection).

name

Name of this component (detection).

severity

Severity of this detection.

count

Count.

Inherited from AsDictionary

as_dict

Return a dictionary representation of the object.

pretty

Pretty printed version of this object's dictionary representation.

Inherited from FirstLastSeen

firstseen

Date & time the record was first seen.

firstseen_date

Date record was first seen.

firstseen_raw

Raw firstseen value returned by the API.

lastseen

Date & time the record was most recently observed.

lastseen_date

Date the record was most recently observed.

lastseen_raw

Raw lastseen value returned by the API.

duration

Length of time record was observed, in days.

Inherited from ForPandas

as_df

Get this object as a Pandas DataFrame.

Public Methods:

__init__(attack_surface[, api_response])

__str__()

Return str(self).

__repr__()

Return repr(self).

to_dataframe()

Render this object as a Pandas dataframe.

Inherited from Record

match_all(**kwargs)

Whether attributes of this record match all the key/value arguments.

match_any(**kwargs)

Whether attributes of this record match any of the key/value arguments.

Inherited from ForPandas

to_dataframe()

Render this object as a Pandas dataframe.


property as_df

Get this object as a Pandas DataFrame.

Use to_dataframe() instead if you need to control how the dataframe is built.

Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype: pandas.DataFrame

property as_dict

Return a dictionary representation of the object.

property attack_surface

Attack surface this component is associated with.

Return type

passivetotal.analyzer.illuminate.asi.AttackSurface

property count

Count.

property duration

Length of time record was observed, in days.

Calculates the timedelta between firstseen and lastseen.

Return type

int

property firstseen

Date & time the record was first seen.

Return type

datetime

property firstseen_date

Date record was first seen.

Return type

date

property firstseen_raw

Raw firstseen value returned by the API.

property lastseen

Date & time the record was most recently observed.

Return type

datetime

property lastseen_date

Date the record was most recently observed.

Return type

date

property lastseen_raw

Raw lastseen value returned by the API.

match_all(**kwargs)

Whether attributes of this record match all the key/value arguments.

match_any(**kwargs)

Whether attributes of this record match any of the key/value arguments.

property name

Name of this component (detection).

property pretty

Pretty printed version of this object’s dictionary representation.

property severity

Severity of this detection.

to_dataframe()

Render this object as a Pandas dataframe.

Return type

pandas.DataFrame

property type

Type of this component (detection).

class passivetotal.analyzer.illuminate.vuln.VulnArticle(id=None, api_response=None)

Vulnerabilty report providing details on impacted assets and third-party vendors.

Public Data Attributes:

id

CVE identifier string (alias for cve_id).

cve_id

CVE identifier string.

osint_articles

Get a list of RiskIQ open-source intelligence articles that reference this vulnerability.

description

Narrative description of the CVE.

cwes

List of CWE IDs.

score

RiskIQ-assigned priority score for this vulnerability, ranging between 0 and 100.

cvss2score

The CVSS2 score for this vulnerability.

cvss3score

The CVSSS3 score for this vulnerability.

date_published

The date the article was published.

date_published_raw

The raw (string) value returned from the API with the date the article was published.

date_created

The date the article was created.

date_created_raw

The raw (string) value returned from the API with the date the article was created.

date_publisher_updated

The date the article was updated by the publisher.

date_publisherupdate_raw

The raw (string) value returned from the API with the date the article was updated by the publisher.

references

List of references for this article.

components

List of components (detections) RiskIQ will search for to determine if assets are impacted by this vulnerability.

attack_surfaces

List of Illuminate Attack Surfaces (aka third-party vendors) with assets impacted by this vulnerability.

impacted_attack_surfaces

List of all attack surfaces associated with your API account that are impacted by this vulnerability (they have at least one observation of an impacted asset).

observation_count

Number of observations (assets) within the primary attack surface that are impacted by this vulnerability.

observations

List of observations (assets) within the primary attack surface that are impacted by this vulnerability.

Inherited from AsDictionary

as_dict

Return a dictionary representation of the object.

pretty

Pretty printed version of this object's dictionary representation.

Inherited from ForPandas

as_df

Get this object as a Pandas DataFrame.

Public Methods:

__new__(cls[, id, api_response])

__repr__()

Return repr(self).

__str__()

Return str(self).

to_dataframe([view])

Render this object as a Pandas dataframe.

load(id)

Load a Vulnerability Article by ID.

Inherited from Record

match_all(**kwargs)

Whether attributes of this record match all the key/value arguments.

match_any(**kwargs)

Whether attributes of this record match any of the key/value arguments.

Inherited from ForPandas

to_dataframe([view])

Render this object as a Pandas dataframe.


property as_df

Get this object as a Pandas DataFrame.

Use to_dataframe() instead if you need to control how the dataframe is built.

Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype: pandas.DataFrame

property as_dict

Return a dictionary representation of the object.

property attack_surfaces

List of Illuminate Attack Surfaces (aka third-party vendors) with assets impacted by this vulnerability.

Return type

passivetotal.analyzer.illuminate.vuln.VulnArticleImpacts

property components

List of components (detections) RiskIQ will search for to determine if assets are impacted by this vulnerability.

property cve_id

CVE identifier string.

property cvss2score

The CVSS2 score for this vulnerability.

property cvss3score

The CVSSS3 score for this vulnerability.

property cwes

List of CWE IDs.

property date_created

The date the article was created.

property date_created_raw

The raw (string) value returned from the API with the date the article was created.

property date_published

The date the article was published.

property date_published_raw

The raw (string) value returned from the API with the date the article was published.

property date_publisher_updated

The date the article was updated by the publisher.

property date_publisherupdate_raw

The raw (string) value returned from the API with the date the article was updated by the publisher.

property description

Narrative description of the CVE.

property id

CVE identifier string (alias for cve_id).

property impacted_attack_surfaces

List of all attack surfaces associated with your API account that are impacted by this vulnerability (they have at least one observation of an impacted asset).

Return type

passivetotal.analyzer.illuminate.vuln.VulnArticleImpacts

static load(id)

Load a Vulnerability Article by ID.

Return type

VulnArticle

match_all(**kwargs)

Whether attributes of this record match all the key/value arguments.

match_any(**kwargs)

Whether attributes of this record match any of the key/value arguments.

property observation_count

Number of observations (assets) within the primary attack surface that are impacted by this vulnerability.

property observations

List of observations (assets) within the primary attack surface that are impacted by this vulnerability.

Return type

AttackSurfaceCVEObservations

property osint_articles

Get a list of RiskIQ open-source intelligence articles that reference this vulnerability.

Return type

passivetotal.analyzer.articles.ArticlesList

property pretty

Pretty printed version of this object’s dictionary representation.

property references

List of references for this article.

property score

RiskIQ-assigned priority score for this vulnerability, ranging between 0 and 100.

to_dataframe(view='info')

Render this object as a Pandas dataframe.

Parameters

view – View to generate (info, references, components, or impacts)

Return type

pandas.DataFrame

class passivetotal.analyzer.illuminate.vuln.VulnArticleImpacts(article=None, impacts=[])

Collection of Illuminate Attack Surfaces potentially impacted by a vulnerability as a list-like object containing VulnArticleImpact objects.

Public Data Attributes:

article

Article that describes the vulnerability impacting these attack surfaces.

attack_surfaces

Unfiltered list of all attack surfaces associated with your API account.

cve_id

CVE identifier for the vulnerability this article applies to.

only_impacted

Filter the list to include only articles with at least one observation (asset).

impact_count

Number of attack surfaces with assets impacted by this vulnerability.

Inherited from RecordList

all

All the records as a list.

as_dict

Return the recordlist as a list of dictionary objects.

length

Inherited from AsDictionary

as_dict

Return the recordlist as a list of dictionary objects.

pretty

Pretty printed version of this object's dictionary representation.

Inherited from ForPandas

as_df

Get this object as a Pandas DataFrame.

Public Methods:

__init__([article, impacts])

__repr__()

Return repr(self).

__str__()

Return str(self).

Inherited from RecordList

__init__([article, impacts])

__iter__()

__getitem__(key)

__len__()

parse(api_response)

Implementations must accept an API response and populate themselves with a list of the correct record types.

filter(**kwargs)

Shortcut for filter_and.

filter_fn(fn)

Return only records where a function returns true.

filter_and(**kwargs)

Return only records that match all key/value arguments.

filter_or(**kwargs)

Return only records that match any key/value arguments.

filter_in(**kwargs)

Return only records where a field contains one or more values.

filter_substring(**kwargs)

Return only records where a case-insensitive match on the field returns true.

filter_substring_in(**kwargs)

Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.

sorted_by(field[, reverse])

Return a sorted list.

filter_dateseen_after(date_string)

Filter only results where the firstseen date property is after a specified date.

filter_dateseen_before(date_string)

Filter only results where the lastseen date property is before a specified date.

filter_dateseen_between(start_date_string, ...)

Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.

Inherited from ForPandas

to_dataframe(**kwargs)

Render this object as a Pandas DataFrame.


property all

All the records as a list.

property article

Article that describes the vulnerability impacting these attack surfaces.

Return type

VulnArticle

property as_df

Get this object as a Pandas DataFrame.

Use to_dataframe() instead if you need to control how the dataframe is built.

Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype: pandas.DataFrame

property as_dict

Return the recordlist as a list of dictionary objects.

property attack_surfaces

Unfiltered list of all attack surfaces associated with your API account.

This is the same list you would get if you iterated through an instance of this object. Each record will include a count of assets impacted by a vulnerability, in the “observation_count” property. The count may be zero.

If you need a list of only impacted attack surfaces, use the only_impacted property instead.

Return type

passivetotal.analyzer.illuminate.vuln.VulnArticleImpacts

property cve_id

CVE identifier for the vulnerability this article applies to.

filter(**kwargs)

Shortcut for filter_and.

filter_and(**kwargs)

Return only records that match all key/value arguments.

filter_dateseen_after(date_string: str)

Filter only results where the firstseen date property is after a specified date.

The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’

filter_dateseen_before(date_string: str)

Filter only results where the lastseen date property is before a specified date.

The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’

filter_dateseen_between(start_date_string: str, end_date_string: str)

Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.

Date strings should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’

filter_fn(fn)

Return only records where a function returns true.

filter_in(**kwargs)

Return only records where a field contains one or more values.

Usage:

filter_in(fieldname=[‘value1’,’value2’]) or filter_in(fieldname=’value1,value2)

filter_or(**kwargs)

Return only records that match any key/value arguments.

filter_substring(**kwargs)

Return only records where a case-insensitive match on the field returns true.

filter_substring_in(**kwargs)

Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.

Usage:

filter_substring_in(fieldname=[‘substring’,’othersub’]) or filter_substring_in(fieldname=’substring,othersub’)

property impact_count

Number of attack surfaces with assets impacted by this vulnerability.

Counts the number of records that have more than zero observations.

property only_impacted

Filter the list to include only articles with at least one observation (asset).

Return type

VulnArticleImpacts

parse(api_response)

Implementations must accept an API response and populate themselves with a list of the correct record types.

property pretty

Pretty printed version of this object’s dictionary representation.

sorted_by(field, reverse=False)

Return a sorted list.

Parameters
  • field – name of the attribute to sort on

  • reverse – whether to sort in reverse order.

to_dataframe(**kwargs)

Render this object as a Pandas DataFrame.

Implementations may add additional keywords to customize building the data structure.

Default implementation tries to iterate through self and calls to_dataframe on each record with the same parameters passed to this method. If that fails (usually because self isn’t iterable), it uses the as_dict param of self.

Return type

pandas.DataFrame

class passivetotal.analyzer.illuminate.vuln.VulnArticleImpact(article, api_response=None)

An impacted third-party attack surface with observations (assets) affected by a given vulnerabilty.

Public Data Attributes:

article

Article that describes the vulnerability this observation (asset) is impacted by.

vendor_name

Name of the vendor with observations (assets) impacted by this vulnerability.

vendor_id

The RiskIQ-assigned identifier for this vendor.

attack_surface

Illuminate Attack Surface for the third-party vendor impacted by this vulnerability.

observation_count

Number of observations (assets) within a vendor's attack surface that are impacted by this vulnerability.

observations

List of observations (assets) within this vendor's attack surface that are impacted by this vulnerability.

Inherited from AsDictionary

as_dict

Return a dictionary representation of the object.

pretty

Pretty printed version of this object's dictionary representation.

Inherited from ForPandas

as_df

Get this object as a Pandas DataFrame.

Public Methods:

__init__(article[, api_response])

__repr__()

Return repr(self).

__str__()

Return str(self).

Inherited from Record

match_all(**kwargs)

Whether attributes of this record match all the key/value arguments.

match_any(**kwargs)

Whether attributes of this record match any of the key/value arguments.

Inherited from ForPandas

to_dataframe(**kwargs)

Render this object as a Pandas DataFrame.


property article

Article that describes the vulnerability this observation (asset) is impacted by.

Return type

passivetotal.analyzer.illuminate.vuln.VulnArticle

property as_df

Get this object as a Pandas DataFrame.

Use to_dataframe() instead if you need to control how the dataframe is built.

Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype: pandas.DataFrame

property as_dict

Return a dictionary representation of the object.

property attack_surface

Illuminate Attack Surface for the third-party vendor impacted by this vulnerability.

Return type

passivetotal.analyzer.illuminate.asi.AttackSurface

match_all(**kwargs)

Whether attributes of this record match all the key/value arguments.

match_any(**kwargs)

Whether attributes of this record match any of the key/value arguments.

property observation_count

Number of observations (assets) within a vendor’s attack surface that are impacted by this vulnerability.

property observations

List of observations (assets) within this vendor’s attack surface that are impacted by this vulnerability.

Return type

passivetotal.analyzer.illuminate.vuln.AttackSurfaceCVEObservations

property pretty

Pretty printed version of this object’s dictionary representation.

to_dataframe(**kwargs)

Render this object as a Pandas DataFrame.

Implementations may add additional keywords to customize building the data structure.

Default implementation tries to iterate through self and calls to_dataframe on each record with the same parameters passed to this method. If that fails (usually because self isn’t iterable), it uses the as_dict param of self.

Return type

pandas.DataFrame

property vendor_id

The RiskIQ-assigned identifier for this vendor.

property vendor_name

Name of the vendor with observations (assets) impacted by this vulnerability.