RiskIQ Illuminate¶
Reputation Scoring¶
The RiskIQ Illuminate platform provides dynamic reputation scoring on IPs and hostnames based on real-world activity, indicators, and behaviors.
This library provides access to reptuation scores through the object analyzer (recommended), the command line interface, or the low-level request wrappers.
Review the Getting Started guide for details on setting up your credentials and development environment.
Hostname and IP Reputation Analysis¶
>>> from passivetotal import analyzer
>>> analyzer.init()
>>> reputation = analyzer.Hostname('2020-windows.com').reputation
>>> print(reputation)
72 (SUSPICIOUS)
>>> print(analyzer.IPAddress('123.213.1.23').reputation.score)
88
>>> analyzer.IPAddress('123.213.1.23').reputation.rules
[{'name': 'Third Party Blocklist (vo)',
'description': 'Threat Type: SERVICE SCANNER',
'severity': 5,
'link': None},
{'name': 'Open ports observed',
'description': 'The number of open ports may indicate maliciousness',
'severity': 3,
'link': None}]
The Hostname
and IPAddress
analyzer objects provide a reputation
property that returns an instance of a ReputationScore object. That
object can be treated directly like a string or an integer, or you can
access the properties directly.
- class passivetotal.analyzer.hostname.Hostname(hostname)¶
Represents a hostname such as api.passivetotal.org.
Instances are stored as class members so subsequent instantiations for the same hostname return the same object.
Because of this, storing instances in variables is optional, which can be especially useful in interactive sessions such as Jupyter notebooks.
- property reputation¶
RiskIQ Illuminate Reputation profile for a hostname or IP.
- class passivetotal.analyzer.ip.IPAddress(ip)¶
Represents an IPv4 address such as 8.8.8.8
Instances are stored as class members so subsequent instantiations for the same IP return the same object.
Because of this, storing instances in variables is optional, which can be especially useful in interactive sessions such as Jupyter notebooks.
- property reputation¶
RiskIQ Illuminate Reputation profile for a hostname or IP.
Reputation Score CLI¶
The pt-client
command line script provides quick access to reputation profiles
on one or more hosts in several formats. To get started, view the help for the
illuminate command:
(venv) % pt-client illuminate --help
usage: passivetotal illuminate [-h] [--reputation] [--format {json,csv,text}] [--brief] query [query ...]
positional arguments:
query One or more hostnames or IPs
optional arguments:
-h, --help show this help message and exit
--reputation Get hostname or IP reputation from RiskIQ Illuminate.
--format {json,csv,text}
Format of the output from the query
--brief Create a brief output; for reputation, prints score and classification only
Use --reputation
and pass a space-separated list of hostnames or IPs as the query parameter.
The default format is “json” - for interactive use, try --format=text
:
(venv) % pt-client illuminate --reputation --format=text 2020-windows.com
2020-windows.com 72 (SUSPICIOUS)
Registrant email provider (severity 3)
Domain is registered with an email provider that is
more likely to register malicious domains
Registrar (severity 3)
Domains registered with this registrar are more likely
to be malicious
The --brief
option produces a more compact output with one result per line, which is also
useful with the --format=csv
parameter to prepare a compact dataset for import into another
product.
Pass multiple hostnames or IPs at the end of the command (separated by spaces) to analyze multiple hosts at one time.
Reputation Request Wrapper¶
Use the low-level Illuminate
request wrapper for direct queries to the
API.
- class passivetotal.libs.illuminate.IlluminateRequest(*args, **kwargs)¶
Client to interface with the RiskIQ Illuminate calls from the PassiveTotal API.
- get_reputation(**kwargs)¶
Get RiskIQ Illuminate score for a domain or IP address.
Reference: https://api.riskiq.net/api/reputation/
- Parameters
query – Domain or IP address to search
- Returns
Dict of results
Reputation Score Reference¶
- class passivetotal.analyzer.illuminate.ReputationScore(api_response, query=None)¶
RiskIQ Illuminate Reputation profile for a hostname or an IP.
- property as_dict¶
Representation as a dictionary object.
- property classification¶
Reputation classification as a string.
Typical values include GOOD, SUSPICIOUS, MALICIOUS, or UNKNOWN.
- property rules¶
List of rules that informed the reputation score.
Returns a list of dictionaries.
- property score¶
Reputation score as an integer ranging from 0-100.
Higher values indicate a greater likelihood of maliciousness.
- to_dataframe(explode_rules=False, drop_links=False)¶
Render this object as a Pandas DataFrame.
- Parameters
explode_rules – Whether to create a row for each rule using pandas.DataFrame.explode (optional, defaults to False)
drop_links – Whether to include links when present in exploded rules (optional, defaults to False)
- Return type
pandas.DataFrame
Attack Surface Intelligence¶
RiskIQ Illuminate offers Attack Surface Intelligence (ASI) that delivers prioritized insights on an organization’s attack surface, including impact assets (observations).
ASI is available to licensed users of the RiskIQ Illuminate API.
The analyzer module provides an easy-to-use overlay to interact with the Attack Surface API endpoints and quickly obtain a list of impacted hosts.
Review the Getting Started guide for details on setting up your credentials and development environment.
Your Attack Surface¶
An essential use case for the RiskIQ Illuminate ASI API is to understand your own organization’s attack surface.
>>> from passivetotal import analyzer
>>> analyzer.init()
>>> my_asi = analyzer.AttackSurface()
>>> my_asi
<AttackSurface #99901 "RiskIQ, Inc.">
>>> my_asi.name
'RiskIQ, Inc.'
>>> my_asi.high_priority_observation_count
0
For a complete reference of the properties available in an AttackSurface object, see
passivetotal.analyzer.illuminate.asi.AttackSurface
Attack Surfaces contain a list of insights organized by priority. Insights are included in the response even if there are no impacted assets, but it is easy to filter the list to focus on only the insights with “observations”.
>>> for insight in my_asi.medium_priority_insights:
print(insight.name, insight.observation_count)
ASI: CVE-2021-123 Potential vulnerability 0
...
ASI: Multiple vulnerabilities in System X 1
>>> for insight in my_asi.medium_priority_insights.only_active_insights:
print(insight.name)
ASI: Multiple vulnerabilities in System Q
Insight lists are of type
passivetotal.analyzer.illuminate.asi.AttackSurfaceInsights
and contain a list of
passivetotal.analyzer.illuminate.asi.AttackSurfaceInsight
objects. Each of
these objects provide properties to filter, sort, and render a list of insights.
See the API reference below or click the class references here to see other options.
You can obtain the entire list of insights across all three priority
levels (high, medium, and low) at once. Use the
passivetotal.analyzer.illuminate.asi.AttackSurface.all_insights
or
passivetotal.analyzer.illuminate.asi.AttackSurface.all_active_insights
properties
to get the complete list.
Attack Surface Observations¶
Attack Surface Observations are assets related to a given insight within the context of a specific Attack Surface. Assets typically include IPs and hostnames, and include the “first seen” and “last seen” dates that describe when RiskIQ detected an indication the asset was potentially impacted.
Observations are available in the observations property of an AttackSurfaceInsight.
>>> insights = my_asi.medium_priority_insights.only_active_insights
>>> first_insight = insights[0]
>>> for observation in first_insight.observations:
print(observation.type)
print(observation.name)
HOST
subdomain.passivetotal.org
Notice we filtered the list to only include active insights. If you skip this step, be prepared to catch AnalyzerAPIError exceptions thrown by the API when there are no observations available for a given insight.
The list of observations is returned as a
passivetotal.analyzer.illuminate.asi.AttackSurfaceObservations
object that includes a list of
passivetotal.analyzer.illuminate.asi.AttackSurfaceObservation
objects. Like nearly all
analyzer objects, these objects can be easily rendered as a Python dictionary for integration
with other systems using the as_dict property.
>>> insights = my_asi.medium_priority_insights.only_active_insights
>>> first_insight = insights[0]
>>> observations = first_insight.observations.as_dict
>>> observations
{'records': [{'type': 'HOST', 'name': 'subdomain.passivetotal.org', 'firstseen': '2021-02-03 04:05:06', 'lastseen': '2021-06-07 08:09:10'}]}
Third-Party Attack Surfaces¶
The Third-Party ASI module of Illuminate provides access to the Attack Surfaces of other organizations (aka “vendors”). Your API credentials must be specifically licensed to access third-party Attack Surfaces.
To obtain a list of Attack Surfaces, use the load() method of the
passivetotal.analyzer.illuminate.asi.AttackSurfaces
class.
>>> vendor_attack_surfaces = analyzer.illuminate.AttackSurfaces.load()
>>> for vendor_asi in vendor_attack_surfaces:
print(vendor_asi.name, vendor_asi.high_priority_observation_count)
Example Vendor, Inc 13
SaaS Provider 0
Solutions Systems 9
Or, if you already know the RiskIQ ID of the third-party vendor you want to load, pass it as a parameter to the top-level analyzer.AttackSurface() method we used to load our own attack surface.
>>> vendor_asi = analyzer.AttackSurface('12345') # load by ID
>>> print(vendor_asi.name)
Example Vendor, Inc.
You can also load an attack surface by name, if you use a string that is precise enough to find exactly one vendor.
>>> vendor_asi = analyzer.AttackSurface('ample') # load by name match
>>> print(vendor_asi.name)
Example Vendor, Inc.
This will load the entire list of attack surfaces before searching - use it sparingly and primarily in interactive use. Automated processes should load vendor attack surfaces by ID whenever possible.
The passivetotal.analyzer.illuminate.asi.AttackSurface
objects returned in this list
provide the same functionality as the objects described above that represent your own attack surface.
Use the same techniques to enumerate the insights and observations (assets) for a vendor ASI.
>>> vendor_attack_surfaces = analyzer.illuminate.AttackSurfaces.load()
>>> for vendor_asi in vendor_attack_surfaces:
if vendor_asi.high_priority_observation_count > 0:
print(vendor_asi.name)
for insight in vendor_asi.high_priority_insights.only_active_insights:
print('--- {0.name}'.format(insight))
Example Vendor, Inc
--- ASI: CVE 123
--- ASI: CVE 445
SaaS Provider
--- [Potential] Expired items
Solutions Systems
--- Deprecated Technologies
Examples & Notebooks¶
ASI Reference¶
Collection of RiskIQ Illuminate Attack Surfaces in a list-like object containing |
|
RiskIQ Illuminate Attack Surface Intelligence. |
|
Collection of insights associated with an attack surface in a list-like object containing |
|
An insight associated with an attack surface. |
|
|
Collection of observations (assets) associated with an attack surface insight in a list-like object containing |
|
An attack surface asset (typically identified by an IP address or hostname) that is impacted by an observation. |
- class passivetotal.analyzer.illuminate.asi.AttackSurfaces(pagesize=400)¶
Collection of RiskIQ Illuminate Attack Surfaces in a list-like object containing
AttackSurface
objects.Primarily used for enumerating a set of third-party vendors.
Public Data Attributes:
Inherited from
RecordList
All the records as a list.
Return the recordlist as a list of dictionary objects.
length
Inherited from
AsDictionary
Return the recordlist as a list of dictionary objects.
Pretty printed version of this object's dictionary representation.
Inherited from
PagedRecordList
Total number of available records as reported by the API.
Whether more records are available.
Inherited from
ForPandas
Get this object as a Pandas DataFrame.
Public Methods:
__init__
([pagesize])__getitem__
(key)load
([pagesize])Get a list of all third-party (vendor) attack surfaces authorized for this API account.
Inherited from
RecordList
__init__
([pagesize])__iter__
()__getitem__
(key)__len__
()parse
(api_response)Implementations must accept an API response and populate themselves with a list of the correct record types.
filter
(**kwargs)Shortcut for filter_and.
filter_fn
(fn)Return only records where a function returns true.
filter_and
(**kwargs)Return only records that match all key/value arguments.
filter_or
(**kwargs)Return only records that match any key/value arguments.
filter_in
(**kwargs)Return only records where a field contains one or more values.
filter_substring
(**kwargs)Return only records where a case-insensitive match on the field returns true.
filter_substring_in
(**kwargs)Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.
sorted_by
(field[, reverse])Return a sorted list.
filter_dateseen_after
(date_string)Filter only results where the firstseen date property is after a specified date.
filter_dateseen_before
(date_string)Filter only results where the lastseen date property is before a specified date.
filter_dateseen_between
(start_date_string, ...)Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.
Inherited from
PagedRecordList
Load the next page of results from the API.
Load all pages of results from the API.
Inherited from
ForPandas
to_dataframe
(**kwargs)Render this object as a Pandas DataFrame.
- property all¶
All the records as a list.
- property as_df¶
Get this object as a Pandas DataFrame.
Use to_dataframe() instead if you need to control how the dataframe is built.
Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype:
pandas.DataFrame
- property as_dict¶
Return the recordlist as a list of dictionary objects.
- filter(**kwargs)¶
Shortcut for filter_and.
- filter_and(**kwargs)¶
Return only records that match all key/value arguments.
- filter_dateseen_after(date_string: str)¶
Filter only results where the firstseen date property is after a specified date.
The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’
- filter_dateseen_before(date_string: str)¶
Filter only results where the lastseen date property is before a specified date.
The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’
- filter_dateseen_between(start_date_string: str, end_date_string: str)¶
Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.
Date strings should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’
- filter_fn(fn)¶
Return only records where a function returns true.
- filter_in(**kwargs)¶
Return only records where a field contains one or more values.
- Usage:
filter_in(fieldname=[‘value1’,’value2’]) or filter_in(fieldname=’value1,value2)
- filter_or(**kwargs)¶
Return only records that match any key/value arguments.
- filter_substring(**kwargs)¶
Return only records where a case-insensitive match on the field returns true.
- filter_substring_in(**kwargs)¶
Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.
- Usage:
filter_substring_in(fieldname=[‘substring’,’othersub’]) or filter_substring_in(fieldname=’substring,othersub’)
- static load(pagesize=400)¶
Get a list of all third-party (vendor) attack surfaces authorized for this API account.
- Parameters
pagesize – Size of pages to retrieve from the API (optional).
- Return type
- load_all_pages()¶
Load all pages of results from the API.
- load_next_page()¶
Load the next page of results from the API.
Throws AnalyzerError when has_more_records is False.
- parse(api_response)¶
Implementations must accept an API response and populate themselves with a list of the correct record types.
- property pretty¶
Pretty printed version of this object’s dictionary representation.
- sorted_by(field, reverse=False)¶
Return a sorted list.
- Parameters
field – name of the attribute to sort on
reverse – whether to sort in reverse order.
- to_dataframe(**kwargs)¶
Render this object as a Pandas DataFrame.
Implementations may add additional keywords to customize building the data structure.
Default implementation tries to iterate through self and calls to_dataframe on each record with the same parameters passed to this method. If that fails (usually because self isn’t iterable), it uses the as_dict param of self.
- Return type
pandas.DataFrame
- property totalrecords¶
Total number of available records as reported by the API.
- class passivetotal.analyzer.illuminate.asi.AttackSurface(id=None, api_response=None)¶
RiskIQ Illuminate Attack Surface Intelligence.
Public Data Attributes:
All insights across all levels (high, medium, and low).
All insights with active observations across all levels.
id
Whether this attack surface represents the org associated with the API credentials.
Whether this is a third-party attack surface.
name
Number of high-priority insights with impacted assets (observations).
Number of medium-priority insights with impacted assets (observations).
Number of low-priority insights with impacted assets (observations).
Get high priority insights.
Get medium priority insights.
Get low priority insights.
Get a list of CVEs associated with this attack surface.
List of components (detections) vulnerable to this CVE in this attack surface.
Inherited from
AsDictionary
Return a dictionary representation of the object.
Pretty printed version of this object's dictionary representation.
Inherited from
ForPandas
Get this object as a Pandas DataFrame.
Public Methods:
__new__
(cls[, id, api_response])__str__
()Return str(self).
__repr__
()Return repr(self).
load
([id])find
([id_or_name])Find one attack surface.
get_insights
(level)Get insights at a level (high, medium or low).
get_observation_count
(level)Get number of insights with impacted assets (observations) at a given level.
get_cves
([pagesize])Get a list of CVEs impacting assets in this attack surface.
get_components
([pagesize])Get a list of vulnerable components (detections) in this attack surface.
Render this object as a Pandas DataFrame.
Inherited from
Record
match_all
(**kwargs)Whether attributes of this record match all the key/value arguments.
match_any
(**kwargs)Whether attributes of this record match any of the key/value arguments.
Inherited from
ForPandas
Render this object as a Pandas DataFrame.
- property all_active_insights¶
All insights with active observations across all levels.
- property all_insights¶
All insights across all levels (high, medium, and low).
- property as_df¶
Get this object as a Pandas DataFrame.
Use to_dataframe() instead if you need to control how the dataframe is built.
Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype:
pandas.DataFrame
- property as_dict¶
Return a dictionary representation of the object.
- property components¶
List of components (detections) vulnerable to this CVE in this attack surface.
- property cves¶
Get a list of CVEs associated with this attack surface.
- static find(id_or_name=None)¶
Find one attack surface.
Call with no parameters to find your Attack Surface. Pass a number to load a specific Attack Surface by ID, or pass a string to load the entire list of Attack Surfaces and search them by case-insensitive substring.
Raises AnalyzerError if no attack surfaces are found or if more than one Attack Surface name matches the parameter.
- Returns
passivetotal.illuminate.asi.AttackSurface
- get_components(pagesize=400)¶
Get a list of vulnerable components (detections) in this attack surface.
- Parameters
pagesize – Size of pages to retrieve from the API.
- Return type
passivetotal.analyzer.illuminate.vuln.AttackSurfaceComponents
- get_cves(pagesize=400)¶
Get a list of CVEs impacting assets in this attack surface.
- Parameters
pagesize – Size of pages to retrieve from the API.
- Return type
- get_insights(level)¶
Get insights at a level (high, medium or low).
- Parameters
level – Priority level (high, medium, or low).
- Returns
- get_observation_count(level)¶
Get number of insights with impacted assets (observations) at a given level.
- Parameters
level – Priority level (high, medium, or low).
- Return type
int Count of insights with observations
- property high_priority_insights¶
Get high priority insights.
- property high_priority_observation_count¶
Number of high-priority insights with impacted assets (observations).
- property is_own¶
Whether this attack surface represents the org associated with the API credentials.
- property is_third_party¶
Whether this is a third-party attack surface.
- property low_priority_insights¶
Get low priority insights.
- property low_priority_observation_count¶
Number of low-priority insights with impacted assets (observations).
- match_all(**kwargs)¶
Whether attributes of this record match all the key/value arguments.
- match_any(**kwargs)¶
Whether attributes of this record match any of the key/value arguments.
- property medium_priority_insights¶
Get medium priority insights.
- property medium_priority_observation_count¶
Number of medium-priority insights with impacted assets (observations).
- property pretty¶
Pretty printed version of this object’s dictionary representation.
- to_dataframe()¶
Render this object as a Pandas DataFrame.
- Return type
pandas.DataFrame
- class passivetotal.analyzer.illuminate.asi.AttackSurfaceInsights(attack_surface=None, level=None, api_response=None)¶
Collection of insights associated with an attack surface in a list-like object containing
AttackSurfaceInsight
objects.- __init__(attack_surface=None, level=None, api_response=None)¶
Public Data Attributes:
Attack surface these insights are associated with.
level
active_insight_count
Filter to only active insights (insights with active observations).
total_insight_count
total_observations
Inherited from
RecordList
All the records as a list.
Return the recordlist as a list of dictionary objects.
length
Inherited from
AsDictionary
Return the recordlist as a list of dictionary objects.
Pretty printed version of this object's dictionary representation.
Inherited from
ForPandas
Get this object as a Pandas DataFrame.
Public Methods:
__init__
([attack_surface, level, api_response])load
(attack_surface, level)Inherited from
RecordList
__init__
([attack_surface, level, api_response])__iter__
()__getitem__
(key)__len__
()parse
(api_response)Implementations must accept an API response and populate themselves with a list of the correct record types.
filter
(**kwargs)Shortcut for filter_and.
filter_fn
(fn)Return only records where a function returns true.
filter_and
(**kwargs)Return only records that match all key/value arguments.
filter_or
(**kwargs)Return only records that match any key/value arguments.
filter_in
(**kwargs)Return only records where a field contains one or more values.
filter_substring
(**kwargs)Return only records where a case-insensitive match on the field returns true.
filter_substring_in
(**kwargs)Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.
sorted_by
(field[, reverse])Return a sorted list.
filter_dateseen_after
(date_string)Filter only results where the firstseen date property is after a specified date.
filter_dateseen_before
(date_string)Filter only results where the lastseen date property is before a specified date.
filter_dateseen_between
(start_date_string, ...)Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.
Inherited from
ForPandas
to_dataframe
(**kwargs)Render this object as a Pandas DataFrame.
- property all¶
All the records as a list.
- property as_df¶
Get this object as a Pandas DataFrame.
Use to_dataframe() instead if you need to control how the dataframe is built.
Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype:
pandas.DataFrame
- property as_dict¶
Return the recordlist as a list of dictionary objects.
- property attack_surface¶
Attack surface these insights are associated with.
- filter(**kwargs)¶
Shortcut for filter_and.
- filter_and(**kwargs)¶
Return only records that match all key/value arguments.
- filter_dateseen_after(date_string: str)¶
Filter only results where the firstseen date property is after a specified date.
The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’
- filter_dateseen_before(date_string: str)¶
Filter only results where the lastseen date property is before a specified date.
The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’
- filter_dateseen_between(start_date_string: str, end_date_string: str)¶
Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.
Date strings should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’
- filter_fn(fn)¶
Return only records where a function returns true.
- filter_in(**kwargs)¶
Return only records where a field contains one or more values.
- Usage:
filter_in(fieldname=[‘value1’,’value2’]) or filter_in(fieldname=’value1,value2)
- filter_or(**kwargs)¶
Return only records that match any key/value arguments.
- filter_substring(**kwargs)¶
Return only records where a case-insensitive match on the field returns true.
- filter_substring_in(**kwargs)¶
Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.
- Usage:
filter_substring_in(fieldname=[‘substring’,’othersub’]) or filter_substring_in(fieldname=’substring,othersub’)
- property only_active_insights¶
Filter to only active insights (insights with active observations).
- Return type
- parse(api_response)¶
Implementations must accept an API response and populate themselves with a list of the correct record types.
- property pretty¶
Pretty printed version of this object’s dictionary representation.
- sorted_by(field, reverse=False)¶
Return a sorted list.
- Parameters
field – name of the attribute to sort on
reverse – whether to sort in reverse order.
- to_dataframe(**kwargs)¶
Render this object as a Pandas DataFrame.
Implementations may add additional keywords to customize building the data structure.
Default implementation tries to iterate through self and calls to_dataframe on each record with the same parameters passed to this method. If that fails (usually because self isn’t iterable), it uses the as_dict param of self.
- Return type
pandas.DataFrame
- class passivetotal.analyzer.illuminate.asi.AttackSurfaceInsight(attack_surface, level, api_response)¶
An insight associated with an attack surface.
Public Data Attributes:
Unique ID for the insight.
Priority level of this insight.
Attack surface this insight is associated with.
Insight short name.
Insight long description.
API link to get observations.
Number of observations (assets) impacted by this insight.
Whether this insight has any active observations (assets).
List of impacted assets.
Inherited from
AsDictionary
Return a dictionary representation of the object.
Pretty printed version of this object's dictionary representation.
Inherited from
ForPandas
Get this object as a Pandas DataFrame.
Public Methods:
__init__
(attack_surface, level, api_response)__str__
()Return str(self).
Render this insight as a Pandas DataFrame.
get_observations
([pagesize])Get a list of impacted assets (observations).
Inherited from
Record
match_all
(**kwargs)Whether attributes of this record match all the key/value arguments.
match_any
(**kwargs)Whether attributes of this record match any of the key/value arguments.
Inherited from
ForPandas
Render this insight as a Pandas DataFrame.
- property as_df¶
Get this object as a Pandas DataFrame.
Use to_dataframe() instead if you need to control how the dataframe is built.
Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype:
pandas.DataFrame
- property as_dict¶
Return a dictionary representation of the object.
- property attack_surface¶
Attack surface this insight is associated with.
- property description¶
Insight long description.
- get_observations(pagesize=400)¶
Get a list of impacted assets (observations).
- Parameters
pagesize – Size of pages to retrieve from the API.
- Return type
- property has_observations¶
Whether this insight has any active observations (assets).
- property id¶
Unique ID for the insight.
- property level¶
Priority level of this insight.
- property link¶
API link to get observations.
- match_all(**kwargs)¶
Whether attributes of this record match all the key/value arguments.
- match_any(**kwargs)¶
Whether attributes of this record match any of the key/value arguments.
- property name¶
Insight short name.
- property observation_count¶
Number of observations (assets) impacted by this insight.
- property observations¶
List of impacted assets.
- Return type
- property pretty¶
Pretty printed version of this object’s dictionary representation.
- to_dataframe()¶
Render this insight as a Pandas DataFrame.
- Return type
pandas.DataFrame
- class passivetotal.analyzer.illuminate.asi.AttackSurfaceObservations(insight, group_by, segment_by, pagesize=400)¶
Collection of observations (assets) associated with an attack surface insight in a list-like object containing
AttackSurfaceObservation
objects.Public Data Attributes:
List of unique asset types in this observation list.
List of unique hostnames in this observation list.
List of unique IP addresses in this observation list.
Inherited from
RecordList
All the records as a list.
Return the recordlist as a list of dictionary objects.
length
Inherited from
AsDictionary
Return the recordlist as a list of dictionary objects.
Pretty printed version of this object's dictionary representation.
Inherited from
PagedRecordList
Total number of available records as reported by the API.
Whether more records are available.
Inherited from
ForPandas
Get this object as a Pandas DataFrame.
Public Methods:
__init__
(insight, group_by, segment_by[, ...])Inherited from
RecordList
__init__
(insight, group_by, segment_by[, ...])__iter__
()__getitem__
(key)__len__
()parse
(api_response)Implementations must accept an API response and populate themselves with a list of the correct record types.
filter
(**kwargs)Shortcut for filter_and.
filter_fn
(fn)Return only records where a function returns true.
filter_and
(**kwargs)Return only records that match all key/value arguments.
filter_or
(**kwargs)Return only records that match any key/value arguments.
filter_in
(**kwargs)Return only records where a field contains one or more values.
filter_substring
(**kwargs)Return only records where a case-insensitive match on the field returns true.
filter_substring_in
(**kwargs)Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.
sorted_by
(field[, reverse])Return a sorted list.
filter_dateseen_after
(date_string)Filter only results where the firstseen date property is after a specified date.
filter_dateseen_before
(date_string)Filter only results where the lastseen date property is before a specified date.
filter_dateseen_between
(start_date_string, ...)Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.
Inherited from
PagedRecordList
Load the next page of results from the API.
Load all pages of results from the API.
Inherited from
ForPandas
to_dataframe
(**kwargs)Render this object as a Pandas DataFrame.
- property all¶
All the records as a list.
- property as_df¶
Get this object as a Pandas DataFrame.
Use to_dataframe() instead if you need to control how the dataframe is built.
Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype:
pandas.DataFrame
- property as_dict¶
Return the recordlist as a list of dictionary objects.
- property asset_types¶
List of unique asset types in this observation list.
- filter(**kwargs)¶
Shortcut for filter_and.
- filter_and(**kwargs)¶
Return only records that match all key/value arguments.
- filter_dateseen_after(date_string: str)¶
Filter only results where the firstseen date property is after a specified date.
The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’
- filter_dateseen_before(date_string: str)¶
Filter only results where the lastseen date property is before a specified date.
The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’
- filter_dateseen_between(start_date_string: str, end_date_string: str)¶
Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.
Date strings should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’
- filter_fn(fn)¶
Return only records where a function returns true.
- filter_in(**kwargs)¶
Return only records where a field contains one or more values.
- Usage:
filter_in(fieldname=[‘value1’,’value2’]) or filter_in(fieldname=’value1,value2)
- filter_or(**kwargs)¶
Return only records that match any key/value arguments.
- filter_substring(**kwargs)¶
Return only records where a case-insensitive match on the field returns true.
- filter_substring_in(**kwargs)¶
Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.
- Usage:
filter_substring_in(fieldname=[‘substring’,’othersub’]) or filter_substring_in(fieldname=’substring,othersub’)
- property hostnames¶
List of unique hostnames in this observation list.
- Return type
- property ips¶
List of unique IP addresses in this observation list.
- Return type
- load_all_pages()¶
Load all pages of results from the API.
- load_next_page()¶
Load the next page of results from the API.
Throws AnalyzerError when has_more_records is False.
- parse(api_response)¶
Implementations must accept an API response and populate themselves with a list of the correct record types.
- property pretty¶
Pretty printed version of this object’s dictionary representation.
- sorted_by(field, reverse=False)¶
Return a sorted list.
- Parameters
field – name of the attribute to sort on
reverse – whether to sort in reverse order.
- to_dataframe(**kwargs)¶
Render this object as a Pandas DataFrame.
Implementations may add additional keywords to customize building the data structure.
Default implementation tries to iterate through self and calls to_dataframe on each record with the same parameters passed to this method. If that fails (usually because self isn’t iterable), it uses the as_dict param of self.
- Return type
pandas.DataFrame
- property totalrecords¶
Total number of available records as reported by the API.
- class passivetotal.analyzer.illuminate.asi.AttackSurfaceObservation(insight, api_response)¶
An attack surface asset (typically identified by an IP address or hostname) that is impacted by an observation.
Public Data Attributes:
Type of impacted asset (i.e.
Name of impacted asset (i.e.
Attack surface insight associated with this observation.
Shortcut property to access an analyzer.Hostname object from this observation, if the observation type is HOST, else returns None.
Shortcut property to access an analyzer.IPAddress object from this observation, if the observation type is IP_ADDRESS, else returns None.
Inherited from
AsDictionary
Return a dictionary representation of the object.
Pretty printed version of this object's dictionary representation.
Inherited from
FirstLastSeen
Date & time the record was first seen.
Date record was first seen.
Raw firstseen value returned by the API.
Date & time the record was most recently observed.
Date the record was most recently observed.
Raw lastseen value returned by the API.
Length of time record was observed, in days.
Inherited from
ForPandas
Get this object as a Pandas DataFrame.
Public Methods:
__init__
(insight, api_response)__str__
()Return str(self).
__repr__
()Return repr(self).
Render this object as a Pandas dataframe.
Inherited from
Record
match_all
(**kwargs)Whether attributes of this record match all the key/value arguments.
match_any
(**kwargs)Whether attributes of this record match any of the key/value arguments.
Inherited from
ForPandas
Render this object as a Pandas dataframe.
- property as_df¶
Get this object as a Pandas DataFrame.
Use to_dataframe() instead if you need to control how the dataframe is built.
Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype:
pandas.DataFrame
- property as_dict¶
Return a dictionary representation of the object.
- property duration¶
Length of time record was observed, in days.
Calculates the timedelta between firstseen and lastseen.
- Return type
- property firstseen¶
Date & time the record was first seen.
- Return type
datetime
- property firstseen_date¶
Date record was first seen.
- Return type
date
- property firstseen_raw¶
Raw firstseen value returned by the API.
- property hostname¶
Shortcut property to access an analyzer.Hostname object from this observation, if the observation type is HOST, else returns None.
- Return type
- property insight¶
Attack surface insight associated with this observation.
- Return type
- property ip¶
Shortcut property to access an analyzer.IPAddress object from this observation, if the observation type is IP_ADDRESS, else returns None.
- Return type
- property lastseen¶
Date & time the record was most recently observed.
- Return type
datetime
- property lastseen_date¶
Date the record was most recently observed.
- Return type
date
- property lastseen_raw¶
Raw lastseen value returned by the API.
- match_all(**kwargs)¶
Whether attributes of this record match all the key/value arguments.
- match_any(**kwargs)¶
Whether attributes of this record match any of the key/value arguments.
- property name¶
Name of impacted asset (i.e. the IP address or hostname).
- property pretty¶
Pretty printed version of this object’s dictionary representation.
- to_dataframe()¶
Render this object as a Pandas dataframe.
- Return type
pandas.DataFrame
- property type¶
Type of impacted asset (i.e. HOST or IP_ADDRESS).
Intelligence Profiles (CTI)¶
The RiskIQ Illuminate platform offers a Cyber Threat Intelligence (CTI) module that delivers insights into adversary threat infrastructure organized around a set of purpose-built intelligence profiles, each with a curated set of indicators.
The analyzer module provides the optimal interface to query the CTI dataset, but low-level request wrappers are also available. See below for the API reference docs.
Review the Getting Started guide for details on setting up your credentials and development environment.
Important: this module must be specifically enabled for your API credentials.
Intel Profiles¶
For most use cases, start with the list of RiskIQ Illuminate intel profiles. These
are returned as a passivetotal.analyzer.illuminate.cti.IntelProfiles
type that
can be iterated and filtered like a regular list.
>>> from passivetotal import analyzer
>>> analyzer.init()
>>> intel_profiles = analyzer.illuminate.IntelProfiles.load()
>>> for profile in intel_profiles:
... print(profile.id, profile.title, profile.indicatorcount_riskiq)
Each record in the list is of type passivetotal.analyzer.illuminate.cti.IntelProfile
Intel Profiles are identified with a unique string in the id parameter. Once you know the profile you want to focus on, you can instantiate it directly using that id.
>>> profile = analyzer.illuminate.IntelProfile('apt33')
>>> print(profile.pretty)
{ 'id': 'apt33',
'indicatorcount_osint': 33333,
'indicatorcount_riskiq': 55555,
'tags_raw': [ {'countryCode': None, 'label': 'Malicious'},
{'countryCode': 'us', 'label': 'Target: USA'}],
'title': 'APT33'}
Indicator Lists¶
The RiskIQ research team curates lists of indicators associated with each intel profile, some sourced from open-source intelligence, and others surfaced directly from RiskIQ proprietary datasets.
There are several ways to obtain the list of indicators associated with a specific
intel profile. Each method will return an object of type
passivetotal.analyzer.illuminate.cti.IntelProfileIndicator
that can be iterated like a standard list and also offers several built-in methods
and properties to filter, sort, and render the list.
Obtain the list of indicators directly as a property of an
passivetotal.analyzer.illuminate.cti.IntelProfile
:
>>> profile = analyzer.illuminate.IntelProfile('apt33')
>>> for indicator in profile.indicators:
... print(indicator.pretty)
Each indicator is of type
passivetotal.analyzer.illuminate.cti.IntelProfileIndicator
For more granular control, call the get_indicators() method and set additional parameters supported by the API to narrow the list:
>>> profile = analyzer.illuminate.IntelProfile('apt33')
>>> indicators = profile.get_indicators(sources='riskiq')
>>> len(indicators)
55555
Or, skip the intel profile entirely and go straight to the indicator list.
>>> ioc_list = analyzer.illuminate.IntelProfileIndicatorList('apt33')
>>> ioc_list.load_all_pages()
>>> len(ioc_list)
55555
The underlying API calls require pagination, but the analyzer module handles that automatically when you access either the indicators property or call get_indicators() directly on the profile objects. Here, we are using the load_all_pages() method to populate the list directly.
If you only need the values of the indicators, you can quickly access them as a plain Python list with the values property:
>>> profile = analyzer.illuminate.IntelProfile('apt33')
>>> indicators = profile.get_indicators(sources='riskiq')
>>> indicators.values
['.....']
>>> indicators.only_riskiq.values
['.....']
>>> indicators.filter_in(type='domain,ip').values
['.....']
Search By Indicator¶
The CTI API provides a mechanism to search for intel profiles by the value of an indicator. You can query the API directly for any of the indicator types stored in the RiskIQ dataset, or you can access the list as a property of analyzer.Hostname or analyzer.IPAddress objects.
To search the API directly, use the
passivetotal.analyzer.illuminate.cti.IntelProfiles.find_by_indicator()
static method:
>>> results = analyzer.illuminate.IntelProfiles.find_by_indicator('threat_actor@gmail.com')
>>> len(results)
1
>>> print(results[0].title)
APT123
>>> for ioc in results[0].indicators:
... print(ioc)
Or, if you are working with a hostname or IP address, access the intel_profiles property to obtain the profile list. Be sure to test the length of the list before accessing properties to avoid runtime exceptions.
>>> if len(analyzer.IPAddress('123.123.123.123').intel_profiles) > 0:
... print(intel_profiles.pretty)
Examples & Notebooks¶
CTI Reference¶
List of RiskIQ Intel Profiles from the Illuminate CTI module. |
|
RiskIQ Intel Profile on a specific actor group. |
|
|
|
An indicator associated with an intel profile. |
- class passivetotal.analyzer.illuminate.cti.IntelProfiles(api_response=None, query=None)¶
List of RiskIQ Intel Profiles from the Illuminate CTI module.
Public Data Attributes:
Total number of profiles available in this record list.
Inherited from
RecordList
All the records as a list.
Return the recordlist as a list of dictionary objects.
length
Inherited from
AsDictionary
Return the recordlist as a list of dictionary objects.
Pretty printed version of this object's dictionary representation.
Inherited from
ForPandas
Get this object as a Pandas DataFrame.
Public Methods:
__getitem__
(key)load
([query, profile_type])Get a list of all RiskIQ Intel Profiles.
find_by_indicator
(query, **kwargs)Search profiles by indicator.
parse
(api_response)Parse an API response.
to_dataframe
([ignore_index])Render this object as a Pandas dataframe.
Inherited from
RecordList
__init__
([api_response, query])__iter__
()__getitem__
(key)__len__
()parse
(api_response)Parse an API response.
filter
(**kwargs)Shortcut for filter_and.
filter_fn
(fn)Return only records where a function returns true.
filter_and
(**kwargs)Return only records that match all key/value arguments.
filter_or
(**kwargs)Return only records that match any key/value arguments.
filter_in
(**kwargs)Return only records where a field contains one or more values.
filter_substring
(**kwargs)Return only records where a case-insensitive match on the field returns true.
filter_substring_in
(**kwargs)Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.
sorted_by
(field[, reverse])Return a sorted list.
filter_dateseen_after
(date_string)Filter only results where the firstseen date property is after a specified date.
filter_dateseen_before
(date_string)Filter only results where the lastseen date property is before a specified date.
filter_dateseen_between
(start_date_string, ...)Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.
Inherited from
ForPandas
to_dataframe
([ignore_index])Render this object as a Pandas dataframe.
- property all¶
All the records as a list.
- property as_df¶
Get this object as a Pandas DataFrame.
Use to_dataframe() instead if you need to control how the dataframe is built.
Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype:
pandas.DataFrame
- property as_dict¶
Return the recordlist as a list of dictionary objects.
- filter(**kwargs)¶
Shortcut for filter_and.
- filter_and(**kwargs)¶
Return only records that match all key/value arguments.
- filter_dateseen_after(date_string: str)¶
Filter only results where the firstseen date property is after a specified date.
The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’
- filter_dateseen_before(date_string: str)¶
Filter only results where the lastseen date property is before a specified date.
The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’
- filter_dateseen_between(start_date_string: str, end_date_string: str)¶
Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.
Date strings should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’
- filter_fn(fn)¶
Return only records where a function returns true.
- filter_in(**kwargs)¶
Return only records where a field contains one or more values.
- Usage:
filter_in(fieldname=[‘value1’,’value2’]) or filter_in(fieldname=’value1,value2)
- filter_or(**kwargs)¶
Return only records that match any key/value arguments.
- filter_substring(**kwargs)¶
Return only records where a case-insensitive match on the field returns true.
- filter_substring_in(**kwargs)¶
Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.
- Usage:
filter_substring_in(fieldname=[‘substring’,’othersub’]) or filter_substring_in(fieldname=’substring,othersub’)
- static find_by_indicator(query, **kwargs)¶
Search profiles by indicator.
- Parameters
query – Indicator value as a string
types – Types of indicators (optional)
categories – Categories of indicators (optional)
sources – Sources of indicators [riskiq, osint] (optional)
- static load(query=None, profile_type=None)¶
Get a list of all RiskIQ Intel Profiles.
- Parameters
query – Submit a query param to the API to limit results to only matching providers (optional)
profile_type – Submit a type param to the API to limit results to only certain profile types (optional)
- parse(api_response)¶
Parse an API response.
- property pretty¶
Pretty printed version of this object’s dictionary representation.
- sorted_by(field, reverse=False)¶
Return a sorted list.
- Parameters
field – name of the attribute to sort on
reverse – whether to sort in reverse order.
- to_dataframe(ignore_index=False, **kwargs)¶
Render this object as a Pandas dataframe.
- property totalrecords¶
Total number of profiles available in this record list.
- class passivetotal.analyzer.illuminate.cti.IntelProfile(id=None, api_response=None)¶
RiskIQ Intel Profile on a specific actor group.
Public Data Attributes:
List of alternative names for this actor group.
RiskIQ identifier for this actor group.
Count of available indicators from open source intelligence reports.
Count of available indicators sourced from RiskIQ primary research.
Unfiltered indicator list associated with this intel profile.
List of profile tags associated with this actor group.
List of profile tags as returned by the API.
RiskIQ title for this actor profile.
Inherited from
AsDictionary
Return a dictionary representation of the object.
Pretty printed version of this object's dictionary representation.
Inherited from
ForPandas
Get this object as a Pandas DataFrame.
Public Methods:
__new__
(cls[, id, api_response])__str__
()Return str(self).
__repr__
()Return repr(self).
load
(profile_id)Load an intel profile by the RiskIQ-assigned identifier string.
get_indicators
([all_pages, types, ...])Get a list of indicators associated with this intel profile.
Render this profile as a Pandas DataFrame.
Inherited from
Record
match_all
(**kwargs)Whether attributes of this record match all the key/value arguments.
match_any
(**kwargs)Whether attributes of this record match any of the key/value arguments.
Inherited from
ForPandas
Render this profile as a Pandas DataFrame.
- class ProfileTag(label, country)¶
- property country¶
Alias for field number 1
- property label¶
Alias for field number 0
- property aliases¶
List of alternative names for this actor group.
- property as_df¶
Get this object as a Pandas DataFrame.
Use to_dataframe() instead if you need to control how the dataframe is built.
Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype:
pandas.DataFrame
- property as_dict¶
Return a dictionary representation of the object.
- get_indicators(all_pages=True, types=None, categories=None, sources=None, pagesize=400)¶
Get a list of indicators associated with this intel profile.
Loads all pages of indicators by default. Results with identical params are cached.
- Parameters
all_pages – Whether to retrieve all pages (optional, defaults to True)
types – Types of indicators to search for (optional).
categories – Categories of indicators to filter on (optional).
sources – Sources of indicators [osint, riskiq] (optional).
pagesize – Size of pages to return from the API (defaults to INDICATOR_PAGE_SIZE).
- Return type
passivetotal.analyzer.illuminate.cti.IntelProfileIndicatorList
- property id¶
RiskIQ identifier for this actor group.
- property indicatorcount_osint¶
Count of available indicators from open source intelligence reports.
- property indicatorcount_riskiq¶
Count of available indicators sourced from RiskIQ primary research.
- property indicators¶
Unfiltered indicator list associated with this intel profile.
Calls passivetotal.analyzer.illuminate.IntelProfile.get_indicators() with default parameters. Use that method directly for more granular control.
- static load(profile_id)¶
Load an intel profile by the RiskIQ-assigned identifier string.
- match_all(**kwargs)¶
Whether attributes of this record match all the key/value arguments.
- match_any(**kwargs)¶
Whether attributes of this record match any of the key/value arguments.
- property pretty¶
Pretty printed version of this object’s dictionary representation.
- property tags¶
List of profile tags associated with this actor group.
- property tags_raw¶
List of profile tags as returned by the API.
- property title¶
RiskIQ title for this actor profile.
- to_dataframe()¶
Render this profile as a Pandas DataFrame.
- Return type
pandas.DataFrame
- class passivetotal.analyzer.illuminate.cti.IntelProfileIndicatorList(profile_id=None, query=None, types=None, categories=None, sources=None, pagesize=400)¶
- __init__(profile_id=None, query=None, types=None, categories=None, sources=None, pagesize=400)¶
Collection of indicators associated with a RiskIQ Intel Profile as a list-like object containing
IntelProfileIndicator
objects.- Parameters
profile_id – Threat intel profile ID to search for.
query – Indicator value to query for.
types – Types of indicators to search for (optional).
categories – Categories of indicators to filter on (optional).
sources – Sources of indicators [osint, riskiq] (optiona).
Public Data Attributes:
Filtered list with only indicators from open sources.
Filtered list with only indicators sourced by RiskIQ.
List of indicator types in the list.
List of all values in the indicator list.
Inherited from
RecordList
All the records as a list.
Return the recordlist as a list of dictionary objects.
length
Inherited from
AsDictionary
Return the recordlist as a list of dictionary objects.
Pretty printed version of this object's dictionary representation.
Inherited from
PagedRecordList
Total number of available records as reported by the API.
Whether more records are available.
Inherited from
ForPandas
Get this object as a Pandas DataFrame.
Public Methods:
__init__
([profile_id, query, types, ...])Collection of indicators associated with a RiskIQ Intel Profile as a list-like object containing
IntelProfileIndicator
objects.to_dataframe
([ignore_index])Render this object as a Pandas dataframe.
Inherited from
RecordList
__init__
([profile_id, query, types, ...])Collection of indicators associated with a RiskIQ Intel Profile as a list-like object containing
IntelProfileIndicator
objects.__iter__
()__getitem__
(key)__len__
()parse
(api_response)Implementations must accept an API response and populate themselves with a list of the correct record types.
filter
(**kwargs)Shortcut for filter_and.
filter_fn
(fn)Return only records where a function returns true.
filter_and
(**kwargs)Return only records that match all key/value arguments.
filter_or
(**kwargs)Return only records that match any key/value arguments.
filter_in
(**kwargs)Return only records where a field contains one or more values.
filter_substring
(**kwargs)Return only records where a case-insensitive match on the field returns true.
filter_substring_in
(**kwargs)Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.
sorted_by
(field[, reverse])Return a sorted list.
filter_dateseen_after
(date_string)Filter only results where the firstseen date property is after a specified date.
filter_dateseen_before
(date_string)Filter only results where the lastseen date property is before a specified date.
filter_dateseen_between
(start_date_string, ...)Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.
Inherited from
PagedRecordList
Load the next page of results from the API.
Load all pages of results from the API.
Inherited from
ForPandas
to_dataframe
([ignore_index])Render this object as a Pandas dataframe.
- property all¶
All the records as a list.
- property as_df¶
Get this object as a Pandas DataFrame.
Use to_dataframe() instead if you need to control how the dataframe is built.
Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype:
pandas.DataFrame
- property as_dict¶
Return the recordlist as a list of dictionary objects.
- filter(**kwargs)¶
Shortcut for filter_and.
- filter_and(**kwargs)¶
Return only records that match all key/value arguments.
- filter_dateseen_after(date_string: str)¶
Filter only results where the firstseen date property is after a specified date.
The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’
- filter_dateseen_before(date_string: str)¶
Filter only results where the lastseen date property is before a specified date.
The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’
- filter_dateseen_between(start_date_string: str, end_date_string: str)¶
Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.
Date strings should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’
- filter_fn(fn)¶
Return only records where a function returns true.
- filter_in(**kwargs)¶
Return only records where a field contains one or more values.
- Usage:
filter_in(fieldname=[‘value1’,’value2’]) or filter_in(fieldname=’value1,value2)
- filter_or(**kwargs)¶
Return only records that match any key/value arguments.
- filter_substring(**kwargs)¶
Return only records where a case-insensitive match on the field returns true.
- filter_substring_in(**kwargs)¶
Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.
- Usage:
filter_substring_in(fieldname=[‘substring’,’othersub’]) or filter_substring_in(fieldname=’substring,othersub’)
- load_all_pages()¶
Load all pages of results from the API.
- load_next_page()¶
Load the next page of results from the API.
Throws AnalyzerError when has_more_records is False.
- property only_osint¶
Filtered list with only indicators from open sources.
- property only_riskiq¶
Filtered list with only indicators sourced by RiskIQ.
- parse(api_response)¶
Implementations must accept an API response and populate themselves with a list of the correct record types.
- property pretty¶
Pretty printed version of this object’s dictionary representation.
- sorted_by(field, reverse=False)¶
Return a sorted list.
- Parameters
field – name of the attribute to sort on
reverse – whether to sort in reverse order.
- to_dataframe(ignore_index=False, **kwargs)¶
Render this object as a Pandas dataframe.
- property totalrecords¶
Total number of available records as reported by the API.
- property types¶
List of indicator types in the list.
- property values¶
List of all values in the indicator list.
- class passivetotal.analyzer.illuminate.cti.IntelProfileIndicator(api_response)¶
An indicator associated with an intel profile.
Public Data Attributes:
RiskIQ identifier for this indicator.
RiskIQ identifier for the intel profile associated with this indicator.
RiskIQ threat intel profile associated with this indicator.
Type of indicator.
Value of the indicator.
Indicator category.
Whether this indicator was published in open source intelligence articles.
URL for the OSINT source of the indicator, or none if this is not an OSINT indicator.
List of RiskIQ OSINT article GUIDs associated with this indicator.
Inherited from
AsDictionary
Return a dictionary representation of the object.
Pretty printed version of this object's dictionary representation.
Inherited from
FirstLastSeen
Date & time the record was first seen.
Date record was first seen.
Raw firstseen value returned by the API.
Date & time the record was most recently observed.
Date the record was most recently observed.
Raw lastseen value returned by the API.
Length of time record was observed, in days.
Inherited from
ForPandas
Get this object as a Pandas DataFrame.
Public Methods:
__init__
(api_response)__repr__
()Return repr(self).
__str__
()Return str(self).
Render this object as a Pandas DataFrame.
Inherited from
Record
match_all
(**kwargs)Whether attributes of this record match all the key/value arguments.
match_any
(**kwargs)Whether attributes of this record match any of the key/value arguments.
Inherited from
ForPandas
Render this object as a Pandas DataFrame.
- property articleguids¶
List of RiskIQ OSINT article GUIDs associated with this indicator.
- property as_df¶
Get this object as a Pandas DataFrame.
Use to_dataframe() instead if you need to control how the dataframe is built.
Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype:
pandas.DataFrame
- property as_dict¶
Return a dictionary representation of the object.
- property category¶
Indicator category.
- property duration¶
Length of time record was observed, in days.
Calculates the timedelta between firstseen and lastseen.
- Return type
- property firstseen¶
Date & time the record was first seen.
- Return type
datetime
- property firstseen_date¶
Date record was first seen.
- Return type
date
- property firstseen_raw¶
Raw firstseen value returned by the API.
- property id¶
RiskIQ identifier for this indicator.
- property intel_profile¶
RiskIQ threat intel profile associated with this indicator.
- property is_osint¶
Whether this indicator was published in open source intelligence articles.
- property lastseen¶
Date & time the record was most recently observed.
- Return type
datetime
- property lastseen_date¶
Date the record was most recently observed.
- Return type
date
- property lastseen_raw¶
Raw lastseen value returned by the API.
- match_all(**kwargs)¶
Whether attributes of this record match all the key/value arguments.
- match_any(**kwargs)¶
Whether attributes of this record match any of the key/value arguments.
- property osint_link¶
URL for the OSINT source of the indicator, or none if this is not an OSINT indicator.
- property pretty¶
Pretty printed version of this object’s dictionary representation.
- property profile_id¶
RiskIQ identifier for the intel profile associated with this indicator.
- to_dataframe()¶
Render this object as a Pandas DataFrame.
- Return type
pandas.DataFrame
- property type¶
Type of indicator.
- property value¶
Value of the indicator.
Vulnerability Intelligence¶
RiskIQ’s Vulnerability Intelligence (Vuln Intel) provides a practical picture of vulnerability risk, focused on a specific Attack Surface (your own or a third-party vendor). It returns a list of “CVEs” (Common Vulnerabilities and Exposures) each identified by a name and offering a list of assets known to be vulnerable to the exploits or weaknesses described in the vuln report.
CVEs for your Attack Surface¶
In the analyzer module, Vuln Intel is offered primarily through the cves
property of an
Attack Surface.
>>> cves = analyzer.AttackSurface().cves
>>> for cve in cves:
... print(cve)
Each record can be printed as a string, but like other analyzer
objects, it offers a rich
set of properties to display and iterate through the list of CVEs. See the reference for the
passivetotal.analyzer.illuminate.vuln.AttackSurfaceCVEs
object that represents the
list of CVEs, and reference the
passivetotal.analyzer.illuminate.vuln.AttackSurfaceCVE
object for details on each CVE.
CVEs for Third Parties¶
Use the same cves
property of a third-party vendor attack surface to discover which
CVEs they may be vulnerable to. In this example, we load an attack surface for vendr ID
“12345”.
>>> cves = analyzer.AttackSurface(12345).cves
>>> for cve in cves:
... print(cve)
CVE Observations (Assets)¶
Each AttackSurfaceCVE object offers an observations
property that delivers a list of
assets (typically IPs or hosts) within a given attack surface that are known to be impacted
by the CVE. The list is returned as type
passivetotal.analyzer.illuminate.vuln.AttackSurfaceObservations
and contains a list of
passivetotal.analyzer.illuminate.vuln.AttackSurfaceObservation
objects.
>>> cves = analyzer.AttackSurface(12345).cves
>>> scored_cves = cves.filter_fn(lambda c: c.score > 80).sorted_by('score',True)
>>> highest_scored_cve = scored_cves[0]
>>> for observation in highest_scored_cve.observations:
... print(observation)
In this example, we used the filter_fn
method available on most list-like analyzer
objects
to apply a function similar to the Python filter()
method. This helps us find one CVE to focus
on, which will then give us a list of observations.
Vulnerability Articles¶
If you already know the identifier for a CVE article, you can access the complete details of the article, including the description, date published, other scores, and a top-level assessment of how exposed your attack surface is to the CVE.
Each article is of type
passivetotal.analyzer.illumiante.vuln.VulnArticle
>>> vuln_article = analyzer.illuminate.VulnArticle.load('CVE-2021-23017')
>>> print(vuln_article.description)
... 'This is a known weakness in...'
>>> print(vuln_article.observation_count)
130
>>> for observation in vuln_article.observations:
... print(observation.name, observation.firstseen, observation.lastseen)
Here, the observation_count
property gives us the number of assets in the primary
attack surface associated with our API key that are known to be impacted by this vulnerability.
We use the observation
property to obtain the list, which contains a list of
passivetotal.analyzer.illuminate.vuln.AttackSurfaceCVEObservation
objects.
For third-parties, use the impacts
property to see how many of your third-party vendors’
attack surfaces are impacted by the vulnerability. Each impacted attack surface provides an
observations
property to obtain the list of impacted assets.
>>> vuln_article = analyzer.illuminate.VulnArticle.load('CVE-2021-23017')
>>> for vendor in vuln_article.attack_surfaces:
... print(vendor)
>>> impacted_vendor = focus_article.attack_surfaces.filter_substring(vendor_name='union')[0]
>>> for observation in impacted_vendor.observations:
... print(observation.name, observation.firstseen, observation.lastseen)
Examples & Notebooks¶
The example Jupyter Notebook for Attack Surface Intelligence includes a section on how to access CVEs, observations and articles on the primary attack surface and third-party attack surfaces.
Vuln Reference¶
Collection of CVEs associated with an attack surface as a list-like object containing |
|
A record of a CVE associated with an asset in an attack surface. |
|
|
Collection of observations (assets) associated with a CVE in a specific attack surface as a list-like object containing |
|
An observation (asset) vulnerable to a specific CVE in a given attack surface. |
|
Collection of vulnerable components (detections) associated with an attack surface as a list-like object containing |
|
A vulnerable component (detection) observed in an attack surface. |
Vulnerabilty report providing details on impacted assets and third-party vendors. |
|
Collection of Illuminate Attack Surfaces potentially impacted by a vulnerability as a list-like object containing |
|
An impacted third-party attack surface with observations (assets) affected by a given vulnerabilty. |
- class passivetotal.analyzer.illuminate.vuln.AttackSurfaceCVEs(attack_surface=None, pagesize=400)¶
Collection of CVEs associated with an attack surface as a list-like object containing
AttackSurfaceCVE
objects.Public Data Attributes:
Get the Illuminate Attack Surface associated with this list of CVEs.
Inherited from
RecordList
All the records as a list.
Return the recordlist as a list of dictionary objects.
length
Inherited from
AsDictionary
Return the recordlist as a list of dictionary objects.
Pretty printed version of this object's dictionary representation.
Inherited from
PagedRecordList
Total number of available records as reported by the API.
Whether more records are available.
Inherited from
ForPandas
Get this object as a Pandas DataFrame.
Public Methods:
__init__
([attack_surface, pagesize])Inherited from
RecordList
__init__
([attack_surface, pagesize])__iter__
()__getitem__
(key)__len__
()parse
(api_response)Implementations must accept an API response and populate themselves with a list of the correct record types.
filter
(**kwargs)Shortcut for filter_and.
filter_fn
(fn)Return only records where a function returns true.
filter_and
(**kwargs)Return only records that match all key/value arguments.
filter_or
(**kwargs)Return only records that match any key/value arguments.
filter_in
(**kwargs)Return only records where a field contains one or more values.
filter_substring
(**kwargs)Return only records where a case-insensitive match on the field returns true.
filter_substring_in
(**kwargs)Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.
sorted_by
(field[, reverse])Return a sorted list.
filter_dateseen_after
(date_string)Filter only results where the firstseen date property is after a specified date.
filter_dateseen_before
(date_string)Filter only results where the lastseen date property is before a specified date.
filter_dateseen_between
(start_date_string, ...)Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.
Inherited from
PagedRecordList
Load the next page of results from the API.
Load all pages of results from the API.
Inherited from
ForPandas
to_dataframe
(**kwargs)Render this object as a Pandas DataFrame.
- property all¶
All the records as a list.
- property as_df¶
Get this object as a Pandas DataFrame.
Use to_dataframe() instead if you need to control how the dataframe is built.
Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype:
pandas.DataFrame
- property as_dict¶
Return the recordlist as a list of dictionary objects.
- property attack_surface¶
Get the Illuminate Attack Surface associated with this list of CVEs.
- filter(**kwargs)¶
Shortcut for filter_and.
- filter_and(**kwargs)¶
Return only records that match all key/value arguments.
- filter_dateseen_after(date_string: str)¶
Filter only results where the firstseen date property is after a specified date.
The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’
- filter_dateseen_before(date_string: str)¶
Filter only results where the lastseen date property is before a specified date.
The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’
- filter_dateseen_between(start_date_string: str, end_date_string: str)¶
Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.
Date strings should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’
- filter_fn(fn)¶
Return only records where a function returns true.
- filter_in(**kwargs)¶
Return only records where a field contains one or more values.
- Usage:
filter_in(fieldname=[‘value1’,’value2’]) or filter_in(fieldname=’value1,value2)
- filter_or(**kwargs)¶
Return only records that match any key/value arguments.
- filter_substring(**kwargs)¶
Return only records where a case-insensitive match on the field returns true.
- filter_substring_in(**kwargs)¶
Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.
- Usage:
filter_substring_in(fieldname=[‘substring’,’othersub’]) or filter_substring_in(fieldname=’substring,othersub’)
- load_all_pages()¶
Load all pages of results from the API.
- load_next_page()¶
Load the next page of results from the API.
Throws AnalyzerError when has_more_records is False.
- parse(api_response)¶
Implementations must accept an API response and populate themselves with a list of the correct record types.
- property pretty¶
Pretty printed version of this object’s dictionary representation.
- sorted_by(field, reverse=False)¶
Return a sorted list.
- Parameters
field – name of the attribute to sort on
reverse – whether to sort in reverse order.
- to_dataframe(**kwargs)¶
Render this object as a Pandas DataFrame.
Implementations may add additional keywords to customize building the data structure.
Default implementation tries to iterate through self and calls to_dataframe on each record with the same parameters passed to this method. If that fails (usually because self isn’t iterable), it uses the as_dict param of self.
- Return type
pandas.DataFrame
- property totalrecords¶
Total number of available records as reported by the API.
- class passivetotal.analyzer.illuminate.vuln.AttackSurfaceCVE(attack_surface, api_response={})¶
A record of a CVE associated with an asset in an attack surface.
Public Data Attributes:
CVE article with complete details on this vulnerability.
Attack surface this CVE is associated with.
Description of the CVE, retrieved from the vulnerability article associated with this CVE.
Publication date of the vulnerability article associated with this CVE.
List of CWE IDs for this CVE.
CVE identifier (alias for cve_id).
CVE identifier.
API link to get CVE article.
RiskIQ priority score for this CVE.
Number of observations (assets) in this attack surface impacted by this vulnerabilty.
List of observations (assets) in this attack surface vulnerable to this CVE.
Inherited from
AsDictionary
Return a dictionary representation of the object.
Pretty printed version of this object's dictionary representation.
Inherited from
ForPandas
Get this object as a Pandas DataFrame.
Public Methods:
__init__
(attack_surface[, api_response])__str__
()Return str(self).
__repr__
()Return repr(self).
Render this object as a Pandas dataframe.
get_observations
([pagesize])Get a list of observations(assets) vulnerable to this CVE in this attack surface.
Inherited from
Record
match_all
(**kwargs)Whether attributes of this record match all the key/value arguments.
match_any
(**kwargs)Whether attributes of this record match any of the key/value arguments.
Inherited from
ForPandas
Render this object as a Pandas dataframe.
- property article¶
CVE article with complete details on this vulnerability.
- property as_df¶
Get this object as a Pandas DataFrame.
Use to_dataframe() instead if you need to control how the dataframe is built.
Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype:
pandas.DataFrame
- property as_dict¶
Return a dictionary representation of the object.
- property attack_surface¶
Attack surface this CVE is associated with.
- property cve_id¶
CVE identifier.
- property cwes¶
List of CWE IDs for this CVE.
- property description¶
Description of the CVE, retrieved from the vulnerability article associated with this CVE.
- get_observations(pagesize=400)¶
Get a list of observations(assets) vulnerable to this CVE in this attack surface.
- Parameters
pagesize – Size of pages to retrieve from the API.
- Return type
- property id¶
CVE identifier (alias for cve_id).
- property link¶
API link to get CVE article.
Consider using the article property to access the article directly.
- match_all(**kwargs)¶
Whether attributes of this record match all the key/value arguments.
- match_any(**kwargs)¶
Whether attributes of this record match any of the key/value arguments.
- property observation_count¶
Number of observations (assets) in this attack surface impacted by this vulnerabilty.
- property observations¶
List of observations (assets) in this attack surface vulnerable to this CVE.
- property pretty¶
Pretty printed version of this object’s dictionary representation.
- property publish_date¶
Publication date of the vulnerability article associated with this CVE.
- property score¶
RiskIQ priority score for this CVE.
- to_dataframe()¶
Render this object as a Pandas dataframe.
- Return type
pandas.DataFrame
- class passivetotal.analyzer.illuminate.vuln.AttackSurfaceCVEObservations(cve=None, pagesize=400)¶
Collection of observations (assets) associated with a CVE in a specific attack surface as a list-like object containing
AttackSurfaceCVEObservation
objects.Public Data Attributes:
Get the CVE associated with this list of observations.
Inherited from
RecordList
All the records as a list.
Return the recordlist as a list of dictionary objects.
length
Inherited from
AsDictionary
Return the recordlist as a list of dictionary objects.
Pretty printed version of this object's dictionary representation.
Inherited from
PagedRecordList
Total number of available records as reported by the API.
Whether more records are available.
Inherited from
ForPandas
Get this object as a Pandas DataFrame.
Public Methods:
__init__
([cve, pagesize])Inherited from
RecordList
__init__
([cve, pagesize])__iter__
()__getitem__
(key)__len__
()parse
(api_response)Implementations must accept an API response and populate themselves with a list of the correct record types.
filter
(**kwargs)Shortcut for filter_and.
filter_fn
(fn)Return only records where a function returns true.
filter_and
(**kwargs)Return only records that match all key/value arguments.
filter_or
(**kwargs)Return only records that match any key/value arguments.
filter_in
(**kwargs)Return only records where a field contains one or more values.
filter_substring
(**kwargs)Return only records where a case-insensitive match on the field returns true.
filter_substring_in
(**kwargs)Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.
sorted_by
(field[, reverse])Return a sorted list.
filter_dateseen_after
(date_string)Filter only results where the firstseen date property is after a specified date.
filter_dateseen_before
(date_string)Filter only results where the lastseen date property is before a specified date.
filter_dateseen_between
(start_date_string, ...)Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.
Inherited from
PagedRecordList
Load the next page of results from the API.
Load all pages of results from the API.
Inherited from
ForPandas
to_dataframe
(**kwargs)Render this object as a Pandas DataFrame.
- property all¶
All the records as a list.
- property as_df¶
Get this object as a Pandas DataFrame.
Use to_dataframe() instead if you need to control how the dataframe is built.
Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype:
pandas.DataFrame
- property as_dict¶
Return the recordlist as a list of dictionary objects.
- property cve¶
Get the CVE associated with this list of observations.
- filter(**kwargs)¶
Shortcut for filter_and.
- filter_and(**kwargs)¶
Return only records that match all key/value arguments.
- filter_dateseen_after(date_string: str)¶
Filter only results where the firstseen date property is after a specified date.
The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’
- filter_dateseen_before(date_string: str)¶
Filter only results where the lastseen date property is before a specified date.
The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’
- filter_dateseen_between(start_date_string: str, end_date_string: str)¶
Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.
Date strings should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’
- filter_fn(fn)¶
Return only records where a function returns true.
- filter_in(**kwargs)¶
Return only records where a field contains one or more values.
- Usage:
filter_in(fieldname=[‘value1’,’value2’]) or filter_in(fieldname=’value1,value2)
- filter_or(**kwargs)¶
Return only records that match any key/value arguments.
- filter_substring(**kwargs)¶
Return only records where a case-insensitive match on the field returns true.
- filter_substring_in(**kwargs)¶
Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.
- Usage:
filter_substring_in(fieldname=[‘substring’,’othersub’]) or filter_substring_in(fieldname=’substring,othersub’)
- load_all_pages()¶
Load all pages of results from the API.
- load_next_page()¶
Load the next page of results from the API.
Throws AnalyzerError when has_more_records is False.
- parse(api_response)¶
Implementations must accept an API response and populate themselves with a list of the correct record types.
- property pretty¶
Pretty printed version of this object’s dictionary representation.
- sorted_by(field, reverse=False)¶
Return a sorted list.
- Parameters
field – name of the attribute to sort on
reverse – whether to sort in reverse order.
- to_dataframe(**kwargs)¶
Render this object as a Pandas DataFrame.
Implementations may add additional keywords to customize building the data structure.
Default implementation tries to iterate through self and calls to_dataframe on each record with the same parameters passed to this method. If that fails (usually because self isn’t iterable), it uses the as_dict param of self.
- Return type
pandas.DataFrame
- property totalrecords¶
Total number of available records as reported by the API.
- class passivetotal.analyzer.illuminate.vuln.AttackSurfaceCVEObservation(cve, api_response={})¶
An observation (asset) vulnerable to a specific CVE in a given attack surface.
Public Data Attributes:
Attack surface this observation is associated with.
CVE this observation is vulnerable to, in the context of a specific attack surface.
RiskIQ identifier for the CVE this observation is vulnerable to.
Type of this observation (asset).
Name of this observation (asset).
Inherited from
AsDictionary
Return a dictionary representation of the object.
Pretty printed version of this object's dictionary representation.
Inherited from
FirstLastSeen
Date & time the record was first seen.
Date record was first seen.
Raw firstseen value returned by the API.
Date & time the record was most recently observed.
Date the record was most recently observed.
Raw lastseen value returned by the API.
Length of time record was observed, in days.
Inherited from
ForPandas
Get this object as a Pandas DataFrame.
Public Methods:
__init__
(cve[, api_response])__str__
()Return str(self).
__repr__
()Return repr(self).
Render this object as a Pandas dataframe.
Inherited from
Record
match_all
(**kwargs)Whether attributes of this record match all the key/value arguments.
match_any
(**kwargs)Whether attributes of this record match any of the key/value arguments.
Inherited from
ForPandas
Render this object as a Pandas dataframe.
- property as_df¶
Get this object as a Pandas DataFrame.
Use to_dataframe() instead if you need to control how the dataframe is built.
Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype:
pandas.DataFrame
- property as_dict¶
Return a dictionary representation of the object.
- property attack_surface¶
Attack surface this observation is associated with.
- property cve¶
CVE this observation is vulnerable to, in the context of a specific attack surface.
- property cve_id¶
RiskIQ identifier for the CVE this observation is vulnerable to.
- property duration¶
Length of time record was observed, in days.
Calculates the timedelta between firstseen and lastseen.
- Return type
- property firstseen¶
Date & time the record was first seen.
- Return type
datetime
- property firstseen_date¶
Date record was first seen.
- Return type
date
- property firstseen_raw¶
Raw firstseen value returned by the API.
- property lastseen¶
Date & time the record was most recently observed.
- Return type
datetime
- property lastseen_date¶
Date the record was most recently observed.
- Return type
date
- property lastseen_raw¶
Raw lastseen value returned by the API.
- match_all(**kwargs)¶
Whether attributes of this record match all the key/value arguments.
- match_any(**kwargs)¶
Whether attributes of this record match any of the key/value arguments.
- property name¶
Name of this observation (asset).
- property pretty¶
Pretty printed version of this object’s dictionary representation.
- to_dataframe()¶
Render this object as a Pandas dataframe.
- Return type
pandas.DataFrame
- property type¶
Type of this observation (asset).
- class passivetotal.analyzer.illuminate.vuln.AttackSurfaceComponents(attack_surface=None, pagesize=400)¶
Collection of vulnerable components (detections) associated with an attack surface as a list-like object containing
AttackSurfaceComponent
objects.Public Data Attributes:
Get the attack surface associated with this list of observations.
Inherited from
RecordList
All the records as a list.
Return the recordlist as a list of dictionary objects.
length
Inherited from
AsDictionary
Return the recordlist as a list of dictionary objects.
Pretty printed version of this object's dictionary representation.
Inherited from
PagedRecordList
Total number of available records as reported by the API.
Whether more records are available.
Inherited from
ForPandas
Get this object as a Pandas DataFrame.
Public Methods:
__init__
([attack_surface, pagesize])Inherited from
RecordList
__init__
([attack_surface, pagesize])__iter__
()__getitem__
(key)__len__
()parse
(api_response)Implementations must accept an API response and populate themselves with a list of the correct record types.
filter
(**kwargs)Shortcut for filter_and.
filter_fn
(fn)Return only records where a function returns true.
filter_and
(**kwargs)Return only records that match all key/value arguments.
filter_or
(**kwargs)Return only records that match any key/value arguments.
filter_in
(**kwargs)Return only records where a field contains one or more values.
filter_substring
(**kwargs)Return only records where a case-insensitive match on the field returns true.
filter_substring_in
(**kwargs)Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.
sorted_by
(field[, reverse])Return a sorted list.
filter_dateseen_after
(date_string)Filter only results where the firstseen date property is after a specified date.
filter_dateseen_before
(date_string)Filter only results where the lastseen date property is before a specified date.
filter_dateseen_between
(start_date_string, ...)Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.
Inherited from
PagedRecordList
Load the next page of results from the API.
Load all pages of results from the API.
Inherited from
ForPandas
to_dataframe
(**kwargs)Render this object as a Pandas DataFrame.
- property all¶
All the records as a list.
- property as_df¶
Get this object as a Pandas DataFrame.
Use to_dataframe() instead if you need to control how the dataframe is built.
Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype:
pandas.DataFrame
- property as_dict¶
Return the recordlist as a list of dictionary objects.
- property attack_surface¶
Get the attack surface associated with this list of observations.
- filter(**kwargs)¶
Shortcut for filter_and.
- filter_and(**kwargs)¶
Return only records that match all key/value arguments.
- filter_dateseen_after(date_string: str)¶
Filter only results where the firstseen date property is after a specified date.
The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’
- filter_dateseen_before(date_string: str)¶
Filter only results where the lastseen date property is before a specified date.
The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’
- filter_dateseen_between(start_date_string: str, end_date_string: str)¶
Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.
Date strings should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’
- filter_fn(fn)¶
Return only records where a function returns true.
- filter_in(**kwargs)¶
Return only records where a field contains one or more values.
- Usage:
filter_in(fieldname=[‘value1’,’value2’]) or filter_in(fieldname=’value1,value2)
- filter_or(**kwargs)¶
Return only records that match any key/value arguments.
- filter_substring(**kwargs)¶
Return only records where a case-insensitive match on the field returns true.
- filter_substring_in(**kwargs)¶
Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.
- Usage:
filter_substring_in(fieldname=[‘substring’,’othersub’]) or filter_substring_in(fieldname=’substring,othersub’)
- load_all_pages()¶
Load all pages of results from the API.
- load_next_page()¶
Load the next page of results from the API.
Throws AnalyzerError when has_more_records is False.
- parse(api_response)¶
Implementations must accept an API response and populate themselves with a list of the correct record types.
- property pretty¶
Pretty printed version of this object’s dictionary representation.
- sorted_by(field, reverse=False)¶
Return a sorted list.
- Parameters
field – name of the attribute to sort on
reverse – whether to sort in reverse order.
- to_dataframe(**kwargs)¶
Render this object as a Pandas DataFrame.
Implementations may add additional keywords to customize building the data structure.
Default implementation tries to iterate through self and calls to_dataframe on each record with the same parameters passed to this method. If that fails (usually because self isn’t iterable), it uses the as_dict param of self.
- Return type
pandas.DataFrame
- property totalrecords¶
Total number of available records as reported by the API.
- class passivetotal.analyzer.illuminate.vuln.AttackSurfaceComponent(attack_surface, api_response={})¶
A vulnerable component (detection) observed in an attack surface.
Public Data Attributes:
Attack surface this component is associated with.
Type of this component (detection).
Name of this component (detection).
Severity of this detection.
Count.
Inherited from
AsDictionary
Return a dictionary representation of the object.
Pretty printed version of this object's dictionary representation.
Inherited from
FirstLastSeen
Date & time the record was first seen.
Date record was first seen.
Raw firstseen value returned by the API.
Date & time the record was most recently observed.
Date the record was most recently observed.
Raw lastseen value returned by the API.
Length of time record was observed, in days.
Inherited from
ForPandas
Get this object as a Pandas DataFrame.
Public Methods:
__init__
(attack_surface[, api_response])__str__
()Return str(self).
__repr__
()Return repr(self).
Render this object as a Pandas dataframe.
Inherited from
Record
match_all
(**kwargs)Whether attributes of this record match all the key/value arguments.
match_any
(**kwargs)Whether attributes of this record match any of the key/value arguments.
Inherited from
ForPandas
Render this object as a Pandas dataframe.
- property as_df¶
Get this object as a Pandas DataFrame.
Use to_dataframe() instead if you need to control how the dataframe is built.
Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype:
pandas.DataFrame
- property as_dict¶
Return a dictionary representation of the object.
- property attack_surface¶
Attack surface this component is associated with.
- property count¶
Count.
- property duration¶
Length of time record was observed, in days.
Calculates the timedelta between firstseen and lastseen.
- Return type
- property firstseen¶
Date & time the record was first seen.
- Return type
datetime
- property firstseen_date¶
Date record was first seen.
- Return type
date
- property firstseen_raw¶
Raw firstseen value returned by the API.
- property lastseen¶
Date & time the record was most recently observed.
- Return type
datetime
- property lastseen_date¶
Date the record was most recently observed.
- Return type
date
- property lastseen_raw¶
Raw lastseen value returned by the API.
- match_all(**kwargs)¶
Whether attributes of this record match all the key/value arguments.
- match_any(**kwargs)¶
Whether attributes of this record match any of the key/value arguments.
- property name¶
Name of this component (detection).
- property pretty¶
Pretty printed version of this object’s dictionary representation.
- property severity¶
Severity of this detection.
- to_dataframe()¶
Render this object as a Pandas dataframe.
- Return type
pandas.DataFrame
- property type¶
Type of this component (detection).
- class passivetotal.analyzer.illuminate.vuln.VulnArticle(id=None, api_response=None)¶
Vulnerabilty report providing details on impacted assets and third-party vendors.
Public Data Attributes:
CVE identifier string (alias for cve_id).
CVE identifier string.
Get a list of RiskIQ open-source intelligence articles that reference this vulnerability.
Narrative description of the CVE.
List of CWE IDs.
RiskIQ-assigned priority score for this vulnerability, ranging between 0 and 100.
The CVSS2 score for this vulnerability.
The CVSSS3 score for this vulnerability.
The date the article was published.
The raw (string) value returned from the API with the date the article was published.
The date the article was created.
The raw (string) value returned from the API with the date the article was created.
The date the article was updated by the publisher.
The raw (string) value returned from the API with the date the article was updated by the publisher.
List of references for this article.
List of components (detections) RiskIQ will search for to determine if assets are impacted by this vulnerability.
List of Illuminate Attack Surfaces (aka third-party vendors) with assets impacted by this vulnerability.
List of all attack surfaces associated with your API account that are impacted by this vulnerability (they have at least one observation of an impacted asset).
Number of observations (assets) within the primary attack surface that are impacted by this vulnerability.
List of observations (assets) within the primary attack surface that are impacted by this vulnerability.
Inherited from
AsDictionary
Return a dictionary representation of the object.
Pretty printed version of this object's dictionary representation.
Inherited from
ForPandas
Get this object as a Pandas DataFrame.
Public Methods:
__new__
(cls[, id, api_response])__repr__
()Return repr(self).
__str__
()Return str(self).
to_dataframe
([view])Render this object as a Pandas dataframe.
load
(id)Load a Vulnerability Article by ID.
Inherited from
Record
match_all
(**kwargs)Whether attributes of this record match all the key/value arguments.
match_any
(**kwargs)Whether attributes of this record match any of the key/value arguments.
Inherited from
ForPandas
to_dataframe
([view])Render this object as a Pandas dataframe.
- property as_df¶
Get this object as a Pandas DataFrame.
Use to_dataframe() instead if you need to control how the dataframe is built.
Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype:
pandas.DataFrame
- property as_dict¶
Return a dictionary representation of the object.
- property attack_surfaces¶
List of Illuminate Attack Surfaces (aka third-party vendors) with assets impacted by this vulnerability.
- property components¶
List of components (detections) RiskIQ will search for to determine if assets are impacted by this vulnerability.
- property cve_id¶
CVE identifier string.
- property cvss2score¶
The CVSS2 score for this vulnerability.
- property cvss3score¶
The CVSSS3 score for this vulnerability.
- property cwes¶
List of CWE IDs.
- property date_created¶
The date the article was created.
- property date_created_raw¶
The raw (string) value returned from the API with the date the article was created.
- property date_published¶
The date the article was published.
- property date_published_raw¶
The raw (string) value returned from the API with the date the article was published.
- property date_publisher_updated¶
The date the article was updated by the publisher.
- property date_publisherupdate_raw¶
The raw (string) value returned from the API with the date the article was updated by the publisher.
- property description¶
Narrative description of the CVE.
- property id¶
CVE identifier string (alias for cve_id).
- property impacted_attack_surfaces¶
List of all attack surfaces associated with your API account that are impacted by this vulnerability (they have at least one observation of an impacted asset).
- static load(id)¶
Load a Vulnerability Article by ID.
- Return type
- match_all(**kwargs)¶
Whether attributes of this record match all the key/value arguments.
- match_any(**kwargs)¶
Whether attributes of this record match any of the key/value arguments.
- property observation_count¶
Number of observations (assets) within the primary attack surface that are impacted by this vulnerability.
- property observations¶
List of observations (assets) within the primary attack surface that are impacted by this vulnerability.
- Return type
- property osint_articles¶
Get a list of RiskIQ open-source intelligence articles that reference this vulnerability.
- Return type
- property pretty¶
Pretty printed version of this object’s dictionary representation.
- property references¶
List of references for this article.
- property score¶
RiskIQ-assigned priority score for this vulnerability, ranging between 0 and 100.
- to_dataframe(view='info')¶
Render this object as a Pandas dataframe.
- Parameters
view – View to generate (info, references, components, or impacts)
- Return type
pandas.DataFrame
- class passivetotal.analyzer.illuminate.vuln.VulnArticleImpacts(article=None, impacts=[])¶
Collection of Illuminate Attack Surfaces potentially impacted by a vulnerability as a list-like object containing
VulnArticleImpact
objects.Public Data Attributes:
Article that describes the vulnerability impacting these attack surfaces.
Unfiltered list of all attack surfaces associated with your API account.
CVE identifier for the vulnerability this article applies to.
Filter the list to include only articles with at least one observation (asset).
Number of attack surfaces with assets impacted by this vulnerability.
Inherited from
RecordList
All the records as a list.
Return the recordlist as a list of dictionary objects.
length
Inherited from
AsDictionary
Return the recordlist as a list of dictionary objects.
Pretty printed version of this object's dictionary representation.
Inherited from
ForPandas
Get this object as a Pandas DataFrame.
Public Methods:
__init__
([article, impacts])__repr__
()Return repr(self).
__str__
()Return str(self).
Inherited from
RecordList
__init__
([article, impacts])__iter__
()__getitem__
(key)__len__
()parse
(api_response)Implementations must accept an API response and populate themselves with a list of the correct record types.
filter
(**kwargs)Shortcut for filter_and.
filter_fn
(fn)Return only records where a function returns true.
filter_and
(**kwargs)Return only records that match all key/value arguments.
filter_or
(**kwargs)Return only records that match any key/value arguments.
filter_in
(**kwargs)Return only records where a field contains one or more values.
filter_substring
(**kwargs)Return only records where a case-insensitive match on the field returns true.
filter_substring_in
(**kwargs)Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.
sorted_by
(field[, reverse])Return a sorted list.
filter_dateseen_after
(date_string)Filter only results where the firstseen date property is after a specified date.
filter_dateseen_before
(date_string)Filter only results where the lastseen date property is before a specified date.
filter_dateseen_between
(start_date_string, ...)Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.
Inherited from
ForPandas
to_dataframe
(**kwargs)Render this object as a Pandas DataFrame.
- property all¶
All the records as a list.
- property article¶
Article that describes the vulnerability impacting these attack surfaces.
- Return type
- property as_df¶
Get this object as a Pandas DataFrame.
Use to_dataframe() instead if you need to control how the dataframe is built.
Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype:
pandas.DataFrame
- property as_dict¶
Return the recordlist as a list of dictionary objects.
- property attack_surfaces¶
Unfiltered list of all attack surfaces associated with your API account.
This is the same list you would get if you iterated through an instance of this object. Each record will include a count of assets impacted by a vulnerability, in the “observation_count” property. The count may be zero.
If you need a list of only impacted attack surfaces, use the only_impacted property instead.
- property cve_id¶
CVE identifier for the vulnerability this article applies to.
- filter(**kwargs)¶
Shortcut for filter_and.
- filter_and(**kwargs)¶
Return only records that match all key/value arguments.
- filter_dateseen_after(date_string: str)¶
Filter only results where the firstseen date property is after a specified date.
The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’
- filter_dateseen_before(date_string: str)¶
Filter only results where the lastseen date property is before a specified date.
The date should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’
- filter_dateseen_between(start_date_string: str, end_date_string: str)¶
Filter only results where the lastseen date property is <= end_date_string and firstseen date property is on or after start_date_string.
Date strings should be parseable by datetime.fromisoformat i.e. ‘2021-01-01’
- filter_fn(fn)¶
Return only records where a function returns true.
- filter_in(**kwargs)¶
Return only records where a field contains one or more values.
- Usage:
filter_in(fieldname=[‘value1’,’value2’]) or filter_in(fieldname=’value1,value2)
- filter_or(**kwargs)¶
Return only records that match any key/value arguments.
- filter_substring(**kwargs)¶
Return only records where a case-insensitive match on the field returns true.
- filter_substring_in(**kwargs)¶
Return only records where a case-insensitive match on any item in a list of substrings returns true for a specific field.
- Usage:
filter_substring_in(fieldname=[‘substring’,’othersub’]) or filter_substring_in(fieldname=’substring,othersub’)
- property impact_count¶
Number of attack surfaces with assets impacted by this vulnerability.
Counts the number of records that have more than zero observations.
- property only_impacted¶
Filter the list to include only articles with at least one observation (asset).
- Return type
- parse(api_response)¶
Implementations must accept an API response and populate themselves with a list of the correct record types.
- property pretty¶
Pretty printed version of this object’s dictionary representation.
- sorted_by(field, reverse=False)¶
Return a sorted list.
- Parameters
field – name of the attribute to sort on
reverse – whether to sort in reverse order.
- to_dataframe(**kwargs)¶
Render this object as a Pandas DataFrame.
Implementations may add additional keywords to customize building the data structure.
Default implementation tries to iterate through self and calls to_dataframe on each record with the same parameters passed to this method. If that fails (usually because self isn’t iterable), it uses the as_dict param of self.
- Return type
pandas.DataFrame
- class passivetotal.analyzer.illuminate.vuln.VulnArticleImpact(article, api_response=None)¶
An impacted third-party attack surface with observations (assets) affected by a given vulnerabilty.
Public Data Attributes:
Article that describes the vulnerability this observation (asset) is impacted by.
Name of the vendor with observations (assets) impacted by this vulnerability.
The RiskIQ-assigned identifier for this vendor.
Illuminate Attack Surface for the third-party vendor impacted by this vulnerability.
Number of observations (assets) within a vendor's attack surface that are impacted by this vulnerability.
List of observations (assets) within this vendor's attack surface that are impacted by this vulnerability.
Inherited from
AsDictionary
Return a dictionary representation of the object.
Pretty printed version of this object's dictionary representation.
Inherited from
ForPandas
Get this object as a Pandas DataFrame.
Public Methods:
__init__
(article[, api_response])__repr__
()Return repr(self).
__str__
()Return str(self).
Inherited from
Record
match_all
(**kwargs)Whether attributes of this record match all the key/value arguments.
match_any
(**kwargs)Whether attributes of this record match any of the key/value arguments.
Inherited from
ForPandas
to_dataframe
(**kwargs)Render this object as a Pandas DataFrame.
- property article¶
Article that describes the vulnerability this observation (asset) is impacted by.
- property as_df¶
Get this object as a Pandas DataFrame.
Use to_dataframe() instead if you need to control how the dataframe is built.
Requires the pandas Python library. Throws AnalyzerError if it is missing. :rtype:
pandas.DataFrame
- property as_dict¶
Return a dictionary representation of the object.
- property attack_surface¶
Illuminate Attack Surface for the third-party vendor impacted by this vulnerability.
- match_all(**kwargs)¶
Whether attributes of this record match all the key/value arguments.
- match_any(**kwargs)¶
Whether attributes of this record match any of the key/value arguments.
- property observation_count¶
Number of observations (assets) within a vendor’s attack surface that are impacted by this vulnerability.
- property observations¶
List of observations (assets) within this vendor’s attack surface that are impacted by this vulnerability.
- property pretty¶
Pretty printed version of this object’s dictionary representation.
- to_dataframe(**kwargs)¶
Render this object as a Pandas DataFrame.
Implementations may add additional keywords to customize building the data structure.
Default implementation tries to iterate through self and calls to_dataframe on each record with the same parameters passed to this method. If that fails (usually because self isn’t iterable), it uses the as_dict param of self.
- Return type
pandas.DataFrame
- property vendor_id¶
The RiskIQ-assigned identifier for this vendor.
- property vendor_name¶
Name of the vendor with observations (assets) impacted by this vulnerability.