Sensitivity Analysis

Pipeline Complete Analysis

class fedot.sensitivity.pipeline_sensitivity_facade.PipelineSensitivityAnalysis(pipeline, train_data, test_data, approaches=None, nodes_to_analyze=None, requirements=None, path_to_save=None, log=None)

Bases: object

This class works as facade and allows to apply all kind of approaches to whole pipeline and separate nodes together.

Parameters

Default: False :param log: log: Log object to record messages

analyze()

Applies defined sensitivity analysis approaches

Pipeline Structure Analysis

class fedot.sensitivity.nodes_sensitivity.NodesAnalysis(pipeline, train_data, test_data, approaches=None, requirements=None, path_to_save=None, log=None, nodes_to_analyze=None)

Bases: object

This class is for nodes sensitivity analysis within a Pipeline . It takes nodes and approaches to be applied to chosen nodes. To define which nodes to analyze pass them to nodes_to_analyze filed or all nodes will be analyzed.

Parameters
analyze()

Main method to run the analyze process for every node.

Return nodes_results

dict with analysis result per Node

Return type

dict

Pipeline Non-Structure Analysis

class fedot.sensitivity.pipeline_sensitivity.PipelineAnalysis(pipeline, train_data, test_data, approaches=None, requirements=None, path_to_save=None, log=None)

Bases: object

This class is for analyzing the Pipeline as the black-box model, using analysis approaches defined for whole pipeline perturbation, i.e. MultiOperationsHPAnalyze.

Parameters

Default: False :param log: log: Log object to record messages

analyze()

Apply defined approaches for the black-box pipeline analysis :return:

Return type

dict

Node Structure Analysis

class fedot.sensitivity.node_sa_approaches.NodeAnalysis(approaches=None, approaches_requirements=None, path_to_save=None, log=None)

Bases: object

Parameters
analyze(pipeline, node, train_data, test_data, is_save=False)

Method runs Node analysis within defined approaches

Parameters
Returns

dict with Node analysis result per approach

Return type

dict

class fedot.sensitivity.node_sa_approaches.NodeAnalyzeApproach(pipeline, train_data, test_data, requirements=None, path_to_save=None, log=None)

Bases: abc.ABC

Base class for analysis approach.

Parameters
abstract analyze(node, **kwargs)

Creates the difference metric(scorer, index, etc) of the changed graph in relation to the original one

Parameters

node (fedot.core.pipelines.node.Node) – the sequence number of the node as in DFS result

Return type

Union[List[dict], List[float]]

abstract sample(*args)

Changes the pipeline according to the approach

Return type

Union[List[fedot.core.pipelines.pipeline.Pipeline], fedot.core.pipelines.pipeline.Pipeline]

class fedot.sensitivity.node_sa_approaches.NodeDeletionAnalyze(pipeline, train_data, test_data, requirements=None, path_to_save=None)

Bases: fedot.sensitivity.node_sa_approaches.NodeAnalyzeApproach

Parameters
analyze(node, **kwargs)
Parameters

node (fedot.core.pipelines.node.Node) – Node object to analyze

Returns

the ratio of modified pipeline score to origin score

Return type

Union[List[dict], List[float]]

sample(node)
Parameters

node (fedot.core.pipelines.node.Node) – Node object to delete from Pipeline object

Returns

Pipeline object without node

class fedot.sensitivity.node_sa_approaches.NodeReplaceOperationAnalyze(pipeline, train_data, test_data, requirements=None, path_to_save=None)

Bases: fedot.sensitivity.node_sa_approaches.NodeAnalyzeApproach

Replace node with operations available for the current task and evaluate the score difference

Parameters
analyze(node, **kwargs)
Parameters

node (fedot.core.pipelines.node.Node) – Node object to analyze

Returns

the ratio of modified pipeline score to origin score

Return type

Union[List[dict], List[float]]

sample(node, nodes_to_replace_to, number_of_random_operations=None)
Parameters
Returns

Sequence of Pipeline objects with new operations instead of old one.

Return type

Union[List[fedot.core.pipelines.pipeline.Pipeline], fedot.core.pipelines.pipeline.Pipeline]

Hyperparameters Analysis

class fedot.sensitivity.operations_hp_sensitivity.multi_operations_sensitivity.MultiOperationsHPAnalyze(pipeline, train_data, test_data, requirements=None, path_to_save=None, log=None)

Bases: object

Provides with analysis of all the Pipeline’s operations hyperparameters using sample and analyze methods from SALib.

Parameters

Default: False :param log: log: Log object to record messages

analyze()

Analyze all the hyperparameters af all Pipeline operations using SA methods. Default: Sobol method with Saltelli sample algorithm :return: Main and total Sobol indices for every parameter per node.

Return type

dict

sample(*args)

Makes hyperparameters samples

Parameters

args – i.e. sample_size

Return List[Pipeline]

List of Pipelines with new sampled hyperparameters

Return type

Union[List[fedot.core.pipelines.pipeline.Pipeline], fedot.core.pipelines.pipeline.Pipeline]

static convert_results_to_json(problem, si)
Parameters

  • problem (fedot.sensitivity.operations_hp_sensitivity.problem.MultiOperationsProblem) –

  • si (dict) –

Return type

dict

Sensitivity Analysis Requirements

class fedot.sensitivity.sa_requirements.HyperparamsAnalysisMetaParams(analyze_method, sample_method, sample_size)

Bases: tuple

property analyze_method

Alias for field number 0

property sample_method

Alias for field number 1

property sample_size

Alias for field number 2

class fedot.sensitivity.sa_requirements.ReplacementAnalysisMetaParams(nodes_to_replace_to, number_of_random_operations)

Bases: tuple

property nodes_to_replace_to

Alias for field number 0

property number_of_random_operations

Alias for field number 1

class fedot.sensitivity.sa_requirements.SensitivityAnalysisRequirements(metric=None, hyperparams_analyze_method='sobol', hyperparams_sample_method='saltelli', hyperparams_analysis_samples_size=100, replacement_nodes_to_replace_to=None, replacement_number_of_random_operations=None, is_visualize=True, is_save_results_to_json=True)

Bases: object

Use this object to pass all the requirements needed for SA.

Parameters

  • hyperparams_analyze_method (str) – defines string name of SA method to use. Defaults: ‘sobol’

  • hyperparams_sample_method (str) – defines string name of sampling method to use. Defaults: ‘saltelli’

  • hyperparams_analysis_samples_size (int) – defines the number of shyperparameters samples used in SA

  • replacement_nodes_to_replace_to (Optional[List[fedot.core.pipelines.node.Node]]) – defines nodes which is used in replacement analysis.

  • replacement_number_of_random_operations (Optional[int]) – if replacement_nodes_to_replace_to is not filled, define the number of randomly chosen operations used in replacement analysis.

  • is_visualize (bool) – defines whether the SA visualization needs to be saved to .png files.

  • is_save_results_to_json (bool) – defines whether the SA indices needs to be saved to .json file.

  • metric – metric used for validation. Default: see MetricByTask