Sensitivity Analysis

Pipeline Complete Analysis

class fedot.sensitivity.pipeline_sensitivity_facade.PipelineSensitivityAnalysis(pipeline: fedot.core.pipelines.pipeline.Pipeline, train_data: fedot.core.data.data.InputData, test_data: fedot.core.data.data.InputData, approaches: Optional[List[Union[Type[fedot.sensitivity.node_sa_approaches.NodeAnalyzeApproach], Type[fedot.sensitivity.operations_hp_sensitivity.multi_operations_sensitivity.MultiOperationsHPAnalyze]]]] = None, nodes_to_analyze: Optional[List[fedot.core.pipelines.node.PipelineNode]] = None, requirements: Optional[fedot.sensitivity.sa_requirements.SensitivityAnalysisRequirements] = None, path_to_save=None)[source]

Bases: object

This class works as facade and allows to apply all kind of approaches to whole pipeline and separate nodes together

Parameters

  • pipeline – pipeline object to analyze

  • train_data – data used for Pipeline training

  • test_data – data used for Pipeline validation

  • approaches – methods applied to pipeline. Default: None

  • nodes_to_analyze – nodes to analyze. Default: all nodes

  • requirements – extra requirements to define specific details for different approaches. See SensitivityAnalysisRequirements class documentation

  • path_to_save – path to save results to. Default: ~home/Fedot/sensitivity/

analyze()[source]

Applies defined sensitivity analysis approaches

Pipeline Structure Analysis

class fedot.sensitivity.nodes_sensitivity.NodesAnalysis(pipeline: fedot.core.pipelines.pipeline.Pipeline, train_data: fedot.core.data.data.InputData, test_data: fedot.core.data.data.InputData, approaches: Optional[List[Type[fedot.sensitivity.node_sa_approaches.NodeAnalyzeApproach]]] = None, requirements: Optional[fedot.sensitivity.sa_requirements.SensitivityAnalysisRequirements] = None, path_to_save=None, nodes_to_analyze: Optional[List[fedot.core.pipelines.node.PipelineNode]] = None)[source]

Bases: object

This class is for nodes sensitivity analysis within a Pipeline

It takes nodes and approaches to be applied to chosen nodes.

To define which nodes to analyze pass them to nodes_to_analyze filed or all nodes will be analyzed.

Parameters

  • pipeline – pipeline object to analyze

  • train_data – data used for Pipeline training

  • test_data – data used for Pipeline validation

  • approaches –

    methods applied to nodes to modify the pipeline or analyze certain operations.

    Default: [NodeDeletionAnalyze, NodeReplaceOperationAnalyze]

  • nodes_to_analyze – nodes to analyze. Default: all nodes

  • path_to_save – path to save results to. Default: ~home/Fedot/sensitivity

analyze() → dict[source]

Main method to run the analyze process for every node.

Returns

with analysis result per Node

Return type

dict

fedot.sensitivity.nodes_sensitivity.get_nodes_degrees(graph: Graph) → Sequence[int][source]
Nodes degree as the number of edges the node has:

degree = #input_edges + #out_edges

Returns

nodes degrees ordered according to the nx_graph representation of this graph

Pipeline Non-Structure Analysis

class fedot.sensitivity.pipeline_sensitivity.PipelineAnalysis(pipeline: fedot.core.pipelines.pipeline.Pipeline, train_data: fedot.core.data.data.InputData, test_data: fedot.core.data.data.InputData, approaches: Optional[List[Type[fedot.sensitivity.operations_hp_sensitivity.multi_operations_sensitivity.MultiOperationsHPAnalyze]]] = None, requirements: Optional[fedot.sensitivity.sa_requirements.SensitivityAnalysisRequirements] = None, path_to_save=None)[source]

Bases: object

This class is for analyzing the Pipeline as the black-box model, using analysis approaches defined for whole pipeline perturbation, i.e. MultiOperationsHPAnalyze

Parameters

  • pipeline – pipeline object to analyze

  • train_data – data used for Pipeline training

  • test_data – data used for Pipeline validation

  • approaches – methods applied to pipeline Default: MultiOperationsHPAnalyze

  • requirements – extra requirements to define specific details for different approaches See SensitivityAnalysisRequirements class documentation

  • path_to_save – path to save results to. Default: ~home/Fedot/sensitivity/pipeline_sa

analyze() → dict[source]

Apply defined approaches for the black-box pipeline analysis

Node Structure Analysis

class fedot.sensitivity.node_sa_approaches.NodeAnalysis(approaches: Optional[List[Type[fedot.sensitivity.node_sa_approaches.NodeAnalyzeApproach]]] = None, approaches_requirements: Optional[fedot.sensitivity.sa_requirements.SensitivityAnalysisRequirements] = None, path_to_save=None)[source]

Bases: object

Parameters

  • approaches – methods applied to nodes to modify the pipeline or analyze certain operations. Default: [NodeDeletionAnalyze, NodeTuneAnalyze, NodeReplaceOperationAnalyze]

  • path_to_save – path to save results to. Default: ~home/Fedot/sensitivity

analyze(pipeline: fedot.core.pipelines.pipeline.Pipeline, node: fedot.core.pipelines.node.PipelineNode, train_data: fedot.core.data.data.InputData, test_data: fedot.core.data.data.InputData, is_save: bool = False) → dict[source]

Method runs Node analysis within defined approaches

Parameters

  • is_save – whether the certain node analysis result is needed to ba saved

  • pipeline – Pipeline containing the analyzed Node

  • node – Node object to analyze in Pipeline

  • train_data – data used for Pipeline training

  • test_data – data used for Pipeline validation

Returns

Node analysis result per approach

Return type

dict

class fedot.sensitivity.node_sa_approaches.NodeAnalyzeApproach(pipeline: fedot.core.pipelines.pipeline.Pipeline, train_data, test_data: fedot.core.data.data.InputData, requirements: Optional[fedot.sensitivity.sa_requirements.SensitivityAnalysisRequirements] = None, path_to_save=None)[source]

Bases: abc.ABC

Base class for analysis approach.

Parameters

  • pipeline – Pipeline containing the analyzed Node

  • train_data – data used for Pipeline training

  • test_data – data used for Pipeline validation

  • path_to_save – path to save results to. Default: ~home/Fedot/sensitivity

abstract analyze(node: fedot.core.pipelines.node.PipelineNode, **kwargs) → Union[List[dict], List[float]][source]

Creates the difference metric(scorer, index, etc) of the changed graph in relation to the original one

Parameters

node – the sequence number of the node as in DFS result

abstract sample(*args) → Union[List[fedot.core.pipelines.pipeline.Pipeline], fedot.core.pipelines.pipeline.Pipeline][source]

Changes the pipeline according to the approach

class fedot.sensitivity.node_sa_approaches.NodeDeletionAnalyze(pipeline: fedot.core.pipelines.pipeline.Pipeline, train_data: fedot.core.data.data.InputData, test_data: fedot.core.data.data.InputData, requirements: Optional[fedot.sensitivity.sa_requirements.SensitivityAnalysisRequirements] = None, path_to_save=None)[source]

Bases: fedot.sensitivity.node_sa_approaches.NodeAnalyzeApproach

analyze(node: fedot.core.pipelines.node.PipelineNode, **kwargs) → Union[List[dict], List[float]][source]
Parameters

node – Node object to analyze

Returns

the ratio of modified pipeline score to origin score

sample(node: fedot.core.pipelines.node.PipelineNode)[source]
Parameters

node – Node object to delete from Pipeline object

Retuens:

Pipeline: pipeline without node

class fedot.sensitivity.node_sa_approaches.NodeReplaceOperationAnalyze(pipeline: fedot.core.pipelines.pipeline.Pipeline, train_data: fedot.core.data.data.InputData, test_data: fedot.core.data.data.InputData, requirements: Optional[fedot.sensitivity.sa_requirements.SensitivityAnalysisRequirements] = None, path_to_save=None)[source]

Bases: fedot.sensitivity.node_sa_approaches.NodeAnalyzeApproach

Replace node with operations available for the current task and evaluate the score difference

analyze(node: fedot.core.pipelines.node.PipelineNode, **kwargs) → Union[List[dict], List[float]][source]
Parameters

node – Node object to analyze

Returns

the ratio of modified pipeline score to origin score

sample(node: fedot.core.pipelines.node.PipelineNode, nodes_to_replace_to: Optional[List[fedot.core.pipelines.node.PipelineNode]], number_of_random_operations: Optional[int] = None) → Union[List[fedot.core.pipelines.pipeline.Pipeline], fedot.core.pipelines.pipeline.Pipeline][source]
Parameters

  • node – Node object to replace

  • nodes_to_replace_to – nodes provided for old_node replacement

  • number_of_random_operations – number of replacement operations, if nodes_to_replace_to not provided

Returns

sequence of Pipeline objects with new operations instead of old one

Return type

Union[List[Pipeline], Pipeline]

Hyperparameters Analysis

class fedot.sensitivity.operations_hp_sensitivity.multi_operations_sensitivity.MultiOperationsHPAnalyze(pipeline: fedot.core.pipelines.pipeline.Pipeline, train_data: fedot.core.data.data.InputData, test_data: fedot.core.data.data.InputData, requirements: Optional[fedot.sensitivity.sa_requirements.SensitivityAnalysisRequirements] = None, path_to_save=None)[source]

Bases: object

Provides with analysis of all the Pipeline’s operations hyperparameters using sample and analyze methods from SALib

Parameters

  • pipeline – Pipeline object to analyze

  • train_data – data used for Pipeline training

  • test_data – data used for Pipeline validation

  • requirements – extra requirements to define specific details for different approaches. See SensitivityAnalysisRequirements class documentation.

  • path_to_save – path to save results to. Default: ~home/Fedot/sensitivity/

analyze() → dict[source]

Analyze all the hyperparameters af all Pipeline operations using SA methods.

Default: Sobol method with Saltelli sample algorithm

Returns

Main and total Sobol indices for every parameter per node

Return type

dict

sample(*args) → Union[List[fedot.core.pipelines.pipeline.Pipeline], fedot.core.pipelines.pipeline.Pipeline][source]

Makes hyperparameters samples

Parameters

args – i.e. sample_size

Returns

new sampled hyperparameters

Return type

List[Pipeline]

class fedot.sensitivity.operations_hp_sensitivity.one_operation_sensitivity.OneOperationHPAnalyze(pipeline: fedot.core.pipelines.pipeline.Pipeline, train_data, test_data: fedot.core.data.data.InputData, requirements: Optional[fedot.sensitivity.sa_requirements.SensitivityAnalysisRequirements] = None, path_to_save=None)[source]

Bases: fedot.sensitivity.node_sa_approaches.NodeAnalyzeApproach

analyze(node: fedot.core.pipelines.node.PipelineNode, is_dispersion_analysis: bool = False) → Union[dict, float][source]

Creates the difference metric(scorer, index, etc) of the changed graph in relation to the original one

Parameters

node – the sequence number of the node as in DFS result

sample(*args) → Union[List[fedot.core.pipelines.pipeline.Pipeline], fedot.core.pipelines.pipeline.Pipeline][source]

Sensitivity Analysis Requirements

class fedot.sensitivity.sa_requirements.HyperparamsAnalysisMetaParams(analyze_method, sample_method, sample_size)

Bases: tuple

analyze_method

Alias for field number 0

sample_method

Alias for field number 1

sample_size

Alias for field number 2

class fedot.sensitivity.sa_requirements.ReplacementAnalysisMetaParams(nodes_to_replace_to, number_of_random_operations)

Bases: tuple

nodes_to_replace_to

Alias for field number 0

number_of_random_operations

Alias for field number 1

class fedot.sensitivity.sa_requirements.SensitivityAnalysisRequirements(metric=None, hyperparams_analyze_method: str = 'sobol', hyperparams_sample_method: str = 'saltelli', hyperparams_analysis_samples_size: int = 100, replacement_nodes_to_replace_to: Optional[List[fedot.core.pipelines.node.PipelineNode]] = None, replacement_number_of_random_operations: Optional[int] = None, visualization: bool = True, is_save_results_to_json: bool = True)[source]

Bases: object

Use this object to pass all the requirements needed for SA

Parameters

  • hyperparams_analyze_method – defines string name of SA method to use. Defaults: sobol

  • hyperparams_sample_method – defines string name of sampling method to use. Defaults: saltelli

  • hyperparams_analysis_samples_size – defines the number of shyperparameters samples used in SA

  • replacement_nodes_to_replace_to – defines nodes which is used in replacement analysis.

  • replacement_number_of_random_operations – if replacement_nodes_to_replace_to is not filled, define the number of randomly chosen operations used in replacement analysis.

  • visualization – defines whether the SA visualization needs to be saved to .png files.

  • is_save_results_to_json – defines whether the SA indices needs to be saved to .json file.

  • metric – metric used for validation. Default: see MetricByTask