Sensitivity Analysis¶
Pipeline Complete Analysis¶
-
class
fedot.sensitivity.pipeline_sensitivity_facade.
PipelineSensitivityAnalysis
(pipeline, train_data, test_data, approaches=None, nodes_to_analyze=None, requirements=None, path_to_save=None, log=None)¶ Bases:
object
This class works as facade and allows to apply all kind of approaches to whole pipeline and separate nodes together.
- Parameters
pipeline (fedot.core.pipelines.pipeline.Pipeline) – pipeline object to analyze
train_data (fedot.core.data.data.InputData) – data used for Pipeline training
test_data (fedot.core.data.data.InputData) – data used for Pipeline validation
approaches (List[Union[Type[fedot.sensitivity.node_sa_approaches.NodeAnalyzeApproach], Type[fedot.sensitivity.operations_hp_sensitivity.multi_operations_sensitivity.MultiOperationsHPAnalyze]]]) – methods applied to pipeline. Default: None
nodes_to_analyze (List[fedot.core.pipelines.node.Node]) – nodes to analyze. Default: all nodes
requirements (fedot.sensitivity.sa_requirements.SensitivityAnalysisRequirements) – extra requirements to define specific details for different approaches. See SensitivityAnalysisRequirements class documentation.
path_to_save – path to save results to. Default: ~home/Fedot/sensitivity/
log (Optional[fedot.core.log.Log]) –
Default: False :param log: log: Log object to record messages
-
analyze
()¶ Applies defined sensitivity analysis approaches
Pipeline Structure Analysis¶
-
class
fedot.sensitivity.nodes_sensitivity.
NodesAnalysis
(pipeline, train_data, test_data, approaches=None, requirements=None, path_to_save=None, log=None, nodes_to_analyze=None)¶ Bases:
object
This class is for nodes sensitivity analysis within a Pipeline . It takes nodes and approaches to be applied to chosen nodes. To define which nodes to analyze pass them to nodes_to_analyze filed or all nodes will be analyzed.
- Parameters
pipeline (fedot.core.pipelines.pipeline.Pipeline) – pipeline object to analyze
train_data (fedot.core.data.data.InputData) – data used for Pipeline training
test_data (fedot.core.data.data.InputData) – data used for Pipeline validation
approaches (Optional[List[Type[fedot.sensitivity.node_sa_approaches.NodeAnalyzeApproach]]]) – methods applied to nodes to modify the pipeline or analyze certain operations. Default: [NodeDeletionAnalyze, NodeReplaceOperationAnalyze]
nodes_to_analyze (List[fedot.core.pipelines.node.Node]) – nodes to analyze. Default: all nodes
path_to_save – path to save results to. Default: ~home/Fedot/sensitivity
log (Optional[fedot.core.log.Log]) – log: Log object to record messages
requirements (fedot.sensitivity.sa_requirements.SensitivityAnalysisRequirements) –
-
analyze
()¶ Main method to run the analyze process for every node.
- Return nodes_results
dict with analysis result per Node
- Return type
dict
Pipeline Non-Structure Analysis¶
-
class
fedot.sensitivity.pipeline_sensitivity.
PipelineAnalysis
(pipeline, train_data, test_data, approaches=None, requirements=None, path_to_save=None, log=None)¶ Bases:
object
This class is for analyzing the Pipeline as the black-box model, using analysis approaches defined for whole pipeline perturbation, i.e. MultiOperationsHPAnalyze.
- Parameters
pipeline (fedot.core.pipelines.pipeline.Pipeline) – pipeline object to analyze
train_data (fedot.core.data.data.InputData) – data used for Pipeline training
test_data (fedot.core.data.data.InputData) – data used for Pipeline validation
approaches (Optional[List[Type[fedot.sensitivity.operations_hp_sensitivity.multi_operations_sensitivity.MultiOperationsHPAnalyze]]]) – methods applied to pipeline Default: [MultiOperationsHPAnalyze]
requirements (fedot.sensitivity.sa_requirements.SensitivityAnalysisRequirements) – extra requirements to define specific details for different approaches. See SensitivityAnalysisRequirements class documentation.
path_to_save – path to save results to. Default: ~home/Fedot/sensitivity/pipeline_sa
log (Optional[fedot.core.log.Log]) –
Default: False :param log: log: Log object to record messages
-
analyze
()¶ Apply defined approaches for the black-box pipeline analysis :return:
- Return type
dict
Node Structure Analysis¶
-
class
fedot.sensitivity.node_sa_approaches.
NodeAnalysis
(approaches=None, approaches_requirements=None, path_to_save=None, log=None)¶ Bases:
object
- Parameters
approaches (Optional[List[Type[NodeAnalyzeApproach]]]) – methods applied to nodes to modify the pipeline or analyze certain operations. Default: [NodeDeletionAnalyze, NodeTuneAnalyze, NodeReplaceOperationAnalyze]
path_to_save – path to save results to. Default: ~home/Fedot/sensitivity
log (Optional[fedot.core.log.Log]) – log: Log object to record messages
approaches_requirements (fedot.sensitivity.sa_requirements.SensitivityAnalysisRequirements) –
-
analyze
(pipeline, node, train_data, test_data, is_save=False)¶ Method runs Node analysis within defined approaches
- Parameters
is_save (bool) – whether the certain node analysis result is needed to ba saved
pipeline (fedot.core.pipelines.pipeline.Pipeline) – Pipeline containing the analyzed Node
node (fedot.core.pipelines.node.Node) – Node object to analyze in Pipeline
train_data (fedot.core.data.data.InputData) – data used for Pipeline training
test_data (fedot.core.data.data.InputData) – data used for Pipeline validation
- Returns
dict with Node analysis result per approach
- Return type
dict
-
class
fedot.sensitivity.node_sa_approaches.
NodeAnalyzeApproach
(pipeline, train_data, test_data, requirements=None, path_to_save=None, log=None)¶ Bases:
abc.ABC
Base class for analysis approach.
- Parameters
pipeline (fedot.core.pipelines.pipeline.Pipeline) – Pipeline containing the analyzed Node
train_data – data used for Pipeline training
test_data (fedot.core.data.data.InputData) – data used for Pipeline validation
path_to_save – path to save results to. Default: ~home/Fedot/sensitivity
log (Optional[fedot.core.log.Log]) – log: Log object to record messages
requirements (fedot.sensitivity.sa_requirements.SensitivityAnalysisRequirements) –
-
abstract
analyze
(node, **kwargs)¶ Creates the difference metric(scorer, index, etc) of the changed graph in relation to the original one
- Parameters
node (fedot.core.pipelines.node.Node) – the sequence number of the node as in DFS result
- Return type
Union[List[dict], List[float]]
-
abstract
sample
(*args)¶ Changes the pipeline according to the approach
- Return type
Union[List[fedot.core.pipelines.pipeline.Pipeline], fedot.core.pipelines.pipeline.Pipeline]
-
class
fedot.sensitivity.node_sa_approaches.
NodeDeletionAnalyze
(pipeline, train_data, test_data, requirements=None, path_to_save=None)¶ Bases:
fedot.sensitivity.node_sa_approaches.NodeAnalyzeApproach
- Parameters
pipeline (fedot.core.pipelines.pipeline.Pipeline) –
train_data (fedot.core.data.data.InputData) –
test_data (fedot.core.data.data.InputData) –
requirements (fedot.sensitivity.sa_requirements.SensitivityAnalysisRequirements) –
-
analyze
(node, **kwargs)¶ - Parameters
node (fedot.core.pipelines.node.Node) – Node object to analyze
- Returns
the ratio of modified pipeline score to origin score
- Return type
Union[List[dict], List[float]]
-
sample
(node)¶ - Parameters
node (fedot.core.pipelines.node.Node) – Node object to delete from Pipeline object
- Returns
Pipeline object without node
-
class
fedot.sensitivity.node_sa_approaches.
NodeReplaceOperationAnalyze
(pipeline, train_data, test_data, requirements=None, path_to_save=None)¶ Bases:
fedot.sensitivity.node_sa_approaches.NodeAnalyzeApproach
Replace node with operations available for the current task and evaluate the score difference
- Parameters
pipeline (fedot.core.pipelines.pipeline.Pipeline) –
train_data (fedot.core.data.data.InputData) –
test_data (fedot.core.data.data.InputData) –
requirements (fedot.sensitivity.sa_requirements.SensitivityAnalysisRequirements) –
-
analyze
(node, **kwargs)¶ - Parameters
node (fedot.core.pipelines.node.Node) – Node object to analyze
- Returns
the ratio of modified pipeline score to origin score
- Return type
Union[List[dict], List[float]]
-
sample
(node, nodes_to_replace_to, number_of_random_operations=None)¶ - Parameters
node (fedot.core.pipelines.node.Node) – Node object to replace
nodes_to_replace_to (Optional[List[fedot.core.pipelines.node.Node]]) – nodes provided for old_node replacement
number_of_random_operations (Optional[int]) – number of replacement operations, if nodes_to_replace_to not provided
- Returns
Sequence of Pipeline objects with new operations instead of old one.
- Return type
Union[List[fedot.core.pipelines.pipeline.Pipeline], fedot.core.pipelines.pipeline.Pipeline]
Hyperparameters Analysis¶
-
class
fedot.sensitivity.operations_hp_sensitivity.multi_operations_sensitivity.
MultiOperationsHPAnalyze
(pipeline, train_data, test_data, requirements=None, path_to_save=None, log=None)¶ Bases:
object
Provides with analysis of all the Pipeline’s operations hyperparameters using sample and analyze methods from SALib.
- Parameters
pipeline (fedot.core.pipelines.pipeline.Pipeline) – pipeline object to analyze
train_data (fedot.core.data.data.InputData) – data used for Pipeline training
test_data (fedot.core.data.data.InputData) – data used for Pipeline validation
requirements (fedot.sensitivity.sa_requirements.SensitivityAnalysisRequirements) – extra requirements to define specific details for different approaches. See SensitivityAnalysisRequirements class documentation.
path_to_save – path to save results to. Default: ~home/Fedot/sensitivity/
log (Optional[fedot.core.log.Log]) –
Default: False :param log: log: Log object to record messages
-
analyze
()¶ Analyze all the hyperparameters af all Pipeline operations using SA methods. Default: Sobol method with Saltelli sample algorithm :return: Main and total Sobol indices for every parameter per node.
- Return type
dict
-
sample
(*args)¶ Makes hyperparameters samples
- Parameters
args – i.e. sample_size
- Return List[Pipeline]
List of Pipelines with new sampled hyperparameters
- Return type
Union[List[fedot.core.pipelines.pipeline.Pipeline], fedot.core.pipelines.pipeline.Pipeline]
-
static
convert_results_to_json
(problem, si)¶ - Parameters
problem (fedot.sensitivity.operations_hp_sensitivity.problem.MultiOperationsProblem) –
si (dict) –
- Return type
dict
Sensitivity Analysis Requirements¶
-
class
fedot.sensitivity.sa_requirements.
HyperparamsAnalysisMetaParams
(analyze_method, sample_method, sample_size)¶ Bases:
tuple
-
property
analyze_method
¶ Alias for field number 0
-
property
sample_method
¶ Alias for field number 1
-
property
sample_size
¶ Alias for field number 2
-
property
-
class
fedot.sensitivity.sa_requirements.
ReplacementAnalysisMetaParams
(nodes_to_replace_to, number_of_random_operations)¶ Bases:
tuple
-
property
nodes_to_replace_to
¶ Alias for field number 0
-
property
number_of_random_operations
¶ Alias for field number 1
-
property
-
class
fedot.sensitivity.sa_requirements.
SensitivityAnalysisRequirements
(metric=None, hyperparams_analyze_method='sobol', hyperparams_sample_method='saltelli', hyperparams_analysis_samples_size=100, replacement_nodes_to_replace_to=None, replacement_number_of_random_operations=None, is_visualize=True, is_save_results_to_json=True)¶ Bases:
object
Use this object to pass all the requirements needed for SA.
- Parameters
hyperparams_analyze_method (str) – defines string name of SA method to use. Defaults: ‘sobol’
hyperparams_sample_method (str) – defines string name of sampling method to use. Defaults: ‘saltelli’
hyperparams_analysis_samples_size (int) – defines the number of shyperparameters samples used in SA
replacement_nodes_to_replace_to (Optional[List[fedot.core.pipelines.node.Node]]) – defines nodes which is used in replacement analysis.
replacement_number_of_random_operations (Optional[int]) – if replacement_nodes_to_replace_to is not filled, define the number of randomly chosen operations used in replacement analysis.
is_visualize (bool) – defines whether the SA visualization needs to be saved to .png files.
is_save_results_to_json (bool) – defines whether the SA indices needs to be saved to .json file.
metric – metric used for validation. Default: see MetricByTask