Fel/operator: Operator PR

[fel-thomas]

Operator(s) Abstraction

This PR introduces the concept of an operator, which aims to generalize attribution methods for a wide range of use-cases. The idea is as follows: to define an attribution method, we need any function that takes in the model (f), a series of inputs (x) and labels (y) and returns a scalar in R.

g(f, x, y) -> R

This function, called an operator, can be defined by the user (or by us) and then provides a common interface for all attribution methods that will call it (or calculate its gradient). As you can see, the goal is for attribution methods to have this function as an attribute (in more detail, this will give self.inference_function = operator at some point).

Some Examples of Operators

1. The most trivial operator is perhaps the classification one, it consists of taking a particular logit to explain a class. In the case where the model f: R^n -> R^c with c being the number of classes and y being one-hot vectors, then our operator simply boils down to:

def g(f, x, y):
    return tf.reduce_sum(f(x) * y, -1)

2. Regarding segmentation, with a model f: R^w×h×c -> R^w×h×r with r being the number of channels, there is no properly defined operator in the literature for segmentation, but we could imagine explaining a whole channel, with y being a one-hot vector that explains channel r, the operator therefore simply boils down to:

def g(f, x, y):
    return tf.reduce_sum(f(x) * y[:,None,None,:], (1,2))

3. Regarding bounding-box, an operator has already been defined in the literature with the D-RISE article. It consists of using the three IOU, objectness, and box classification scores to form... a scalar!

4. To explain concepts, for example with a model f = c ⚬ g(x), with a = g(x) and a factorizer that allows interpreting a in a reduced dimension space u = factorizer(a), we can very well define the following operator:

def g(c, u, y):
    a = factorizer.inverse(u)
    y_pred = c(a)
    return tf.reduce_sum(y_pred * y, -1)

As you can see, many cases can be handled in this manner!

Implementation

Regarding implementation, there is a series of operators available in the file in commons/operators and the most important part -- the operator plug -- is located in the attributions/base.py file. As discussed with @AntoninPoche & @lucashervier, I think you know where I'm coming from but the PyTorch implementation is not far and would be located here!

Once this is done, I simply added the argument to all the attribution methods defined in the library.

This being a quite important PR (not in terms of code line but in terms of internal API change), I'm not against a careful reading and re-reading by the team! :)

[AntoninPoche]

This is great work! I think it is more elegant and flexible than previously. I also like the new exception raised for gradient-based methods.

However, I think we should update the metrics.

[AntoninPoche]

Operator and regression

I am concerned about how this PR would support multi-output regression.

For me what we want for perturbation based-methods is to compare the initial pred to the modified pred on all outputs. In this sense, we cannot reduce the output of the inference function to a scalar. Nonetheless, methods like Occlusion, when they aggregate sensitivity, they do not take this increase of dimension into account. I have no quick fix for this. And I do not think that taking the mean of the output is pertinent.

However, gradient-based functions on multi-output regression task want the output to be reduced, otherwise, they would output one gradient for each output.

Finally, we could reduce the output of the inference function for all methods if we take into account the ground truth. We can say that the inference function output is the mae for example.

What do you think?

My bad, I though about something, I think we can make it work by passing the groung truth or the prediction of the model in the target. Such as:

@tf.function
def regression_operator(model: Callable,
                                       inputs: tf.Tensor,
                                       targets: tf.Tensor) -> tf.Tensor:
    """
    Compute predictions scores, only for the label class, for a batch of samples.
    Parameters
    ----------
    model
        Model used for computing predictions.
    inputs
        Input samples to be explained.
    targets
        Corresponding ground truth or prediction of the model depending on the expected behavior.
    Returns
    -------
    scores
        Predictions scores computed, only for the label class.
    """
    scores = tf.reduce_mean(tf.abs(model(inputs) - targets), axis=-1)
    return scores

[Agustin-Picard]

It might be worth it to define an interface for these operators, or at least, specify more clearly the types of the callable (ex. Callable[[tf.keras.Model, tf.Tensor, tf.Tensor], tf.Tensor]). This should allow us to perform the check that Antonin's asking for.

[lucashervier]

Some minor remarks concerning optimization and documentation!

[AntoninPoche]

Hello guys, I will summarize our discussion on this PR and list things to change:

• Implement a check_operator function used in BlackBoxExplainer.__init__(). This function should check that the operator follows the expected format and guide user through exceptions overwise.
• The operator for BlackBoxExplainer do not need to be differentiable, hence the operator gradient management can be transferred to WhiteBoxExplainer.
• The documentation of operator should be clearer in the docstring of functions.
• The bug on feature viz should be corrected as we do not want to remove possibilities from the library.
• Metrics should use the operator, we will add it as an argument of the metrics. We should be careful to check if the operator is in fact an operator through the aforementioned check. (I will take care of that one and merge the two CausalFidelity metrics).

The following points were deemed as further enhancement that will be taken into account in future PR:

• Go from [3.6, 3.8] to [3.7, 3.10]
• Update Readme
• Cleaner adaptation of BoundingBox to the operator API.
• Support Pytorch.
• Tutorial on operator and target at the same time.
• Other tutorials
• NLP
• Diffusion models

[AntoninPoche]

Just one last modififcation, check_operator function should be moved in my view. Otherwise LGTM