Gram Matrix - ( Style Transferring with PyTorch )

Heading h2

Syntax

def gram_matrix(input_tensor):
    a, b, c, d = input_tensor.size()
    features = input_tensor.view(a * b, c * d)
    gram = torch.mm(features, features.t())
    return gram

Example

import torch
import torchvision.models as models

model = models.vgg19(pretrained=True).features

for param in model.parameters():
    param.requires_grad_(False)

def get_features(image, model):
    layers = {
        '0': 'conv1_1',
        '5': 'conv2_1',
        '10': 'conv3_1',
        '19': 'conv4_1',
        '21': 'conv4_2',
        '28': 'conv5_1'
    }

    features = {}
    x = image
    for name, layer in model._modules.items():
        x = layer(x)
        if name in layers:
            features[layers[name]] = x
    return features


def gram_matrix(input_tensor):
    a, b, c, d = input_tensor.size()
    features = input_tensor.view(a * b, c * d)
    gram = torch.mm(features, features.t())
    return gram

content = torch.randn(1, 3, 224, 224)
content_features = get_features(content, model)
content_grammar = {layer: gram_matrix(content_features[layer]) for layer in content_features}

Output

tensor([[ 8.8693e+09, -4.2926e+09, -3.4583e+09,  ..., -1.3654e+10,
         -1.0104e+10,  3.4444e+10],
        [-4.2926e+09,  2.1956e+09,  1.2474e+09,  ...,  5.5207e+09,
          4.9763e+09, -1.7202e+10],
        [-3.4583e+09,  1.2474e+09,  1.2765e+09,  ...,  3.0398e+09,
          3.1062e+09, -9.3356e+09],
        ...,
        [-1.3654e+10,  5.5207e+09,  3.0398e+09,  ...,  1.9014e+10,
          1.4047e+10, -4.5185e+10],
        [-1.0104e+10,  4.9763e+09,  3.1062e+09,  ...,  1.4047e+10,
          1.2881e+10, -3.9951e+10],
        [ 3.4444e+10, -1.7202e+10, -9.3356e+09,  ..., -4.5185e+10,
         -3.9951e+10,  1.2939e+11]])

Explanation

In neural style transfer, the Gram matrix is used to measure the correlation between features. The Gram matrix is the matrix obtained by multiplying the feature matrix with its transpose.

Use

The Gram matrix is used in various applications such as neural style transfer and image classification.

Important Points

• The Gram matrix is used for measuring the correlation between features
• The Gram matrix is obtained by multiplying the feature matrix with its transpose

Summary

In summary, the Gram matrix is used for measuring the correlation between features in neural style transfer and image classification. The Gram matrix is obtained by multiplying the feature matrix with its transpose. The Gram matrix is an important concept in neural style transfer, which allows for the transfer of style from one image to another.