vision.learner

Learner support for computer vision

Computer Vision Interpret

vision.interpret is the module that implements custom Interpretation classes for different vision tasks by inheriting from it.

class SegmentationInterpretation[source][test]

SegmentationInterpretation(learn:Learner, preds:Tensor, y_true:Tensor, losses:Tensor, ds_type:DatasetType=<DatasetType.Valid: 2>) :: Interpretation No tests found for SegmentationInterpretation. To contribute a test please refer to this guide and this discussion.

Interpretation methods for segmenatation models.

top_losses[source][test]

top_losses(sizes:Tuple, k:int=None, largest=True) No tests found for top_losses. To contribute a test please refer to this guide and this discussion.

Reduce flatten loss to give a single loss value for each image

_interp_show[source][test]

_interp_show(ims:ImageSegment, classes:Collection[T_co]=None, sz:int=20, cmap='tab20', title_suffix:str=None) No tests found for _interp_show. To contribute a test please refer to this guide and this discussion.

Show ImageSegment with color mapping labels

show_xyz[source][test]

show_xyz(i, classes:list=None, sz=10) No tests found for show_xyz. To contribute a test please refer to this guide and this discussion.

show (image, true and pred) from self.ds with color mappings, optionally only plot

_generate_confusion[source][test]

_generate_confusion() No tests found for _generate_confusion. To contribute a test please refer to this guide and this discussion.

Average and Per Image Confusion: intersection of pixels given a true label, true label sums to 1

_plot_intersect_cm[source][test]

_plot_intersect_cm(cm, title='Intersection with Predict given True') No tests found for _plot_intersect_cm. To contribute a test please refer to this guide and this discussion.

Plot confusion matrices: self.mean_cm or self.single_img_cm generated by _generate_confusion

Let’s show how SegmentationInterpretation can be used once we train a segmentation model.

train

camvid = untar_data(URLs.CAMVID_TINY)
path_lbl = camvid/'labels'
path_img = camvid/'images'

codes = np.loadtxt(camvid/'codes.txt', dtype=str)
get_y_fn = lambda x: path_lbl/f'{x.stem}_P{x.suffix}'

data = (SegmentationItemList.from_folder(path_img)
        .split_by_rand_pct()
        .label_from_func(get_y_fn, classes=codes)
        .transform(get_transforms(), tfm_y=True, size=128)
        .databunch(bs=16, path=camvid)
        .normalize(imagenet_stats))

data.show_batch(rows=2, figsize=(7,5))

learn = unet_learner(data, models.resnet18)
learn.fit_one_cycle(3,1e-2)
learn.save('mini_train')

Warning: Following results will not make much sense with this underperforming model but functionality will be explained with ease

interpret

interp = SegmentationInterpretation.from_learner(learn)

Since FlattenedLoss of CrossEntropyLoss() is used we reshape and then take the mean of pixel losses per image. In order to do so we need to pass sizes:tuple to top_losses()

top_losses, top_idxs = interp.top_losses(sizes=(128,128))

(tensor([3.3195, 3.1692, 2.6574, 2.5976, 2.4910, 2.3759, 2.3710, 2.2064, 2.0871,
         2.0834, 2.0479, 1.8645, 1.8412, 1.7956, 1.7013, 1.6126, 1.6015, 1.5470,
         1.4495, 1.3423]),
 tensor([12,  4, 17, 13, 19, 18,  7,  8, 10,  1, 15,  0,  2,  9, 16, 11, 14,  5,
          6,  3]))

Next, we can generate a confusion matrix similar to what we usually have for classification. Two confusion matrices are generated: mean_cm which represents the global label performance and single_img_cm which represents the same thing but for each individual image in dataset.

Values in the matrix are calculated as:

begin{align} CM_{ij} & = IOU(Predicted , True | True) end{align}

Or in plain english: ratio of pixels of predicted label given the true pixels

learn.data.classes

array(['Animal', 'Archway', 'Bicyclist', 'Bridge', 'Building', 'Car', 'CartLuggagePram', 'Child', 'Column_Pole',
       'Fence', 'LaneMkgsDriv', 'LaneMkgsNonDriv', 'Misc_Text', 'MotorcycleScooter', 'OtherMoving', 'ParkingBlock',
       'Pedestrian', 'Road', 'RoadShoulder', 'Sidewalk', 'SignSymbol', 'Sky', 'SUVPickupTruck', 'TrafficCone',
       'TrafficLight', 'Train', 'Tree', 'Truck_Bus', 'Tunnel', 'VegetationMisc', 'Void', 'Wall'], dtype='<U17')

mean_cm, single_img_cm = interp._generate_confusion()

((32, 32), (20, 32, 32))

_plot_intersect_cm first displays a dataframe showing per class score using the IOU definition we made earlier. These are the diagonal values from the confusion matrix which is displayed after.

NaN indicate that these labels were not present in our dataset, in this case validation set. As you can imagine it also helps you to maybe construct a better representing validation set.

df = interp._plot_intersect_cm(mean_cm, "Mean of Ratio of Intersection given True Label")

Next let’s look at the single worst prediction in our dataset. It looks like this dummy model just predicts everything as Road :)

i = top_idxs[0]
df = interp._plot_intersect_cm(single_img_cm[i], f"Ratio of Intersection given True Label, Image:{i}")

Finally we will visually inspect this single prediction

interp.show_xyz(i, sz=15)

Warning: With matplotlib colormaps the max number of unique qualitative colors is 20. So if len(classes) > 20 then close class indexes may be plotted with the same color. Let’s fix this together :)

{'Animal': 0,
 'Archway': 1,
 'Bicyclist': 2,
 'Bridge': 3,
 'Building': 4,
 'Car': 5,
 'CartLuggagePram': 6,
 'Child': 7,
 'Column_Pole': 8,
 'Fence': 9,
 'LaneMkgsDriv': 10,
 'LaneMkgsNonDriv': 11,
 'Misc_Text': 12,
 'MotorcycleScooter': 13,
 'OtherMoving': 14,
 'ParkingBlock': 15,
 'Pedestrian': 16,
 'Road': 17,
 'RoadShoulder': 18,
 'Sidewalk': 19,
 'SignSymbol': 20,
 'Sky': 21,
 'SUVPickupTruck': 22,
 'TrafficCone': 23,
 'TrafficLight': 24,
 'Train': 25,
 'Tree': 26,
 'Truck_Bus': 27,
 'Tunnel': 28,
 'VegetationMisc': 29,
 'Void': 30,
 'Wall': 31}

class ObjectDetectionInterpretation[source][test]

ObjectDetectionInterpretation(learn:Learner, preds:Tensor, y_true:Tensor, losses:Tensor, ds_type:DatasetType=<DatasetType.Valid: 2>) :: Interpretation No tests found for ObjectDetectionInterpretation. To contribute a test please refer to this guide and this discussion.

Interpretation methods for classification models.

Warning: ObjectDetectionInterpretation is not implemented yet. Feel free to implement it :)

©2021 fast.ai. All rights reserved.
Site last generated: Jan 5, 2021