Monte-Carlo rasterisation; arbitrary dimensionality of AlphaCompositor blending

shapovalov · facebook-github-bot · commit 0e377c6850f9 · 2022-03-17T05:12:39.000-07:00
Summary:
Fixes required for MC rasterisation to work.
1) Wrong number of channels for background was used (derived from points dimensions, not features dimensions;
2) split of the results on the wrong dimension was done;
3) CORE CHANGE: blending in alpha compositor was assuming RGBA input.

Reviewed By: davnov134

Differential Revision: D34933673

fbshipit-source-id: a5cc9f201ea21e114639ab9e291a10888d495206
diff --git a/pytorch3d/renderer/points/compositor.py b/pytorch3d/renderer/points/compositor.py
@@ -92,9 +92,8 @@ def _add_background_color_to_images(pix_idxs, images, background_color):
 
     background_color = background_color.to(images)
 
-    # add alpha channel
-    if background_color.shape[0] == 3 and images.shape[1] == 4:
-        # special case to allow giving RGB background for RGBA
+    # add alpha channel if needed
+    if background_color.shape[0] + 1 == images.shape[1]:
         alpha = images.new_ones(1)
         background_color = torch.cat([background_color, alpha])
 
@@ -107,7 +106,7 @@ def _add_background_color_to_images(pix_idxs, images, background_color):
     num_background_pixels = background_mask.sum()
 
     # permute so that features are the last dimension for masked_scatter to work
-    masked_images = images.permute(0, 2, 3, 1)[..., :4].masked_scatter(
+    masked_images = images.permute(0, 2, 3, 1).masked_scatter(
         background_mask[..., None],
         background_color[None, :].expand(num_background_pixels, -1),
     )
