Yuting Yang

White Box Search over Audio Synthesizer Parameters

Yuting Yang, Zeyu Jin, Connelly Barnes, Adam Finkelstein

ISMIR 2023

This paper adapts and extends differentiable rendering to the problem of synthesizer parameter inference, where we search for a set of patch connections and parameters to generate audio that best matches a given target sound. We design novel gradient rules to directly differentiate the synth as a white box program to overcome the discrete nature of the synth programs. Using a simple FM synth, our method better approximates the target sound, and is preferred by AMT users when compared to baselines.

[ project page ]

Exploiting Program Representation with Shader Applications

Yuting Yang

Princeton PhD Dissertation

[ dissertation ] [ slides ]

Aδ: Autodiff for Discontinuous Programs - Applied to Shaders

Yuting Yang, Connelly Barnes, Andrew Adams, Adam Finkelstein

SIGGRAPH 2022

This paper describes a compiler-based framework that extends reverse mode automatic differentiation so as to provide accurate gradients for arbitrary programs, even at discontinuities. We demonstrate the effectiveness of this framework in the context of optimizing parameters of 2D/3D procedural shader programs so that they match target illustrations.

[ project page ] [ paper ] [ video (short) ] [ video (long) ] [ video (live demo) ] [ Github ]

Learning from Shader Program Traces

Yuting Yang, Connelly Barnes, Adam Finkelstein

Eurographics 2022

This paper proposes to learn from program traces of procedural fragment shaders -- programs that generate images. At each pixel, we collect the intermediate values computed at program execution, and these data form the input to the learned model. We investigate this learning task for a variety of applications: denoising, learning from partial computation, learning postprocessing filters and learning non-imagery simulations. Our method outperforms baselines both qualitatively and quantitatively. We also conduct a series of analyses that show certain features are important within the trace: these coincide with intuitively important aspects of the program. The important features can be helpful for selecting a good subset of trace features for learning, and even learning from a small subset of the trace already outperforms the baselines.

[ project page ] [ paper ] [ video (short) ] [ video (long) ] [ Github ]

Hyperparameter Optimization in Black-box Image Processing using Differentiable Proxies

Ethan Tseng, Felix Yu, Yuting Yang, Fahim Mannan, Karl St. Arnaud, Derek Nowrouzezahrai, Jean-François Lalonde, Felix Heide

SIGGRAPH 2019

We present a fully automatic system to optimize the parameters of black-box hardware and software image processing pipelines according to any arbitrary (i.e., application-specific) metric. We leverage a differentiable mapping between the configuration space and evaluation metrics, parameterized by a convolutional neural network that we train in an end-to-end fashion with imaging hardware in-the-loop. Unlike prior art, our differentiable proxies allow for high-dimension parameter search with stochastic first-order optimizers, without explicitly modeling any lower-level image processing transformations.

[ project page ] [ paper ]

Approximate Program Smoothing Using Mean-Variance Statistics, with Application to Procedural Shader Bandlimiting

Yuting Yang, Connelly Barnes

Eurographics 2018

This paper introduces a general method to approximate the convolution of an arbitrary program with a Gaussian kernel which facilitates smoothing the program. We give several approximations including a novel adaptive Gaussian approximation that is accurate up to the second order in the standard deviation of the smoothing kernel. We construct a compiler framework that automatically chooses approximations for different parts of the program. We then apply this framework to bandlimit procedural shaders.

[ project page ] [ paper ] [ supplemental ] [ video (Vimeo) ]

VizGen: Accelerating Visual Computing Prototypes in Dynamic Languages

Yuting Yang, Sam Prestwood, Connelly Barnes

SIGGRAPH Asia 2016

This paper introduces a novel domain-specific compiler, which translates visual computing programs written in dynamic languages to highly efficient code. We define "dynamic" languages as those such as Python and MATLAB, which feature dynamic typing and can be useful for rapid prototyping, but introduces significant overheads in program execution time. We introduce a compiler framework for accelerating visual computing programs written in general-purpose dynamic languages. Our compiler allows frequently orders of magnitude performance gains over general compilers for dynamic languages by specializing the compiler for visual computation.

[ project page ] [ paper ] [ supplemental ] [ slides (pptx) ]

Intersection monitoring from video using 3D reconstruction

Yuting Yang, Camillo Taylor, Daniel Lee

ITS International January February 2016

Traffic information can be collected from existing inexpensive roadside cameras but extracting it often entails manual work or costly commercial software. We proposed a method that tracks and counts vehicles in the video and to use the tracking information to compute a 3D model for the vehicles and visualise the 2D road situation into 3D. The 3D model can provide feedback on the tracking and counting model for future research.

[ article ]