Katie M Lewis

Generative ML Publications

Vision-Language

• GIST: Generating Image-Specific Text for Fine-grained Object Classification. Lewis, K. M.* , Mu, E.*, Dalca, A., Guttag, J. (2023). [paper] [code and new dataset]

Fashion/Clothing

• SizeGAN: Improving Size Representation in Clothing Catalogs. Lewis, K. M. , & Guttag, J. (2022). [paper]
• TryOnGAN: body-aware try-on via layered interpolation. Lewis, K. M., Varadharajan, S., & Kemelmacher-Shlizerman, I. (2021). ACM Transactions on Graphics (TOG), 40(4), 1-10. [paper] [arXiv]

Art

• At the Intersection of Conceptual Art and Deep Learning: The End of Signature. Lewis, K. M.*, Shanmugam, D. M.*, Ortiz, J. J. G.*, Kurant, A., & Guttag, J. NeurIPS Workshop on Broadening Research Collaborations 2022. [paper] [poster] [installation] [NYTimes article]
• Painting Many Pasts: Synthesizing Time Lapse Videos of Paintings. A. Zhao, G. Balakrishnan, KM Lewis, F. Durand, J. Guttag, A.V. Dalca. CVPR 2020 [paper]

Medical Imaging Publications

• ACM Conference on Health, Inference, and Learning (CHIL) 2020 [paper]
• Machine Learning for Healthcare (ML4H) @ NeurIPS 2018 [paper]
• Women in Machine Learning (WiML) @ NeurIPS 2018
• Women in Computer Vision (WiCV) @ CVPR 2019 [poster]
• Amazon Graduate Symposium 2019 [poster]

Exploratory Research

Class Research Projects

• Intuitively assessing ml model reliability through example-based explanations and editing model inputs. Suresh, H., Lewis, K. M., Guttag, J., & Satyanarayan, A. (2022, March). In 27th International Conference on Intelligent User Interfaces (pp. 767-781). [paper]
• From working on try-on, I noticed GANs do not generate images of text well from scratch (e.g. words on clothing become illegible blobs). For my computer vision class, I focused on a short project to see how well I could get a conditional and unconditional GAN working to generate specified words. [presentation]

Diffusion Model Exploration

• I spend a lot of time thinking about the creative process around clothing design. When learning about diffusion models, my first curiosity was how good they are at generating new clothing designs from natural world inspiration and prompts. Ideally, I would be able to give both visualizations and text descriptors to the model (working on it); however Stable Diffusion Inpainting can generate a variety of designs (shown left) from text alone!

Interest Map

My current interest map (a concept introduced to me by my friend and labmate Divya Shanmugam) shows categories of questions that I think about day to day. This map contains topics that I want to learn more about (not necessarily tied to my research) and issues that I would like to work towards changing. A small snapshot is shown to the left and the link to the full interest map is [here]