Maximum entropy methods for extracting the learned features of deep neural networks

Alex Finnegan; Jun S. Song

doi:10.1101/105957

CARVIEW

MOTORHOMES

Select Language

HTTP/2 301 date: Tue, 20 Jan 2026 19:44:35 GMT content-length: 0 location: https://doi.org/10.1101/105957 server: cloudflare vary: Origin expires: Wed, 21 Jan 2026 19:44:35 GMT permissions-policy: interest-cohort=(),browsing-topics=() cf-cache-status: DYNAMIC nel: {"report_to":"cf-nel","success_fraction":0.0,"max_age":604800} strict-transport-security: max-age=31536000; includeSubDomains; preload report-to: {"group":"cf-nel","max_age":604800,"endpoints":[{"url":"https://a.nel.cloudflare.com/report/v4?s=55wI2kL%2FbOKvYHkuM5i0oYm5w%2BEIOh4ZswPKzYiqVxfOxdiDbluolt8%2BPlZhDQKSgs2L637tB69Vl%2Bd2yzWZ%2BlGdt%2FGEqQ%3D%3D"}]} cf-ray: 9c11113cbc60c1b3-BLR alt-svc: h3=":443"; ma=86400 HTTP/2 302 date: Tue, 20 Jan 2026 19:44:35 GMT content-type: text/html;charset=utf-8 location: https://biorxiv.org/lookup/doi/10.1101/105957 server: cloudflare vary: Origin vary: Accept expires: Tue, 20 Jan 2026 20:11:01 GMT permissions-policy: interest-cohort=(),browsing-topics=() cf-cache-status: DYNAMIC nel: {"report_to":"cf-nel","success_fraction":0.0,"max_age":604800} strict-transport-security: max-age=31536000; includeSubDomains; preload report-to: {"group":"cf-nel","max_age":604800,"endpoints":[{"url":"https://a.nel.cloudflare.com/report/v4?s=GfvwB3cmplfbRdxaFjRBDyaqPZD03DaQINzRnPb8Sn8v7QBr1iClVGxVrbwx579UN4SQQvMwFUdq7sE5lpHvEWO8TEpQRw%3D%3D"}]} cf-ray: 9c11113e0ca4c1b3-BLR alt-svc: h3=":443"; ma=86400 HTTP/1.1 302 Found Date: Tue, 20 Jan 2026 19:44:36 GMT Content-Type: text/html; charset=iso-8859-1 Transfer-Encoding: chunked Connection: keep-alive server: cloudflare location: https://www.biorxiv.org/lookup/doi/10.1101/105957 cf-cache-status: DYNAMIC Nel: {"report_to":"cf-nel","success_fraction":0.0,"max_age":604800} Report-To: {"group":"cf-nel","max_age":604800,"endpoints":[{"url":"https://a.nel.cloudflare.com/report/v4?s=WQvaWkzDuio44DYvFcOUX%2B%2BPimFwDFDrZyvyv8NCurCWoT2spg4zTicIKfXUd7%2F2A0Q92CYFZ%2BbEkKRXzYLxRcisdhIiDJSh0UNo"}]} CF-RAY: 9c11113fafa24701-BOM alt-svc: h3=":443"; ma=86400 HTTP/2 301 date: Tue, 20 Jan 2026 19:44:36 GMT content-type: text/html; charset=UTF-8 location: https://www.biorxiv.org/content/10.1101/105957v2 cf-ray: 9c111142f9400989-BLR x-content-type-options: nosniff x-content-type-options: nosniff x-drupal-cache: MISS expires: Tue, 20 Jan 2026 20:14:36 GMT cache-control: public, max-age=1800 x-varnish-ttl: pragma: no-cache vary: Accept-Encoding x-highwire-sitecode: biorxiv x-highwire-smart-code: biorxiv_production x-varnish: 1912783797 x-varnish-cache: via: 1.1 varnish cf-cache-status: MISS set-cookie: __cf_bm=tl6Wmddiur9dj1SLpxefCSlVdWW1dSFH8riap74Z1Bs-1768938276-1.0.1.1-fFvmP9kum7idZsHJmKeWzMHHlWylVeIOBNcLX2cKHBFJq583SW2sri3mOcpYXREcoLnJZD2dJJQjvlNaJ_DcjFIi39OECt238aFeLz6i_1c; path=/; expires=Tue, 20-Jan-26 20:14:36 GMT; domain=.www.biorxiv.org; HttpOnly; Secure; SameSite=None server: cloudflare HTTP/2 200 date: Tue, 20 Jan 2026 19:44:38 GMT content-type: text/html; charset=utf-8 content-encoding: gzip x-content-type-options: nosniff x-content-type-options: nosniff x-drupal-cache: MISS expires: Sun, 19 Nov 1978 05:00:00 GMT cache-control: no-cache, must-revalidate set-cookie: SSESS1dd6867f1a1b90340f573dcdef3076bc=_FoaN42XnNmfjWut-urlP74twK4hHNPIzA_HLgzK9Aw; expires=Thu, 12-Feb-2026 23:17:57 GMT; path=/; domain=.biorxiv.org; secure; HttpOnly content-language: en x-frame-options: SAMEORIGIN x-generator: Drupal 7 (https://drupal.org) link: ; rel="canonical",; rel="shortlink" vary: Accept-Encoding x-highwire-sitecode: biorxiv x-highwire-smart-code: biorxiv_production x-varnish: 1912783855 age: 0 via: 1.1 varnish x-varnish-ttl: x-varnish-cache: cf-cache-status: DYNAMIC server: cloudflare cf-ray: 9c111146cb8c0989-BLR Maximum entropy methods for extracting the learned features of deep neural networks | bioRxiv

Abstract

New architectures of multilayer artificial neural networks and new methods for training them are rapidly revolutionizing the application of machine learning in diverse fields, including business, social science, physical sciences, and biology. Interpreting deep neural networks, however, currently remains elusive, and a critical challenge lies in understanding which meaningful features a network is actually learning. We present a general method for interpreting deep neural networks and extracting network-learned features from input data. We describe our algorithm in the context of biological sequence analysis. Our approach, based on ideas from statistical physics, samples from the maximum entropy distribution over possible sequences, anchored at an input sequence and subject to constraints implied by the empirical function learned by a network. Using our framework, we demonstrate that local transcription factor binding motifs can be identified from a network trained on ChIP-seq data and that nucleosome positioning signals are indeed learned by a network trained on chemical cleavage nucleosome maps. Imposing a further constraint on the maximum entropy distribution also allows us to probe whether a network is learning global sequence features, such as the high GC content in nucleosome-rich regions. This work thus provides valuable mathematical tools for interpreting and extracting learned features from feed-forward neural networks.

The copyright holder for this preprint is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under a CC-BY-NC 4.0 International license.