A modern, fully typed Python library for converting HTML to Markdown. This library is a completely rewritten fork of markdownify with a modernized codebase, strict type safety and support for Python 3.9+.
- Full HTML5 Support: Comprehensive support for all modern HTML5 elements including semantic, form, table, ruby, interactive, structural, SVG, and math elements
- Type Safety: Strict MyPy adherence with comprehensive type hints
- Metadata Extraction: Automatic extraction of document metadata (title, meta tags) as comment headers
- Streaming Support: Memory-efficient processing for large documents with progress callbacks
- Highlight Support: Multiple styles for highlighted text (
<mark>
elements) - Task List Support: Converts HTML checkboxes to GitHub-compatible task list syntax
- Flexible Configuration: 20+ configuration options for customizing conversion behavior
- CLI Tool: Full-featured command-line interface with all API options exposed
- Custom Converters: Extensible converter system for custom HTML tag handling
- BeautifulSoup Integration: Support for pre-configured BeautifulSoup instances
- Extensive Test Coverage: 100% test coverage requirement with comprehensive test suite
pip install html-to-markdown
For improved performance, you can install with the optional lxml parser:
pip install html-to-markdown[lxml]
The lxml parser offers:
- ~30% faster HTML parsing compared to the default html.parser
- Better handling of malformed HTML
- More robust parsing for complex documents
Once installed, lxml is automatically used by default for better performance. You can explicitly specify a parser if needed:
result = convert_to_markdown(html) # Auto-detects: uses lxml if available, otherwise html.parser
result = convert_to_markdown(html, parser="lxml") # Force lxml (requires installation)
result = convert_to_markdown(html, parser="html.parser") # Force built-in parser
Convert HTML to Markdown with a single function call:
from html_to_markdown import convert_to_markdown
html = """
<!DOCTYPE html>
<html>
<head>
<title>Sample Document</title>
<meta name="description" content="A sample HTML document">
</head>
<body>
<article>
<h1>Welcome</h1>
<p>This is a <strong>sample</strong> with a <a href="https://example.com">link</a>.</p>
<p>Here's some <mark>highlighted text</mark> and a task list:</p>
<ul>
<li><input type="checkbox" checked> Completed task</li>
<li><input type="checkbox"> Pending task</li>
</ul>
</article>
</body>
</html>
"""
markdown = convert_to_markdown(html)
print(markdown)
Output:
<!--
title: Sample Document
meta-description: A sample HTML document
-->
# Welcome
This is a **sample** with a [link](https://example.com).
Here's some ==highlighted text== and a task list:
* [x] Completed task
* [ ] Pending task
If you need more control over HTML parsing, you can pass a pre-configured BeautifulSoup instance:
from bs4 import BeautifulSoup
from html_to_markdown import convert_to_markdown
# Configure BeautifulSoup with your preferred parser
soup = BeautifulSoup(html, "lxml") # Note: lxml requires additional installation
markdown = convert_to_markdown(soup)
The library offers extensive customization through various options:
from html_to_markdown import convert_to_markdown
html = "<div>Your content here...</div>"
markdown = convert_to_markdown(
html,
# Document processing
extract_metadata=True, # Extract metadata as comment header
convert_as_inline=False, # Treat as block-level content
strip_newlines=False, # Preserve original newlines
# Formatting options
heading_style="atx", # Use # style headers
strong_em_symbol="*", # Use * for bold/italic
bullets="*+-", # Define bullet point characters
highlight_style="double-equal", # Use == for highlighted text
# Text processing
wrap=True, # Enable text wrapping
wrap_width=100, # Set wrap width
escape_asterisks=True, # Escape * characters
escape_underscores=True, # Escape _ characters
escape_misc=True, # Escape other special characters
# Code blocks
code_language="python", # Default code block language
# Streaming for large documents
stream_processing=False, # Enable for memory efficiency
chunk_size=1024, # Chunk size for streaming
)
You can provide your own conversion functions for specific HTML tags:
from bs4.element import Tag
from html_to_markdown import convert_to_markdown
# Define a custom converter for the <b> tag
def custom_bold_converter(*, tag: Tag, text: str, **kwargs) -> str:
return f"IMPORTANT: {text}"
html = "<p>This is a <b>bold statement</b>.</p>"
markdown = convert_to_markdown(html, custom_converters={"b": custom_bold_converter})
print(markdown)
# Output: This is a IMPORTANT: bold statement.
Custom converters take precedence over the built-in converters and can be used alongside other configuration options.
Option | Type | Default | Description |
---|---|---|---|
extract_metadata |
bool | True |
Extract document metadata as comment header |
convert_as_inline |
bool | False |
Treat content as inline elements only |
heading_style |
str | 'underlined' |
Header style ('underlined' , 'atx' , 'atx_closed' ) |
highlight_style |
str | 'double-equal' |
Highlight style ('double-equal' , 'html' , 'bold' ) |
stream_processing |
bool | False |
Enable streaming for large documents |
parser |
str | auto-detect | BeautifulSoup parser (auto-detects 'lxml' or 'html.parser' ) |
autolinks |
bool | True |
Auto-convert URLs to Markdown links |
bullets |
str | '*+-' |
Characters to use for bullet points |
escape_asterisks |
bool | True |
Escape * characters |
wrap |
bool | False |
Enable text wrapping |
wrap_width |
int | 80 |
Text wrap width |
For a complete list of all 20+ options, see the Configuration Reference section below.
Convert HTML files directly from the command line with full access to all API options:
# Convert a file
html_to_markdown input.html > output.md
# Process stdin
cat input.html | html_to_markdown > output.md
# Use custom options
html_to_markdown --heading-style atx --wrap --wrap-width 100 input.html > output.md
# Advanced options
html_to_markdown \
--no-extract-metadata \
--convert-as-inline \
--highlight-style html \
--stream-processing \
--show-progress \
input.html > output.md
# Content processing
--convert-as-inline # Treat content as inline elements
--no-extract-metadata # Disable metadata extraction
--strip-newlines # Remove newlines from input
# Formatting
--heading-style {atx,atx_closed,underlined}
--highlight-style {double-equal,html,bold}
--strong-em-symbol {*,_}
--bullets CHARS # e.g., "*+-"
# Text escaping
--no-escape-asterisks # Disable * escaping
--no-escape-underscores # Disable _ escaping
--no-escape-misc # Disable misc character escaping
# Large document processing
--stream-processing # Enable streaming mode
--chunk-size SIZE # Set chunk size (default: 1024)
--show-progress # Show progress for large files
# Text wrapping
--wrap # Enable text wrapping
--wrap-width WIDTH # Set wrap width (default: 80)
View all available options:
html_to_markdown --help
For existing projects using Markdownify, a compatibility layer is provided:
# Old code
from markdownify import markdownify as md
# New code - works the same way
from html_to_markdown import markdownify as md
The markdownify
function is an alias for convert_to_markdown
and provides identical functionality.
Note: While the compatibility layer ensures existing code continues to work, new projects should use convert_to_markdown
directly as it provides better type hints and clearer naming.
Complete list of all configuration options:
extract_metadata
(bool, default:True
): Extract document metadata (title, meta tags) as comment headerconvert_as_inline
(bool, default:False
): Treat content as inline elements only (no block elements)strip_newlines
(bool, default:False
): Remove newlines from HTML input before processingconvert
(list, default:None
): List of HTML tags to convert (None = all supported tags)strip
(list, default:None
): List of HTML tags to remove from outputcustom_converters
(dict, default:None
): Mapping of HTML tag names to custom converter functions
stream_processing
(bool, default:False
): Enable streaming processing for large documentschunk_size
(int, default:1024
): Size of chunks when using streaming processingchunk_callback
(callable, default:None
): Callback function called with each processed chunkprogress_callback
(callable, default:None
): Callback function called with (processed_bytes, total_bytes)
heading_style
(str, default:'underlined'
): Header style ('underlined'
,'atx'
,'atx_closed'
)highlight_style
(str, default:'double-equal'
): Style for highlighted text ('double-equal'
,'html'
,'bold'
)strong_em_symbol
(str, default:'*'
): Symbol for strong/emphasized text ('*'
or'_'
)bullets
(str, default:'*+-'
): Characters to use for bullet points in listsnewline_style
(str, default:'spaces'
): Style for handling newlines ('spaces'
or'backslash'
)sub_symbol
(str, default:''
): Custom symbol for subscript textsup_symbol
(str, default:''
): Custom symbol for superscript text
escape_asterisks
(bool, default:True
): Escape*
characters to prevent unintended formattingescape_underscores
(bool, default:True
): Escape_
characters to prevent unintended formattingescape_misc
(bool, default:True
): Escape miscellaneous characters to prevent Markdown conflicts
autolinks
(bool, default:True
): Automatically convert valid URLs to Markdown linksdefault_title
(bool, default:False
): Use default titles for elements like linkskeep_inline_images_in
(list, default:None
): Tags where inline images should be preserved
code_language
(str, default:''
): Default language identifier for fenced code blockscode_language_callback
(callable, default:None
): Function to dynamically determine code block language
wrap
(bool, default:False
): Enable text wrappingwrap_width
(int, default:80
): Width for text wrapping
This library is open to contribution. Feel free to open issues or submit PRs. Its better to discuss issues before submitting PRs to avoid disappointment.
-
Clone the repo
-
Install system dependencies (requires Python 3.9+)
-
Install the project dependencies:
uv sync --all-extras --dev
-
Install pre-commit hooks:
uv run pre-commit install
-
Run tests to ensure everything works:
uv run pytest
-
Run code quality checks:
uv run pre-commit run --all-files
-
Make your changes and submit a PR
# Run tests with coverage
uv run pytest --cov=html_to_markdown --cov-report=term-missing
# Lint and format code
uv run ruff check --fix .
uv run ruff format .
# Type checking
uv run mypy
# Test CLI during development
uv run python -m html_to_markdown input.html
# Build package
uv build
The library is optimized for performance with several key features:
- Efficient ancestor caching: Reduces repeated DOM traversals using context-aware caching
- Streaming support: Process large documents in chunks to minimize memory usage
- Optional lxml parser: ~30% faster parsing for complex HTML documents
- Optimized string operations: Minimizes string concatenations in hot paths
Typical throughput: ~2 MB/s for regular processing on modern hardware.
This library uses the MIT license.
This library provides comprehensive support for all modern HTML5 elements:
<article>
,<aside>
,<figcaption>
,<figure>
,<footer>
,<header>
,<hgroup>
,<main>
,<nav>
,<section>
<abbr>
,<bdi>
,<bdo>
,<cite>
,<data>
,<dfn>
,<kbd>
,<mark>
,<samp>
,<small>
,<time>
,<var>
<del>
,<ins>
(strikethrough and insertion tracking)
<form>
,<fieldset>
,<legend>
,<label>
,<input>
,<textarea>
,<select>
,<option>
,<optgroup>
<button>
,<datalist>
,<output>
,<progress>
,<meter>
- Task list support:
<input type="checkbox">
converts to- [x]
/- [ ]
<table>
,<thead>
,<tbody>
,<tfoot>
,<tr>
,<th>
,<td>
,<caption>
,<col>
,<colgroup>
<details>
,<summary>
,<dialog>
,<menu>
<ruby>
,<rb>
,<rt>
,<rtc>
,<rp>
(for East Asian typography)
<img>
,<picture>
,<audio>
,<video>
,<iframe>
- SVG support with data URI conversion
<math>
(MathML support)
This version introduces several breaking changes for improved consistency and functionality:
- Enhanced Metadata Extraction: Now enabled by default with comprehensive extraction of title, meta tags, and link relations
- Improved Newline Handling: Better normalization of excessive newlines (max 2 consecutive)
- Extended HTML5 Support: Added support for 40+ new HTML5 elements
- Streaming API: New streaming parameters for large document processing
- Task List Support: Automatic conversion of HTML checkboxes to GitHub-compatible task lists
- Highlight Styles: New
highlight_style
parameter with multiple options for<mark>
elements
Special thanks to the original markdownify project creators and contributors.