harvardnlp/im2markup

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1259 stars · 215 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A deep learning system (built on Torch) that converts images of rendered text into presentational markup such as LaTeX or HTML, using a CNN encoder with attention-based neural machine translation. It is the reference implementation of the 'What You Get Is What You See' image-to-markup decompiler paper.

Use cases

• convert images of math formulas to latex
• image to markup generation
• ocr for mathematical expressions
• extract latex from rendered formula screenshots
• train an image-to-text sequence model
• research on attention-based image captioning of markup

When to choose

• you need to decompile rendered LaTeX or HTML images back into source markup
• you are reproducing or extending the im2markup research paper
• you need attention alignment visualizations between output markup and input image

When to avoid

• you need a maintained production OCR tool - the project depends on the deprecated Torch framework
• you have no GPU, since only GPU execution is supported
• you want a plug-and-play API rather than a research codebase

Facets

library · maturity maintenance

machine-learning deep-learning ocr image-processing nlp deep-learning computer-vision artificial-intelligence python lua image-to-latex encoder-decoder attention torch math-ocr markup-generation natural-language-processing gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="harvardnlp/im2markup")