A heap corruption condition was observed in Oracle Java Runtime Environment version 8u40 (latest at the time of this writing) while fuzz-testing the processing of TrueType and Type-1 font files, implemented in a proprietary t2k library. It manifests itself in the form of the following (or similar) crash:
--- cut ---
$ java -cp . DisplaySfntFont sample.ttf
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007f6401811837, pid=12246, tid=140068591810304
#
# JRE version: Java(TM) SE Runtime Environment (8.0_40-b25) (build 1.8.0_40-b25)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.40-b25 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C [libt2k.so+0x2e837]*** Error in `jre1.8.0_40/bin/java': malloc(): memory corruption: 0x00007f643c3042a0 ***
Aborted (core dumped)
--- cut ---
Due to a severely corrupted heap, the exception handler crashes while attempting to print out the faulty stack trace. The crash reliably reproduces on both Windows and Linux platforms. The immediate cause of the crash appears to be a heap-based buffer overflow of significant size which occurs in the sc_FindExtrema4 function, as illustrated below in the Valgrind log:
--- cut ---
==12992== Invalid read of size 8
==12992== at 0x2A27B9F7: sc_FindExtrema4 (in jre1.8.0_40/lib/amd64/libt2k.so)
==12992== by 0x2A2630E8: fs_FindBitMapSize4 (in jre1.8.0_40/lib/amd64/libt2k.so)
==12992== by 0x2A29EB5C: MakeBWBits (in jre1.8.0_40/lib/amd64/libt2k.so)
==12992== by 0x2A29629B: T2K_RenderGlyphInternal (in jre1.8.0_40/lib/amd64/libt2k.so)
==12992== by 0x2A29665E: T2K_RenderGlyph (in jre1.8.0_40/lib/amd64/libt2k.so)
==12992== by 0x2A281D3E: Java_sun_font_T2KFontScaler_getGlyphImageNative (in jre1.8.0_40/lib/amd64/libt2k.so)
==12992== by 0x7774993: ???
==12992== by 0x776675F: ???
==12992== by 0x776675F: ???
==12992== by 0x776675F: ???
==12992== by 0x776675F: ???
==12992== by 0x776682F: ???
==12992== Address 0x22fc6ce0 is 16 bytes inside a block of size 18 alloc'd
==12992== at 0x40307C4: malloc (valgrind/coregrind/m_replacemalloc/vg_replace_malloc.c:270)
==12992== by 0x2A2A5C1D: tsi_AllocMem (in jre1.8.0_40/lib/amd64/libt2k.so)
==12992== by 0x2A27B573: sc_FindExtrema4 (in jre1.8.0_40/lib/amd64/libt2k.so)
==12992== by 0x2A2630E8: fs_FindBitMapSize4 (in jre1.8.0_40/lib/amd64/libt2k.so)
==12992== by 0x2A29EB5C: MakeBWBits (in jre1.8.0_40/lib/amd64/libt2k.so)
==12992== by 0x2A29629B: T2K_RenderGlyphInternal (in jre1.8.0_40/lib/amd64/libt2k.so)
==12992== by 0x2A29665E: T2K_RenderGlyph (in jre1.8.0_40/lib/amd64/libt2k.so)
==12992== by 0x2A281D3E: Java_sun_font_T2KFontScaler_getGlyphImageNative (in jre1.8.0_40/lib/amd64/libt2k.so)
==12992== by 0x7774993: ???
==12992== by 0x776675F: ???
==12992== by 0x776675F: ???
==12992== by 0x776675F: ???
==12992==
==12992== Invalid write of size 8
==12992== at 0x2A27B9FC: sc_FindExtrema4 (in jre1.8.0_40/lib/amd64/libt2k.so)
==12992== by 0x2A2630E8: fs_FindBitMapSize4 (in jre1.8.0_40/lib/amd64/libt2k.so)
==12992== by 0x2A29EB5C: MakeBWBits (in jre1.8.0_40/lib/amd64/libt2k.so)
==12992== by 0x2A29629B: T2K_RenderGlyphInternal (in jre1.8.0_40/lib/amd64/libt2k.so)
==12992== by 0x2A29665E: T2K_RenderGlyph (in jre1.8.0_40/lib/amd64/libt2k.so)
==12992== by 0x2A281D3E: Java_sun_font_T2KFontScaler_getGlyphImageNative (in jre1.8.0_40/lib/amd64/libt2k.so)
==12992== by 0x7774993: ???
==12992== by 0x776675F: ???
==12992== by 0x776675F: ???
==12992== by 0x776675F: ???
==12992== by 0x776675F: ???
==12992== by 0x776682F: ???
==12992== Address 0x22fc6ce0 is 16 bytes inside a block of size 18 alloc'd
==12992== at 0x40307C4: malloc (valgrind/coregrind/m_replacemalloc/vg_replace_malloc.c:270)
==12992== by 0x2A2A5C1D: tsi_AllocMem (in jre1.8.0_40/lib/amd64/libt2k.so)
==12992== by 0x2A27B573: sc_FindExtrema4 (in jre1.8.0_40/lib/amd64/libt2k.so)
==12992== by 0x2A2630E8: fs_FindBitMapSize4 (in jre1.8.0_40/lib/amd64/libt2k.so)
==12992== by 0x2A29EB5C: MakeBWBits (in jre1.8.0_40/lib/amd64/libt2k.so)
==12992== by 0x2A29629B: T2K_RenderGlyphInternal (in jre1.8.0_40/lib/amd64/libt2k.so)
==12992== by 0x2A29665E: T2K_RenderGlyph (in jre1.8.0_40/lib/amd64/libt2k.so)
==12992== by 0x2A281D3E: Java_sun_font_T2KFontScaler_getGlyphImageNative (in jre1.8.0_40/lib/amd64/libt2k.so)
==12992== by 0x7774993: ???
==12992== by 0x776675F: ???
==12992== by 0x776675F: ???
==12992== by 0x776675F: ???
--- cut ---
The mutated testcases minimize to <4 byte differences, which reside mostly inside of the TrueType program sections. However, it is not clear which mutations specifically trigger the condition, or what the root cause of the problem is.
Attached with this report are three mutated testcases together with original files used to create them, and a simple Java program used to reproduce the vulnerability by loading TrueType/OpenType fonts specified through a command-line parameter.
This bug is subject to a 90 day disclosure deadline. If 90 days elapse without a broadly available patch, then the bug report will automatically become visible to the public.