When I try to acquire MD5 using DigestUtils, java.lang.OutOfMemoryError occurs. [SPR-14427]

[spring-projects-issues]

Mitsuyoshi Hasegawa opened SPR-14427 and commented

When InputStream of a huge file size (approximately 3 GB) is given, OOM occurs by the following method.

• org.springframework.util.DigestUtils#md5Digest(InputStream inputStream)
• org.springframework.util.DigestUtils#md5DigestAsHex(InputStream inputStream)
• org.springframework.util.DigestUtils#appendMd5DigestAsHex(InputStream inputStream, StringBuilder builder)

Stack trace is as follows:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
	at java.util.Arrays.copyOf(Arrays.java:3236)
	at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:118)
	at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
	at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153)
	at org.springframework.util.StreamUtils.copy(StreamUtils.java:128)
	at org.springframework.util.StreamUtils.copyToByteArray(StreamUtils.java:60)
	at org.springframework.util.DigestUtils.digest(DigestUtils.java:131)
	at org.springframework.util.DigestUtils.digestAsHexChars(DigestUtils.java:163)
	at org.springframework.util.DigestUtils.digestAsHexString(DigestUtils.java:141)
	at org.springframework.util.DigestUtils.md5DigestAsHex(DigestUtils.java:80)
	at org.yggd.files.FileMisc.md5String(FileMisc.kt:36)

When I use org.apache.commons.codec.digest.DigestUtils in Apache Commons Codec (version 1.10) referred by the API document, It normally works.

---

Affects: 4.2.6, 4.3.1

Reference URL: http://docs.spring.io/spring/docs/current/javadoc-api/org/springframework/util/DigestUtils.html

Backported to: 4.2.8

[spring-projects-issues]

Juergen Hoeller commented

Good catch! We're processing a given InputStream with a buffered read now instead of as a byte array, even if the underlying stream does not explicitly support message digest updates like our FastByteArrayOutputStream mechanism does.