[CMA-44] RestExecuterImpl.createFilePostMethod causes the buffering of the entire file InputStream to determine the size of the stream, hence causing an OutOfMemory: heap space exception

Fri, 15 Oct 2010 10:50:40 -0400

RestExecuterImpl.createFilePostMethod is creating an instance of InputStreamRequestEntity using http://hc.apache.org/httpclient-3.x/apidocs/org/apache/commons/httpclient/methods/InputStreamRequestEntity.html#InputStreamRequestEntity(java.io.InputStream) constructor, which implies that it will buffer the entire InputStream to determine the size of the InputStream. For large enough files, this is causing:

com.rivetlogic.core.cma.exception.CmaRuntimeException: java.lang.OutOfMemoryError: Java heap space
at com.rivetlogic.core.cma.impl.ContentServiceImpl.writeContentFromStream(ContentServiceImpl.java:124)
at com.rivetlogic.hbsp.dewey.listeners.UploadListenerAction.processUpload(UploadListenerAction.java:153)
...

There is a way to overcome this issue by providing the file size to the content service and channeling it to the RestExecuterImpl.createFilePostMethod method, and using this InputStreamRequestEntity constructor instead: http://hc.apache.org/httpclient-3.x/apidocs/org/apache/commons/httpclient/methods/InputStreamRequestEntity.html#InputStreamRequestEntity(java.io.InputStream,%20long)

This issue affects all versions of CMA/RAAr and will probably require a new writeContentFromStream method to be added to CMA/RAAr.

Rivet Logic Corporation JIRA