OutOfMemoryError while uploading videos

[ravishankar-dale]

Issue: The SDK throws an OutOfMemoryError when I try to upload videos that are larger than 50MB in size
SDK Version: v1.2.2
Android Version: All versions
Device: All devices
Video file size: 65MB
Logs:

Throwing OutOfMemoryError "Failed to allocate a 2060 byte allocation with 1380 free bytes and 1380B until OOM"
		

FATAL EXCEPTION: RxIoScheduler-3
	Process: my.package.name, PID: 5028
         	java.lang.IllegalStateException: Fatal Exception thrown on Scheduler.Worker thread.
                 		   at rx.internal.schedulers.ScheduledAction.run(ScheduledAction.java:59)
                             at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:422)
                             at java.util.concurrent.FutureTask.run(FutureTask.java:237)
                             at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:152)
                             at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:265)
                             at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
                             at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
                             at java.lang.Thread.run(Thread.java:818)
                          Caused by: java.lang.OutOfMemoryError: Failed to allocate a 2060 byte allocation with 1380 free bytes and 1380B until OOM
                             at com.android.okio.Segment.<init>(Segment.java:34)
                             at com.android.okio.SegmentPool.take(SegmentPool.java:48)
                             at com.android.okio.OkBuffer.writableSegment(OkBuffer.java:511)
                             at com.android.okio.OkBuffer.write(OkBuffer.java:424)
                             at com.android.okio.OkBuffer.clone(OkBuffer.java:740)
                             at com.android.okhttp.internal.http.RetryableSink.writeToSocket(RetryableSink.java:77)
                             at com.android.okhttp.internal.http.HttpConnection.writeRequestBody(HttpConnection.java:240)
                             at com.android.okhttp.internal.http.HttpTransport.writeRequestBody(HttpTransport.java:77)
                             at com.android.okhttp.internal.http.HttpEngine.readResponse(HttpEngine.java:622)
                             at com.android.okhttp.internal.http.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:388)
                             at com.android.okhttp.internal.http.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:332)
                             at com.android.okhttp.internal.http.HttpURLConnectionImpl.getResponseCode(HttpURLConnectionImpl.java:500)
                             at com.android.okhttp.internal.http.DelegatingHttpsURLConnection.getResponseCode(DelegatingHttpsURLConnection.java:105)
                             at com.android.okhttp.internal.http.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:25)
                             at com.cloudinary.android.UploaderStrategy.callApi(UploaderStrategy.java:81)
                             at com.cloudinary.Uploader.callApi(Uploader.java:22)
                             at com.cloudinary.Uploader.upload(Uploader.java:55)
                             at my.package.name.common.utils.CloudinaryUtils.uploadMediaInternal(CloudinaryUtils.java:106)
                             at my.package.name.common.utils.CloudinaryUtils.lambda$uploadMedia$0(CloudinaryUtils.java:90)
                             at my.package.name.common.utils.CloudinaryUtils.access$lambda$0(CloudinaryUtils.java:0)
                             at my.package.name.common.utils.CloudinaryUtils$$Lambda$1.call(Unknown Source)
                             at rx.Observable.unsafeSubscribe(Observable.java:9860)
                             at rx.internal.operators.OperatorSubscribeOn$1.call(OperatorSubscribeOn.java:94)
                             at rx.internal.schedulers.CachedThreadScheduler$EventLoopWorker$1.call(CachedThreadScheduler.java:221)
                             at rx.internal.schedulers.ScheduledAction.run(ScheduledAction.java:55)
                             at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:422) 
                             at java.util.concurrent.FutureTask.run(FutureTask.java:237) 
                             at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:152) 
                             at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:265) 
                             at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112) 
                             at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587) 
                             at java.lang.Thread.run(Thread.java:818) 

[MaorGariv]

You can try to allocate a larger heap for you app -

<application android:largeHeap="true"> </application>

Note that it can also happen as a result of a memory leak. There is some useful info and additional reading materials in this S.O answer.

Let me know of any updates.

[ravishankar-dale]

Thanks for the reply!

We did't find any memory leaks in the app, we used Square's Leak Canary to verify.

We don't want to use "largeHeap" as according to the documentation:

Most apps should not need this and should instead focus on reducing their overall memory usage for improved performance. Enabling this also does not guarantee a fixed increase in available memory, because some devices are constrained by their total available memory.

Also, even if we using the largeHeap flag, the SDK is throwing an error saying "trying to upload 64MB where the limit is 40MB". We couldn't find any mention of this limit anywhere on your website.

[MaorGariv]

Looks like the error is due to your plan limits ( video file size limit set to 40 MB).
You can see your plan limits when you login to the console.

If you have further questions or requests regarding the limits, please open a new support ticket (for privacy purposes).

[mustafa01ali]

Just requesting the system for a larger heap for this issue IMO is a bad idea. It doesn't solve the actual problem and can potentially deteriorate the end-user experience since other apps on their devices will be killed more aggressively by the OS.

The actual cause of the OOM here is in the way HttpUrlConnection is being used to POST the file. The SDK is not setting a streaming mode to the connection object before transferring the bytes, which is causing the buffer to exhaust the heap allocated to the app and throw the exception.

As per the official documentation -

For best performance, you should call either setFixedLengthStreamingMode(int) when the body length is known in advance, or setChunkedStreamingMode(int) when it is not. Otherwise HttpURLConnection will be forced to buffer the complete request body in memory before it is transmitted, wasting (and possibly exhausting) heap and increasing latency.

Based on my limited knowledge of the SDK, adding a single-line in com.cloudinary.MultipartUtility's constructor should fix this - connection.setChunkedStreamingMode(0);

[nadavofi]

Thank you, Mustafa, for making this contribution.
We will review this and release a fix accordingly.