Download large files using large byte array causes "Out of memory"

user5395084

I want to download a 100MB file using streams:
I used an AsyncTask class to make download:

    @Override
    protected Void doInBackground(String... params) {
        try {
            URL url = new URL(params[0]);
            connection = (HttpURLConnection) url.openConnection();
            connection.connect();
            inputStream = connection.getInputStream();
            outputStream = new FileOutputStream(new File(Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_DOWNLOADS), params[1]));
            byte[] data = new byte[104857600];
            int length = inputStream.read(data);
            outputStream.write(data, 0, length);
        } catch (Exception e) {
            Log.e(TAG, "Unable to download file with DownloadTask: " + e.getMessage());
        } finally {
            try {
                if (inputStream != null) {
                    inputStream.close();
                }
                if (outputStream != null) {
                    outputStream.close();
                }
                connection.disconnect();
            } catch (Exception e) {
                Log.e(TAG, "Unable to cancel download file with DownloadTask: " + e.getMessage());
            }
        }
        return null;
    }

I'm sure I used the following permissions:

<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />

But when I test my app, it crashes and I see in logcat Out of memory on a 104857616-byte allocation:

10-02 10:28:35.578 29630-2246/com.example.myapp E/dalvikvm-heap: Out of memory on a 104857616-byte allocation.
10-02 10:28:35.601 29630-2246/com.example.myapp E/AndroidRuntime: FATAL EXCEPTION: AsyncTask #1
10-02 10:28:35.601 29630-2246/com.example.myapp E/AndroidRuntime: java.lang.RuntimeException: An error occured while executing doInBackground()
10-02 10:28:35.601 29630-2246/com.example.myapp E/AndroidRuntime:     at android.os.AsyncTask$3.done(AsyncTask.java:299)
10-02 10:28:35.601 29630-2246/com.example.myapp E/AndroidRuntime:     at java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:352)
10-02 10:28:35.601 29630-2246/com.example.myapp E/AndroidRuntime:     at java.util.concurrent.FutureTask.setException(FutureTask.java:219)
10-02 10:28:35.601 29630-2246/com.example.myapp E/AndroidRuntime:     at java.util.concurrent.FutureTask.run(FutureTask.java:239)
10-02 10:28:35.601 29630-2246/com.example.myapp E/AndroidRuntime:     at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:230)
10-02 10:28:35.601 29630-2246/com.example.myapp E/AndroidRuntime:     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1080)
10-02 10:28:35.601 29630-2246/com.example.myapp E/AndroidRuntime:     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:573)
10-02 10:28:35.601 29630-2246/com.example.myapp E/AndroidRuntime:     at java.lang.Thread.run(Thread.java:856)
10-02 10:28:35.601 29630-2246/com.example.myapp E/AndroidRuntime:  Caused by: java.lang.OutOfMemoryError
10-02 10:28:35.601 29630-2246/com.example.myapp E/AndroidRuntime:     at com.example.myapp.DownloaderActivity$DownloadTask.doInBackground(DownloaderActivity.java:126)
10-02 10:28:35.601 29630-2246/com.example.myapp E/AndroidRuntime:     at com.example.myapp.DownloaderActivity$DownloadTask.doInBackground(DownloaderActivity.java:107)
10-02 10:28:35.601 29630-2246/com.example.myapp E/AndroidRuntime:     at android.os.AsyncTask$2.call(AsyncTask.java:287)
10-02 10:28:35.601 29630-2246/com.example.myapp E/AndroidRuntime:     at java.util.concurrent.FutureTask.run(FutureTask.java:234)
10-02 10:28:35.601 29630-2246/com.example.myapp E/AndroidRuntime:     at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:230) 
10-02 10:28:35.601 29630-2246/com.example.myapp E/AndroidRuntime:     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1080) 
10-02 10:28:35.601 29630-2246/com.example.myapp E/AndroidRuntime:     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:573) 
10-02 10:28:35.601 29630-2246/com.example.myapp E/AndroidRuntime:     at java.lang.Thread.run(Thread.java:856)

Is there some way to download large files or accept 104857600 bytes (100MB)?

Ashiq

You have write data portion by portion as chunk. System don't provide huge memory at once for an operation.

Replace those lines:

byte[] data = new byte[104857600];
int length = inputStream.read(data);
outputStream.write(data, 0, length);

by:

byte [] buffer = new byte[1024];
int bytesRead = 0;
while((bytesRead =input.read(buffer)) != -1) {
outputStream .write(buffer, 0,
bytesRead);
}

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

From Dev

Bash script using gzip and bcftools running out of memory with large files

From Dev

Out of memory exception when using xlsx module with large files

From Dev

Download large files using a memorystream

From Dev

C# Large JSON to string causes out of memory exception

From Dev

EF - Query Large Data Sets Causes Out Of Memory Exception

From Dev

Read/download large file in PHP without running out of memory

From Dev

Out of Memory Exception when handling large files in C#

From Dev

Count number of pages in large set of pdf files : Out of memory

From Dev

Download large files using ftp protocol

From Dev

Download large files using ftp protocol

From Dev

C++ allocating large array on heap gives "out of memory exception"

From Dev

Web worker out of memory when processing large array

From Dev

Web worker out of memory when processing large array

From Java

Out of memory exception when decrypt large file using Cipher

From Dev

Compress large file using SharpZipLib causing Out Of Memory Exception

From Dev

Plotting a large number of points using matplotlib and running out of memory

From Dev

Process out of memory error using nodejs when nothing should be that large

From Java

Encrypting large byte array

From Dev

Encrypting large byte array

From Dev

Download large files in Dartlang

From Dev

Download large files not working

From Dev

Download large files in Dartlang

From Dev

large bitmap crushes (out of memory)

From Dev

use xhr.onprogress to process large ajax download without running out of memory?

From Dev

Memory issue when using large UIImage array causes crashing (swift)

From Dev

php download fails on large files

From Dev

Out of memory when creating large number of relationships

From Java

Calculate a large amount without running out of memory

From Dev

Creating a large GIF with CGImageDestinationFinalize - running out of memory

Related Related

  1. 1

    Bash script using gzip and bcftools running out of memory with large files

  2. 2

    Out of memory exception when using xlsx module with large files

  3. 3

    Download large files using a memorystream

  4. 4

    C# Large JSON to string causes out of memory exception

  5. 5

    EF - Query Large Data Sets Causes Out Of Memory Exception

  6. 6

    Read/download large file in PHP without running out of memory

  7. 7

    Out of Memory Exception when handling large files in C#

  8. 8

    Count number of pages in large set of pdf files : Out of memory

  9. 9

    Download large files using ftp protocol

  10. 10

    Download large files using ftp protocol

  11. 11

    C++ allocating large array on heap gives "out of memory exception"

  12. 12

    Web worker out of memory when processing large array

  13. 13

    Web worker out of memory when processing large array

  14. 14

    Out of memory exception when decrypt large file using Cipher

  15. 15

    Compress large file using SharpZipLib causing Out Of Memory Exception

  16. 16

    Plotting a large number of points using matplotlib and running out of memory

  17. 17

    Process out of memory error using nodejs when nothing should be that large

  18. 18

    Encrypting large byte array

  19. 19

    Encrypting large byte array

  20. 20

    Download large files in Dartlang

  21. 21

    Download large files not working

  22. 22

    Download large files in Dartlang

  23. 23

    large bitmap crushes (out of memory)

  24. 24

    use xhr.onprogress to process large ajax download without running out of memory?

  25. 25

    Memory issue when using large UIImage array causes crashing (swift)

  26. 26

    php download fails on large files

  27. 27

    Out of memory when creating large number of relationships

  28. 28

    Calculate a large amount without running out of memory

  29. 29

    Creating a large GIF with CGImageDestinationFinalize - running out of memory

HotTag

Archive