Monday, April 1, 2013

Error with exec() in java (and wget tips)

Of course, i (almost) never read extensively the documentations. I often regret it...

I came upon a problem on which i lost some precious minutes:
- Executing wget from java exec()
- The code runs well, slows down, stops and finally returns an exception.

After investigation, the problem comes from exec() which keep all the standard and error outputs in memory. If you use wget on a large file, or on a large number of file. It comes a time, where memory becomes a problem.

Here is a quick and dirty solution: not output at all.
-q
--quiet
Turn off Wget's output
-nv
--no-verbose
Turn off verbose without being completely quiet (use ‘-q’ for that), which means that error messages and basic information still get printed. 
See http://www.gnu.org/software/wget/manual/html_node/Logging-and-Input-File-Options.html#Logging-and-Input-File-Options for more information.

        String file = extractFileName(url);
        System.err.println("Download: " + url);
        try {
            Process proc = Runtime.getRuntime().exec("wget -q -nv " + url);
            proc.waitFor();
        } catch (IOException e) {
            System.err.println("wget -q -nv " + url+ " failed");
            e.printStackTrace();
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
 The proper way to deal with the problem is to capture and flush the ouput and error stream.

That's all folks!

No comments:

Post a Comment