yanchuan sim avatar yanchuan sim committed c138277

added save_to parameter for downloading

Comments (0)

Files changed (1)

ycutils/urls/webpages.py

 MAX_TRIES = 3
 """Maximum number of tries to download each URL."""
 
-def download(url, referer=None, wget_path=WGET_PATH, user_agent=USER_AGENT, max_tries=MAX_TRIES):
+def download(url, referer=None, wget_path=WGET_PATH, user_agent=USER_AGENT, max_tries=MAX_TRIES, save_to=None):
   """
   Downloads the given URL by passing it through `Wget <http://www.gnu.org/software/wget/>`_.
 
   :param wget_path: path to Wget executable. Defaults to Wget on your system path (see :attr:`WGET_PATH`).
   :param user_agent: user agent string to identify yourself as. Defaults to :attr:`USER_AGENT`.
   :param max_tries: number of times for Wget to retry. Defaults to :attr:`MAX_TRIES` setting.
+  :param save_to: location of the file to save to. Defaults to the temporary directory and file will be removed after downloading.
   :returns: ``dict`` object containing information about the download. 
   
   The ``dict`` will contain the following keys if the download was a success:
   .. note:: If the server sends back a GZip compressed file, we will attempt to deflate it using the Python :mod:`gzip` and :mod:`zlib` libraries.
   """
 
-  save_path = tempfile.NamedTemporaryFile(delete=False).name
+  if save_to: save_path = save_to
+  else: save_path = tempfile.NamedTemporaryFile(delete=False).name
 
   wget_command = [wget_path, url, '--server-response', '--quiet', '--output-document', save_path, '--user-agent', user_agent, '--referer', referer if referer else url, '--no-check-certificate', '--tries', str(max_tries)]
   p = subprocess.Popen(wget_command, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, close_fds=True)
   if return_code == 0 and os.path.exists(save_path):
     with open(save_path, 'rb') as f: content = f.read()
     try:
-      os.unlink(save_path)
+      if not save_to: os.unlink(save_path)
     except OSError: pass
 
     del save_path
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.