David Larlet avatar David Larlet committed b1bd0ff

Add the database storage from snippet 1305, thanks Ivanov E.

Comments (0)

Files changed (4)

+syntax:glob
+
+*.elc
+*.pyc
+*~
+*.orig
+*.log
+*.swp
+*.tmp
+*.DS_Store
+testdb.sqlite
+django
     * Andrew McClain (MogileFS)
     * Rafal Jonca (FTP)
     * Chris McCormick (S3 with Boto)
+    * Ivanov E. (Database)
 
 Extra thanks to Marty for adding this in Django, 
 you can buy his very interesting book (Pro Django).

DatabaseStorage.py

+# DatabaseStorage for django.
+# 2009 (c) GameKeeper Gambling Ltd, Ivanov E.
+
+
+from django.core.files.storage import Storage
+from django.core.files import File
+from django.conf import settings
+
+import StringIO
+import urlparse
+
+import pyodbc
+
+class DatabaseStorage(Storage):
+    """
+    Class DatabaseStorage provides storing files in the database. 
+    """
+
+    def __init__(self, option=settings.DB_FILES):
+        """Constructor. 
+        
+        Constructs object using dictionary either specified in contucotr or
+in settings.DB_FILES. 
+        
+        @param option dictionary with 'db_table', 'fname_column',
+'blob_column', 'size_column', 'base_url'  keys. 
+        
+        option['db_table']
+            Table to work with.
+        option['fname_column']
+            Column in the 'db_table' containing filenames (filenames can
+contain pathes). Values should be the same as where FileField keeps
+filenames. 
+            It is used to map filename to blob_column. In sql it's simply
+used in where clause. 
+        option['blob_column']
+            Blob column (for example 'image' type), created manually in the
+'db_table', used to store image.
+        option['size_column']
+            Column to store file size. Used for optimization of size()
+method (another way is to open file and get size)
+        option['base_url']
+            Url prefix used with filenames. Should be mapped to the view,
+that returns an image as result. 
+        """
+        
+        if not option or not (option.has_key('db_table') and option.has_key('fname_column') and option.has_key('blob_column')
+                              and option.has_key('size_column') and option.has_key('base_url') ):
+            raise ValueError("You didn't specify required options")
+        self.db_table = option['db_table']
+        self.fname_column = option['fname_column']
+        self.blob_column = option['blob_column']
+        self.size_column = option['size_column']
+        self.base_url = option['base_url']
+
+        #get database settings
+        self.DATABASE_ODBC_DRIVER = settings.DATABASE_ODBC_DRIVER
+        self.DATABASE_NAME = settings.DATABASE_NAME
+        self.DATABASE_USER = settings.DATABASE_USER
+        self.DATABASE_PASSWORD = settings.DATABASE_PASSWORD
+        self.DATABASE_HOST = settings.DATABASE_HOST
+        
+        self.connection = pyodbc.connect('DRIVER=%s;SERVER=%s;DATABASE=%s;UID=%s;PWD=%s'%(self.DATABASE_ODBC_DRIVER,self.DATABASE_HOST,self.DATABASE_NAME,
+                                                                                          self.DATABASE_USER, self.DATABASE_PASSWORD) )
+        self.cursor = self.connection.cursor()
+
+    def _open(self, name, mode='rb'):
+        """Open a file from database. 
+        
+        @param name filename or relative path to file based on base_url. path should contain only "/", but not "\". Apache sends pathes with "/".
+        If there is no such file in the db, returs None
+        """
+        
+        assert mode == 'rb', "You've tried to open binary file without specifying binary mode! You specified: %s"%mode
+
+        row = self.cursor.execute("SELECT %s from %s where %s = '%s'"%(self.blob_column,self.db_table,self.fname_column,name) ).fetchone()
+        if row is None:
+            return None
+        inMemFile = StringIO.StringIO(row[0])
+        inMemFile.name = name
+        inMemFile.mode = mode
+        
+        retFile = File(inMemFile)
+        return retFile
+
+    def _save(self, name, content):
+        """Save 'content' as file named 'name'.
+        
+        @note '\' in path will be converted to '/'. 
+        """
+        
+        name = name.replace('\\', '/')
+        binary = pyodbc.Binary(content.read())
+        size = len(binary)
+        
+        #todo: check result and do something (exception?) if failed.
+        if self.exists(name):
+            self.cursor.execute("UPDATE %s SET %s = ?, %s = ? WHERE %s = '%s'"%(self.db_table,self.blob_column,self.size_column,self.fname_column,name), 
+                                 (binary, size)  )
+        else:
+            self.cursor.execute("INSERT INTO %s VALUES(?, ?, ?)"%(self.db_table), (name, binary, size)  )
+        self.connection.commit()
+        return name
+
+    def exists(self, name):
+        row = self.cursor.execute("SELECT %s from %s where %s = '%s'"%(self.fname_column,self.db_table,self.fname_column,name)).fetchone()
+        return row is not None
+    
+    def get_available_name(self, name):
+        return name
+
+    def delete(self, name):
+        if self.exists(name):
+            self.cursor.execute("DELETE FROM %s WHERE %s = '%s'"%(self.db_table,self.fname_column,name))
+            self.connection.commit()
+
+    def url(self, name):
+        if self.base_url is None:
+            raise ValueError("This file is not accessible via a URL.")
+        return urlparse.urljoin(self.base_url, name).replace('\\', '/')
+    
+    def size(self, name):
+        row = self.cursor.execute("SELECT %s from %s where %s = '%s'"%(self.size_column,self.db_table,self.fname_column,name)).fetchone()
+        if row is None:
+            return 0
+        else:
+            return int(row[0])

django-mogilefs-storage.txt

 stored by mogile (say, an avatar), the URL gets passed to a view which, using 
 a client to the mogile tracker, retrieves the "correct" path (the path that 
 points to the actual file data). The view will then either return the path(s) 
-to perlbal to reproxy, or, if youre not using perlbal to reproxy 
+to perlbal to reproxy, or, if you're not using perlbal to reproxy 
 (which you should), it serves the data of the file directly from django.
 
 In order for the backend to work, we need to add a few settings variables:
 
     * ``MOGILEFS_DOMAIN``: The mogile domain that files should read 
-      from/written to, e.g production
+      from/written to, e.g "production"
     * ``MOGILEFS_TRACKERS``: A list of trackers to connect to, 
       e.g. ["foo.sample.com:7001", "bar.sample.com:7001"]
     * ``MOGILEFS_MEDIA_URL`` (optional): The prefix for URLs that point to 
     * ``SERVE_WITH_PERLBAL``: Boolean that, when True, will pass the paths 
       back in the response in the ``X-REPROXY-URL`` header. If False, django 
       will serve all mogile media files itself (bad idea for production, 
-      but useful if youre testing on a setup that doesnt have perlbal 
+      but useful if you're testing on a setup that doesn't have perlbal 
       running)
-    * ``DEFAULT_FILE_STORAGE``: This is the class that’s used for the backend.
-      You’ll want to set this to ``project.app.storages.MogileFSStorage``
-      (or wherever you’ve installed the backend) 
+    * ``DEFAULT_FILE_STORAGE``: This is the class that's used for the backend.
+      You'll want to set this to ``project.app.storages.MogileFSStorage``
+      (or wherever you've installed the backend) 
 
  
 
 -------------------------
 
 The great thing about file backends is that we just need to specify the 
-backend in the model file and everything is taken care for us  all the 
+backend in the model file and everything is taken care for us - all the 
 default save() methods work correctly.
 
 For Fluther, we have two main media types we use mogile for: avatars and 
-thumbnails. Mogile defines “classes” that dictate how each type of file is 
-replicated — so you can make sure you have 3 copies of the original avatar 
+thumbnails. Mogile defines "classes" that dictate how each type of file is 
+replicated - so you can make sure you have 3 copies of the original avatar 
 but only 1 of the thumbnail.
 
-In order for classes to behave nicely with the backend framework, weve had to 
+In order for classes to behave nicely with the backend framework, we've had to 
 do a little tomfoolery. (This is something that may change in future versions 
 of the filestorage framework).
 
-Heres what the models.py file looks like for the avatars::
+Here's what the models.py file looks like for the avatars::
 
     from django.core.filestorage import storage
     
     # TODO: Find a better way to deal with classes. Maybe a generator?
     class AvatarStorage(storage.__class__):
-        mogile_class = avatar 
+        mogile_class = 'avatar' 
     
     class ThumbnailStorage(storage.__class__):
-        mogile_class = thumb
+        mogile_class = 'thumb'
     
     class Avatar(models.Model):
         user = models.ForeignKey(User, null=True, blank=True)
         thumb = models.ImageField(storage=ThumbnailStorage())
 
 Each of the custom storage classes defines a ``class`` attribute which gets 
-passed to the mogile backend behind the scenes.  If you don’t want to worry 
-about mogile classes, don’t need to define a custom storage engine or specify 
-it in the field — the default should work just fine.
+passed to the mogile backend behind the scenes.  If you don't want to worry 
+about mogile classes, don't need to define a custom storage engine or specify 
+it in the field - the default should work just fine.
 
 Serving files from mogile
 -------------------------
 
 Now, all we need to do is plug in the view that serves up mogile data. 
 
-Heres what we use::
+Here's what we use::
 
-  urlpatterns += patterns(”,
-      (r’^%s(?P<key>.*)’ % settings.MOGILEFS_MEDIA_URL[1:], 
+  urlpatterns += patterns(",
+      (r'^%s(?P<key>.*)' % settings.MOGILEFS_MEDIA_URL[1:], 
           'MogileFSStorage.serve_mogilefs_file')
   )
 
 ``MEDIA_URL``), we strip that off and pass the rest of the url over to the 
 view.
 
-Thats it! Happy mogiling!
+That's it! Happy mogiling!
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.