Split charset encoding form content-type

andrebrantom avatarandrebrantom created an issue

When interfacing interfacing with piston from a web-app, I kept getting bad request responses but couldn't understand why.

It turned out that piston wasn't separating out the content-type and the charset when it was getting a Content-Type like this:

"application/x-www-form-urlencoded; charset=utf-8"

I updated Mimer.content_type() as follows to deal with the problem. If you need a branch let me know, but just wanted to report this quickly!

    def content_type(self):
        """
        Returns the content type of the request in all cases where it is
        different than a submitted form - application/x-www-form-urlencoded
        """
        type_formencoded = "application/x-www-form-urlencoded"

        # added the .split(";")[0] to strip out any charset specifier that might accompany the encoding. 
        # I was getting requests through as "application/x-www-form-urlencoded; charset=utf-8" 
        # that piston did not recognise as forms. But were. 
        ctype = self.request.META.get('CONTENT_TYPE', type_formencoded).split(";")[0]
        if ctype == type_formencoded:
            return None
        return ctype

Comments (32)

  1. Anonymous

    I keep getting bad requests myself when I try setting a content type. only text/xml seems to work...

    my code: http://pastebin.org/56727

    (uncommenting the req.add_header gives bad request for anything but text/xml...) If this is not the same issue, I'm sorry...

  2. andrebrantom

    I don't think it is the same problem. In your snippet, you are encoding the sent data using urlencode, that will cause urllib2 to expect the content type to, by default, be "application/x-www-form-urlencoded" mime type.

    Just by changing the header doesn't mean that urllib2 converts the data in anyway. I think you need to json-encode your data instead (can't remember the function call off hand) and use that, and then set content-type to "application/json" and see if that works...

    Also, you might want to check out httplib2 - it seems more appropriate than straight urllib. See http://httplib2.googlecode.com/hg/doc/html/libhttplib2.html#id1 At the bottom is an example that shows sending of atom XML, following the same pattern you can send json.

  3. Anonymous

    I have bumped against this issue too. I approached it slightly differently, I just altered the ctype check with: if ctype.startswith(type_formencoded):

  4. Anonymous

    andrebrantom, thank you for the tip, using httplib2 I don't get the bad requests anymore, althought I don't know what I did wrong in my code, since it is json encoded data (using the same params in the httplib2 example just works?)

    maybe I shouldn't have urlencoded it first?

    anyway, my issue (comment #1) was unrelated to this bug it seems...

  5. Anonymous

    Same issue for me !

    I've tried to PUT some data using json.dumps and set headers={'content-type':'application/json; utf-8'} but when I overload the update method in my Handler I always receivd bad request.

    Here is the request I send to django:

    PUT /api/dns/zone/1/ HTTP/1.1 Host: localhost:6969 Accept-Encoding: identity Content-Length: 39 content-type: application/json; utf-8 user-agent: Python-httplib2/$Rev: 259 $

    "{primary: \"foobar\"}"

  6. andrebrantom

    Ryan, thanks for the pointer to GitHub. It's all a bit curious given that django-piston is the second most followed project on BitBucket (see see https://bitbucket.org/repo/all) ...would have thought they'd be a bit more responsive. Although maybe its just that all the followers they are waiting for all the defects to be fixed...

  7. Anonymous

    The current release 0.2.3 contains:

    if ctype.startswith(mime):
    

    This is wrong, as there are many mime types which are prefixes of others. For example:

    application/sgml
    application/sgml-open-catalog
    
    audio/amr	
    audio/amr-wb
    
    audio/mpeg
    audio/mpegurl
    
    video/mp4
    video/mp4v-es
    
    video/x-ms-wm
    video/x-ms-wmx
    
  8. Log in to comment
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.