Skip to content

Instantly share code, notes, and snippets.

@sirrice

sirrice/get_me_utf8_unicode.py

Created March 22, 2013 22:23
Show Gist options
  • Save sirrice/5225239 to your computer and use it in GitHub Desktop.
Save sirrice/5225239 to your computer and use it in GitHub Desktop.
Download ZIP
For those times when you have no idea what type of string you received are, and what encoding scheme it's encoded in. Just turn the damn thing into utf-8 unicode.
Raw
get_me_utf8_unicode.py
def get_me_unicode(v):
if isinstance(v, unicode):
s = v.encode('utf-8', errors='ignore')
elif isinstance(v, basestring):
s = unicode(v, 'utf-8', errors='ignore').encode('utf-8', errors='ignore')
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment