[blivet master] Handle unicode strings in Size spec parsing.
Anne Mulhern
amulhern at redhat.com
Wed Oct 22 15:02:18 UTC 2014
----- Original Message -----
> From: "David Shea" <dshea at redhat.com>
> To: anaconda-patches at lists.fedorahosted.org
> Sent: Wednesday, October 22, 2014 10:03:51 AM
> Subject: Re: [blivet master] Handle unicode strings in Size spec parsing.
>
> On 10/22/2014 08:48 AM, Anne Mulhern wrote:
> >
> >
> >
> > ----- Original Message -----
> >> From: "David Shea" <dshea at redhat.com>
> >> To: anaconda-patches at lists.fedorahosted.org
> >> Sent: Tuesday, October 21, 2014 5:08:43 PM
> >> Subject: [blivet master] Handle unicode strings in Size spec parsing.
> >>
> >> Since _() and friends now return unicode objects, most of size.py no
> >> longer needs to decode strings into unicode objects. Change
> >> humanReadable to return unicode objects that will be converted back to
> >> str by __str__ if necessary. Handle unicode inputs to the Size
> >> constructor.
> >> ---
> >> blivet/size.py | 24 ++++++++++++------------
> >> 1 file changed, 12 insertions(+), 12 deletions(-)
> >>
> >> diff --git a/blivet/size.py b/blivet/size.py
> >> index ec09bf0..6dafdcc 100644
> >> --- a/blivet/size.py
> >> +++ b/blivet/size.py
> >> @@ -75,12 +75,11 @@ _BYTES = [N_(b'B'), N_(b'b'), N_(b'byte'),
> >> N_(b'bytes')]
> >> _PREFIXES = _BINARY_PREFIXES + _DECIMAL_PREFIXES
> >>
> >> # Translated versions of the byte and prefix arrays as lazy
> >> comprehensions
> >> -# All strings are decoded as utf-8 so that locale-specific upper/lower
> >> functions work
> >> def _xlated_bytes():
> >> - return (_(b).decode("utf-8") for b in _BYTES)
> >> + return (_(b) for b in _BYTES)
> >>
> >> def _xlated_prefix(p):
> >> - return _Prefix(p.factor, _(p.prefix).decode("utf-8"),
> >> _(p.abbr).decode("utf-8"))
> >> + return _Prefix(p.factor, _(p.prefix), _(p.abbr))
> >>
> >> def _xlated_binary_prefixes():
> >> return (_xlated_prefix(p) for p in _BINARY_PREFIXES)
> >> @@ -102,15 +101,15 @@ def _makeSpecs(prefix, abbr, xlate):
> >>
> >> if prefix:
> >> if xlate:
> >> - specs.append(prefix.lower() + _(b"byte").decode("utf-8"))
> >> - specs.append(prefix.lower() + _(b"bytes").decode("utf-8"))
> >> + specs.append(prefix.lower() + _(b"byte"))
> >> + specs.append(prefix.lower() + _(b"bytes"))
> >> else:
> >> specs.append(_lowerASCII(prefix) + "byte")
> >> specs.append(_lowerASCII(prefix) + "bytes")
> >>
> >> if abbr:
> >> if xlate:
> >> - specs.append(abbr.lower() + _(b"b").decode("utf-8"))
> >> + specs.append(abbr.lower() + _(b"b"))
> >> specs.append(abbr.lower())
> >> else:
> >> specs.append(_lowerASCII(abbr) + "b")
> >> @@ -172,10 +171,13 @@ def _parseSpec(spec):
> >> if spec_ascii in check:
> >> return size * factor
> >>
> >> - # No English match found, try localized size specs. Accept any utf-8
> >> - # character and leave the result as a (unicode object.
> >> + # No English match found, try localized size specs. Convert the
> >> + # specifier to a unicode string if it's not one already.
> >> if six.PY2:
> >> - spec_local = specifier.decode("utf-8")
> >> + if isinstance(specifier, unicode):
> >> + spec_local = specifier
> >> + else:
> >> + spec_local = specifier.decode("utf-8")
> >> else:
> >> # str = unicode in Python3
> >> spec_local = specifier
> >> @@ -357,7 +359,5 @@ class Size(Decimal):
> >> if radix != '.':
> >> retval_str = retval_str.replace('.', radix)
> >>
> >> - # Convert unicode objects to str before concatenating so that the
> >> - # resulting expression is a str.
> >> # pylint: disable=undefined-loop-variable
> >> - return retval_str + " " + abbr.encode("utf-8") + _("B")
> >> + return retval_str + " " + abbr + _("B")
> >> --
> >> 2.1.0
> >>
> >> _______________________________________________
> >> anaconda-patches mailing list
> >> anaconda-patches at lists.fedorahosted.org
> >> https://lists.fedorahosted.org/mailman/listinfo/anaconda-patches
> >>
> > If try block under:
> >
> > # Only attempt to parse as English if all characters are ASCII
> >
> > comment still make sense in current situation, Ack.
> >
> > - mulhern
>
> It's still valid in the sense that it still works, since python2
> unicode.decode('ascii') can be used as a test for non-ascii characters
> in the same way as str.decode('ascii')
>
> >>> u'asdf'.decode('ascii')
> u'asdf'
> >>> 'asdf'.decode('ascii')
> u'asdf'
> >>> u'ásdf'.decode('ascii')
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> UnicodeEncodeError: 'ascii' codec can't encode character u'\xe1' in
> position 0: ordinal not in range
>
> The part where it gets converted back to str may not be the cleanest or
> clearest, but it makes string.maketrans work for the _lowerASCII method.
> I'll change the "Convert back to a str type to match the _BYTES and
> _PREFIXES arrays" comment to mention that, instead.
OK, thanks.
> In Python 3 there are greater problems than the str being converted to
> bytes, such as string.maketrans not existing at all.
>
Hmmm, yeah, I see we're not so Python 3 compatible yet.
_______________________________________________
> anaconda-patches mailing list
> anaconda-patches at lists.fedorahosted.org
> https://lists.fedorahosted.org/mailman/listinfo/anaconda-patches
>
- mulhern
More information about the anaconda-patches
mailing list