[blivet master] Handle unicode strings in Size spec parsing.
David Shea
dshea at redhat.com
Wed Oct 22 14:03:51 UTC 2014
On 10/22/2014 08:48 AM, Anne Mulhern wrote:
>
>
>
> ----- Original Message -----
>> From: "David Shea" <dshea at redhat.com>
>> To: anaconda-patches at lists.fedorahosted.org
>> Sent: Tuesday, October 21, 2014 5:08:43 PM
>> Subject: [blivet master] Handle unicode strings in Size spec parsing.
>>
>> Since _() and friends now return unicode objects, most of size.py no
>> longer needs to decode strings into unicode objects. Change
>> humanReadable to return unicode objects that will be converted back to
>> str by __str__ if necessary. Handle unicode inputs to the Size
>> constructor.
>> ---
>> blivet/size.py | 24 ++++++++++++------------
>> 1 file changed, 12 insertions(+), 12 deletions(-)
>>
>> diff --git a/blivet/size.py b/blivet/size.py
>> index ec09bf0..6dafdcc 100644
>> --- a/blivet/size.py
>> +++ b/blivet/size.py
>> @@ -75,12 +75,11 @@ _BYTES = [N_(b'B'), N_(b'b'), N_(b'byte'), N_(b'bytes')]
>> _PREFIXES = _BINARY_PREFIXES + _DECIMAL_PREFIXES
>>
>> # Translated versions of the byte and prefix arrays as lazy comprehensions
>> -# All strings are decoded as utf-8 so that locale-specific upper/lower
>> functions work
>> def _xlated_bytes():
>> - return (_(b).decode("utf-8") for b in _BYTES)
>> + return (_(b) for b in _BYTES)
>>
>> def _xlated_prefix(p):
>> - return _Prefix(p.factor, _(p.prefix).decode("utf-8"),
>> _(p.abbr).decode("utf-8"))
>> + return _Prefix(p.factor, _(p.prefix), _(p.abbr))
>>
>> def _xlated_binary_prefixes():
>> return (_xlated_prefix(p) for p in _BINARY_PREFIXES)
>> @@ -102,15 +101,15 @@ def _makeSpecs(prefix, abbr, xlate):
>>
>> if prefix:
>> if xlate:
>> - specs.append(prefix.lower() + _(b"byte").decode("utf-8"))
>> - specs.append(prefix.lower() + _(b"bytes").decode("utf-8"))
>> + specs.append(prefix.lower() + _(b"byte"))
>> + specs.append(prefix.lower() + _(b"bytes"))
>> else:
>> specs.append(_lowerASCII(prefix) + "byte")
>> specs.append(_lowerASCII(prefix) + "bytes")
>>
>> if abbr:
>> if xlate:
>> - specs.append(abbr.lower() + _(b"b").decode("utf-8"))
>> + specs.append(abbr.lower() + _(b"b"))
>> specs.append(abbr.lower())
>> else:
>> specs.append(_lowerASCII(abbr) + "b")
>> @@ -172,10 +171,13 @@ def _parseSpec(spec):
>> if spec_ascii in check:
>> return size * factor
>>
>> - # No English match found, try localized size specs. Accept any utf-8
>> - # character and leave the result as a (unicode object.
>> + # No English match found, try localized size specs. Convert the
>> + # specifier to a unicode string if it's not one already.
>> if six.PY2:
>> - spec_local = specifier.decode("utf-8")
>> + if isinstance(specifier, unicode):
>> + spec_local = specifier
>> + else:
>> + spec_local = specifier.decode("utf-8")
>> else:
>> # str = unicode in Python3
>> spec_local = specifier
>> @@ -357,7 +359,5 @@ class Size(Decimal):
>> if radix != '.':
>> retval_str = retval_str.replace('.', radix)
>>
>> - # Convert unicode objects to str before concatenating so that the
>> - # resulting expression is a str.
>> # pylint: disable=undefined-loop-variable
>> - return retval_str + " " + abbr.encode("utf-8") + _("B")
>> + return retval_str + " " + abbr + _("B")
>> --
>> 2.1.0
>>
>> _______________________________________________
>> anaconda-patches mailing list
>> anaconda-patches at lists.fedorahosted.org
>> https://lists.fedorahosted.org/mailman/listinfo/anaconda-patches
>>
> If try block under:
>
> # Only attempt to parse as English if all characters are ASCII
>
> comment still make sense in current situation, Ack.
>
> - mulhern
It's still valid in the sense that it still works, since python2
unicode.decode('ascii') can be used as a test for non-ascii characters
in the same way as str.decode('ascii')
>>> u'asdf'.decode('ascii')
u'asdf'
>>> 'asdf'.decode('ascii')
u'asdf'
>>> u'ásdf'.decode('ascii')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe1' in
position 0: ordinal not in range
The part where it gets converted back to str may not be the cleanest or
clearest, but it makes string.maketrans work for the _lowerASCII method.
I'll change the "Convert back to a str type to match the _BYTES and
_PREFIXES arrays" comment to mention that, instead.
In Python 3 there are greater problems than the str being converted to
bytes, such as string.maketrans not existing at all.
More information about the anaconda-patches
mailing list