[Fwd: [Python-Dev] PEP 383: Non-decodable Bytes in System Character Interfaces]
by Toshio Kuratomi
This PEP was just sent to the python-dev list as a method to go between
unicode and C byte strings in python-3.1. It would be great to get some
more input on this as we don't want the situation to get worse than it
already is.
-Toshio
-------- Original Message --------
Subject: [Python-Dev] PEP 383: Non-decodable Bytes in System Character
Interfaces
Date: Wed, 22 Apr 2009 08:50:22 +0200
From: "Martin v. Löwis" <martin(a)v.loewis.de>
To: Python-Dev <python-dev(a)python.org>, Python List <python-list(a)python.org>
I'm proposing the following PEP for inclusion into Python 3.1.
Please comment.
Regards,
Martin
PEP: 383
Title: Non-decodable Bytes in System Character Interfaces
Version: $Revision: 71793 $
Last-Modified: $Date: 2009-04-22 08:42:06 +0200 (Mi, 22. Apr 2009) $
Author: Martin v. Löwis <martin(a)v.loewis.de>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 22-Apr-2009
Python-Version: 3.1
Post-History:
Abstract
========
File names, environment variables, and command line arguments are
defined as being character data in POSIX; the C APIs however allow
passing arbitrary bytes - whether these conform to a certain encoding
or not. This PEP proposes a means of dealing with such irregularities
by embedding the bytes in character strings in such a way that allows
recreation of the original byte string.
Rationale
=========
The C char type is a data type that is commonly used to represent both
character data and bytes. Certain POSIX interfaces are specified and
widely understood as operating on character data, however, the system
call interfaces make no assumption on the encoding of these data, and
pass them on as-is. With Python 3, character strings use a
Unicode-based internal representation, making it difficult to ignore
the encoding of byte strings in the same way that the C interfaces can
ignore the encoding.
On the other hand, Microsoft Windows NT has correct the original
design limitation of Unix, and made it explicit in its system
interfaces that these data (file names, environment variables, command
line arguments) are indeed character data, by providing a
Unicode-based API (keeping a C-char-based one for backwards
compatibility).
For Python 3, one proposed solution is to provide two sets of APIs: a
byte-oriented one, and a character-oriented one, where the
character-oriented one would be limited to not being able to represent
all data accurately. Unfortunately, for Windows, the situation would
be exactly the opposite: the byte-oriented interface cannot represent
all data; only the character-oriented API can. As a consequence,
libraries and applications that want to support all user data in a
cross-platform manner have to accept mish-mash of bytes and characters
exactly in the way that caused endless troubles for Python 2.x.
With this PEP, a uniform treatment of these data as characters becomes
possible. The uniformity is achieved by using specific encoding
algorithms, meaning that the data can be converted back to bytes on
POSIX systems only if the same encoding is used.
Specification
=============
On Windows, Python uses the wide character APIs to access
character-oriented APIs, allowing direct conversion of the
environmental data to Python str objects.
On POSIX systems, Python currently applies the locale's encoding to
convert the byte data to Unicode. If the locale's encoding is UTF-8,
it can represent the full set of Unicode characters, otherwise, only a
subset is representable. In the latter case, using private-use
characters to represent these bytes would be an option. For UTF-8,
doing so would create an ambiguity, as the private-use characters may
regularly occur in the input also.
To convert non-decodable bytes, a new error handler "python-escape" is
introduced, which decodes non-decodable bytes using into a private-use
character U+F01xx, which is believed to not conflict with private-use
characters that currently exist in Python codecs.
The error handler interface is extended to allow the encode error
handler to return byte strings immediately, in addition to returning
Unicode strings which then get encoded again.
If the locale's encoding is UTF-8, the file system encoding is set to
a new encoding "utf-8b". The UTF-8b codec decodes non-decodable bytes
(which must be >= 0x80) into half surrogate codes U+DC80..U+DCFF.
Discussion
==========
While providing a uniform API to non-decodable bytes, this interface
has the limitation that chosen representation only "works" if the data
get converted back to bytes with the python-escape error handler
also. Encoding the data with the locale's encoding and the (default)
strict error handler will raise an exception, encoding them with UTF-8
will produce non-sensical data.
For most applications, we assume that they eventually pass data
received from a system interface back into the same system
interfaces. For example, and application invoking os.listdir() will
likely pass the result strings back into APIs like os.stat() or
open(), which then encodes them back into their original byte
representation. Applications that need to process the original byte
strings can obtain them by encoding the character strings with the
file system encoding, passing "python-escape" as the error handler
name.
Copyright
=========
This document has been placed in the public domain.
_______________________________________________
Python-Dev mailing list
Python-Dev(a)python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/a.badger%40gmail.com
15 years, 1 month
Fwd: Broken dependencies: sems
by Peter Lemenkov
Hello All!
=============================================
---------- Forwarded message ----------
From: <buildsys(a)fedoraproject.org>
Date: 2009/4/22
Subject: Broken dependencies: sems
To: sems-owner(a)fedoraproject.org
Копия: openssl-owner(a)fedoraproject.org
sems has broken dependencies in the development tree:
On ppc:
sems-xmlrpc2di-1.1.0-5.fc10.ppc requires libssl.so.7
On x86_64:
sems-xmlrpc2di-1.1.0-5.fc10.x86_64 requires libssl.so.7()(64bit)
On i386:
sems-xmlrpc2di-1.1.0-5.fc10.i386 requires libssl.so.7
On ppc64:
sems-xmlrpc2di-1.1.0-5.fc10.ppc64 requires libssl.so.7()(64bit)
On ppc:
sems-ivr-1.1.0-5.fc10.ppc requires libpython2.5.so.1.0
On x86_64:
sems-ivr-1.1.0-5.fc10.x86_64 requires libpython2.5.so.1.0()(64bit)
On i386:
sems-ivr-1.1.0-5.fc10.i386 requires libpython2.5.so.1.0
On ppc64:
sems-ivr-1.1.0-5.fc10.ppc64 requires libpython2.5.so.1.0()(64bit)
On ppc:
sems-python-1.1.0-5.fc10.ppc requires libpython2.5.so.1.0
On x86_64:
sems-python-1.1.0-5.fc10.x86_64 requires libpython2.5.so.1.0()(64bit)
On i386:
sems-python-1.1.0-5.fc10.i386 requires libpython2.5.so.1.0
On ppc64:
sems-python-1.1.0-5.fc10.ppc64 requires libpython2.5.so.1.0()(64bit)
Please resolve this as soon as possible.
=============================================
Ok, I rebuild it for F-11 branch and even submitted it as an update
few days ago:
https://admin.fedoraproject.org/updates/sems-1.1.0-6.fc11
Why it still not pushed to devel? Maybe I missed something?
--
With best regards!
15 years, 1 month
rawhide report: 20090421 changes
by Fedora compose checker
Compose started at Tue Apr 21 06:15:04 UTC 2009
Removed package R-Matrix
Removed package nautilus-cd-burner
Updated Packages:
PackageKit-0.4.6-4.fc11
-----------------------
* Sun Apr 19 2009 Richard Hughes <rhughes(a)redhat.com> - 0.4.6-4
- Backport 2 patches from upstream to make Presto work better with PK.
R-2.9.0-2.fc11
--------------
* Mon Apr 20 2009 Tom "spot" Callaway <tcallawa(a)redhat.com> - 2.9.0-2
- properly Provide/Obsolete R-Matrix
brasero-2.26.1-2.fc11
---------------------
* Fri Apr 17 2009 Denis Leroy <denis(a)poolshark.org> - 2.26.1-2
- Obsoletes nautilus-cd-burner
celt-0.5.2-2.fc11
-----------------
* Mon Feb 23 2009 Fedora Release Engineering <rel-eng(a)lists.fedoraproject.org> - 0.5.2-2
- Rebuilt for https://fedoraproject.org/wiki/Fedora_11_Mass_Rebuild
dmraid-1.0.0.rc15-7.fc11
------------------------
* Fri Apr 17 2009 Hans de Goede <hdegoede(a)redhat.com> - 1.0.0.rc15-7
- Fix activation of isw raid sets when the disks have serialnumber longer
then 16 characters (#490121)
fedora-logos-11.0.2-1.fc11
--------------------------
* Mon Apr 20 2009 Tom "spot" Callaway <tcallawa(a)redhat.com> 11.0.2-1
- fix missing progress files
* Sun Apr 19 2009 Lubomir Rintel <lkundrak(a)v3.sk> - 11.0.1-2
- fix bootsplash to be a bit more psychadelic
flex-2.5.35-5.fc11
------------------
* Mon Apr 20 2009 Debarshi Ray <rishi(a)fedoraproject.org> - 2.5.35-5
- Resolves: #496548.
* Tue Feb 24 2009 Fedora Release Engineering <rel-eng(a)lists.fedoraproject.org> - 2.5.35-3
- Rebuilt for https://fedoraproject.org/wiki/Fedora_11_Mass_Rebuild
kdelibs-4.2.2-7.fc11
--------------------
* Sun Apr 19 2009 Rex Dieter <rdieter(a)fedoraproject.org> 4.2.2-7
- fix and simplify the child struct disposal (kde#180785)
* Sat Apr 18 2009 Rex Dieter <rdieter(a)fedoraproject.org> 4.2.2-6
- squash leaky file descriptors in kdeinit (kde#180785,rhbz#484370)
kdelibs3-3.5.10-11.fc11
-----------------------
* Sun Apr 19 2009 Rex Dieter <rdieter(a)fedoraproject.org> - 3.5.10-11
- update openssl patch (for 0.9.8k)
kernel-2.6.29.1-102.fc11
------------------------
* Mon Apr 20 2009 Kyle McMartin <kyle(a)redhat.com> 2.6.29.1-102
- git-bluetooth2.patch: Bluetooth fixes from 2.6.30-rc2.
* Sun Apr 19 2009 Mark McLoughlin <markmc(a)redhat.com> - 2.6.29.1-101
- Fix xen boot on machines without NX support (#492523)
ldm-2.0.33-3.fc11
-----------------
* Sun Apr 19 2009 Ville Skyttä <ville.skytta at iki.fi> - 2.0.33-3
- Build using $RPM_OPT_FLAGS.
- Build with dependency tracking disabled for possible speedup and cleaner logs.
libnet10-1.0.2a-17.fc11
-----------------------
* Sat Apr 18 2009 Robert Scheck <robert(a)fedoraproject.org> 1.0.2a-17
- Enabled a shared library and made lots of spec file cleanups
moe-1.0-5.fc11
--------------
* Sun Apr 19 2009 Debarshi Ray <rishi(a)fedoraproject.org> 1.0-5
- Fixed configure to respect the environment's CFLAGS and CXXFLAGS settings.
nautilus-sendto-1.1.4.1-1.fc11
------------------------------
* Mon Apr 20 2009 - Bastien Nocera <bnocera(a)redhat.com> - 1.1.4.1-1
- Update to 1.1.4.1
ntp-4.2.4p6-4.fc11
------------------
* Mon Apr 20 2009 Miroslav Lichvar <mlichvar(a)redhat.com> 4.2.4p6-4
- don't restart ntpd in dhclient script with every renewal
- fix buffer overflow in ntpq (#490617)
- check status in condrestart (#481261)
- don't crash when compiled with HAVE_TIMER_CREATE (#486217)
osmo-0.2.4-6.fc11
-----------------
* Sun Apr 19 2009 Debarshi Ray <rishi(a)fedoraproject.org> - 0.2.4-6
- Fixed configure to ensure correct usage of CFLAGS and CPPFLAGS, and respect
the environment's settings.
pygame-1.8.1-6.fc11
-------------------
qemu-0.10-11.fc11
-----------------
* Mon Apr 20 2009 Mark McLoughlin <markmc(a)redhat.com> - 2:0.10-11
- Fix qcow2 image corruption (#496642)
* Sun Apr 19 2009 Mark McLoughlin <markmc(a)redhat.com> - 2:0.10-9
- Align VGA ROM to 4k boundary - fixes 'qemu-kvm -std vga' (#494376)
* Sun Apr 19 2009 Mark McLoughlin <markmc(a)redhat.com> - 2:0.10-10
- Run sysconfig.modules from %post on x86_64 too (#494739)
qgis-1.0.1-2.fc11
-----------------
* Wed Feb 25 2009 Fedora Release Engineering <rel-eng(a)lists.fedoraproject.org> - 1.0.1-2
- Rebuilt for https://fedoraproject.org/wiki/Fedora_11_Mass_Rebuild
sabayon-2.25.0-3.fc11
---------------------
* Mon Apr 20 2009 Tomas Bzatek <tbzatek(a)redhat.com> - 2.25.0-3
- Another, more complete fix for panel gconf save issues (gnome #542604)
selinux-policy-3.6.12-9.fc11
----------------------------
* Mon Apr 20 2009 Dan Walsh <dwalsh(a)redhat.com> 3.6.12-9
- Add ability to run postdrop from confined users
* Sat Apr 18 2009 Dan Walsh <dwalsh(a)redhat.com> 3.6.12-8
- Fixes for podsleuth
* Fri Apr 17 2009 Dan Walsh <dwalsh(a)redhat.com> 3.6.12-6
- Allow cupsd_t to create link files in print_spool_t
- Fix iscsi_stream_connect typo
- Fix labeling on /etc/acpi/actions
- Don't reinstall unconfine and unconfineuser on upgrade if they are not installed
* Fri Apr 17 2009 Dan Walsh <dwalsh(a)redhat.com> 3.6.12-7
- Turn off nsplugin transition
- Remove Konsole leaked file descriptors for release
* Tue Apr 14 2009 Dan Walsh <dwalsh(a)redhat.com> 3.6.12-5
- Allow audioentroy to read etc files
sssd-0.3.2-2.fc11
-----------------
* Mon Apr 20 2009 Jakub Hrozek <jhrozek(a)redhat.com> - 0.3.2-1
- bugfix release 0.3.2
- includes previous release patches
- change permissions of the /etc/sssd/sssd.conf to 0600
* Mon Apr 20 2009 Simo Sorce <ssorce(a)redhat.com> - 0.3.2-2
- release out of the official 0.3.2 tarball
system-config-date-1.9.38-1.fc11
--------------------------------
* Mon Apr 20 2009 Nils Philippsen <nils(a)redhat.com> - 1.9.38-1
- restore SELinux context of /etc/localtime (#490323, patch by Daniel Walsh)
wesnoth-1.6.1-1.fc11
--------------------
* Mon Apr 20 2009 Jon Ciesla <limb(a)jcomserv.net> - 1.6.1-1
- 1.6.1 maintenance release.
Summary:
Added Packages: 0
Removed Packages: 2
Modified Packages: 24
Broken deps for i386
----------------------------------------------------------
gnome-desktop-sharp-2.26.0-1.fc11.i586 requires libnautilus-burn.so.4
gnome-python2-nautilus-cd-burner-2.26.0-1.fc11.i586 requires libnautilus-burn.so.4
sems-ivr-1.1.0-5.fc10.i386 requires libpython2.5.so.1.0
sems-python-1.1.0-5.fc10.i386 requires libpython2.5.so.1.0
sems-xmlrpc2di-1.1.0-5.fc10.i386 requires libssl.so.7
Broken deps for x86_64
----------------------------------------------------------
gnome-desktop-sharp-2.26.0-1.fc11.x86_64 requires libnautilus-burn.so.4()(64bit)
gnome-python2-nautilus-cd-burner-2.26.0-1.fc11.x86_64 requires libnautilus-burn.so.4()(64bit)
sems-ivr-1.1.0-5.fc10.x86_64 requires libpython2.5.so.1.0()(64bit)
sems-python-1.1.0-5.fc10.x86_64 requires libpython2.5.so.1.0()(64bit)
sems-xmlrpc2di-1.1.0-5.fc10.x86_64 requires libssl.so.7()(64bit)
Broken deps for ppc
----------------------------------------------------------
fedora-ksplice-0.5-3.fc11.noarch requires ksplice
gnome-desktop-sharp-2.26.0-1.fc11.ppc requires libnautilus-burn.so.4
gnome-python2-nautilus-cd-burner-2.26.0-1.fc11.ppc requires libnautilus-burn.so.4
sems-ivr-1.1.0-5.fc10.ppc requires libpython2.5.so.1.0
sems-python-1.1.0-5.fc10.ppc requires libpython2.5.so.1.0
sems-xmlrpc2di-1.1.0-5.fc10.ppc requires libssl.so.7
Broken deps for ppc64
----------------------------------------------------------
cabal2spec-0.12-1.fc11.noarch requires ghc > 0:6.10.1-7
fedora-ksplice-0.5-3.fc11.noarch requires ksplice
gnome-python2-nautilus-cd-burner-2.26.0-1.fc11.ppc64 requires libnautilus-burn.so.4()(64bit)
sems-ivr-1.1.0-5.fc10.ppc64 requires libpython2.5.so.1.0()(64bit)
sems-python-1.1.0-5.fc10.ppc64 requires libpython2.5.so.1.0()(64bit)
sems-xmlrpc2di-1.1.0-5.fc10.ppc64 requires libssl.so.7()(64bit)
15 years, 1 month
Re: Can we have a Delta-DVD release
by Andre Robatino
> So there is! Doesn't work on F9 -> F10 but on F10 -> F11 beta DVD
(x86_64)
> I get a delta iso image of 2.2GB, about 53% less than the full iso.
> Hmm, not as good as I was expecting, probably not worth making a fuss
> about unless we could get <1GB.
A lot of people would probably be very happy with cutting the download
in half. On a 768 Kbit/s connection it would reduce download time from
about 12 hours to six. If the tool doesn't take too long to do the
reconstruction after downloading, and if it's reliable (so one doesn't
have to download 2GB only to find out it doesn't work, then download the
full 4GB anyway) the deltaiso should be available as a download option.
15 years, 1 month
Re: Can we have a Delta-DVD release
by James Gallagher
--- On Tue, 21/4/09, Jonathan Dieter <jdieter(a)gmail.com> wrote:
> From: Jonathan Dieter <jdieter(a)gmail.com>
> Subject: Re: Can we have a Delta-DVD release
> To: "Development discussions related to Fedora" <fedora-devel-list(a)redhat.com>
> Date: Tuesday, 21 April, 2009, 7:56 AM
> On Mon, 2009-04-20 at 23:40 -0400,
> Warren Togami wrote:
> > On 04/20/2009 10:15 PM, James Gallagher wrote:
> > >
> > > After all, I think some of the large language
> packages would have a
> > > pretty small delta between 6 month release
> cycles.
> > >
> >
> > Note that nearly every package is rebuilt between
> releases.
> >
> > For this to make any difference at all, it would have
> to delta not only
> > the bits of the RPM packages, but delta the
> uncompressed RPMS against
> > each other.
> >
> > Does any tool do this today across an entire ISO
> image?
>
> There is makedeltaiso in the deltarpm package. I
> haven't tested it much
> as I don't use the DVD iso images to install.
>
> Jonathan
>
So there is! Doesn't work on F9 -> F10 but on F10 -> F11 beta DVD (x86_64)
I get a delta iso image of 2.2GB, about 53% less than the full iso. Hmm, not as good as I was expecting, probably not worth making a fuss about unless we could get <1GB.
Are there any plans to optimise this tool or use it for future releases?
James
15 years, 1 month
Rawhide: No module named gtk
by Gerry Reno
I installed Rawhide today and when I went to launch
system-config-network it's saying "Import Error: No module named gtk".
Is there a path problem? I did a 'yum whatprovides gtk' but it returned
No packages. I've rarely installed Rawhide, so maybe I'm missing
something here.
????
Regards,
Gerry
15 years, 1 month
Proposal: Single GPG key per Fedora release (starting with 11)
by Jesse Keating
As I mentioned in an earlier thread I was interested in reducing the
number of gpg keys down to one per release. Currently we have two, one
we sign development builds with during beta/preview and updates-testing,
and then one we sign the released packages with and the stable updates
with. Multiple keys per release creates a lot of churn, reduces the
number of hardlinks we can maintain, and causes a lot of delay in
getting package sets prepped for the different releases. As such I'm
proposing that we reduce the keys down to one per release, used for all
the scenarios listed, starting with Fedora 11. There is already a
Fedora 11 key that was used to sign beta and will be used to sign
preview release, I would just revoke / delete the current ID which
mentions testing and replace it with an ID of just "Fedora 11".
fedora-release will be modified to handle this in the repo files as
well.
If there are no strong reasonable objections this will happen early this
week in time for the Preview release.
--
Jesse Keating
Fedora -- Freedom² is a feature!
identi.ca: http://identi.ca/jkeating
15 years, 1 month
Package Review Stats for the week ending April 19th, 2009
by Brian Pepple
Top three FAS account holders who have completed reviewing "Package
review" components on bugzilla for the week ending April 19th, 2009
were Parag AN(पराग), Chris Weyl, and Jussi Lehtola. Below is the
number of package reviews completed.
Parag AN(पराग) - 8
Chris Weyl - 7
Jussi Lehtola - 7
Orcan 'oget' Ogetbil - 5
Iain Arnell - 3
Michael Schwendt - 3
Robert Scheck - 3
Christoph Wickert - 2
Kalev Lember - 2
Mamoru Tasaka - 2
Marek Mahut - 2
Mattias Ellert - 2
Till Maas - 2
Dan Horák - 1
Daniel Berrange - 1
Gianluca Sforna - 1
Hans de Goede - 1
Jan Klepek - 1
Jason Tibbitts - 1
Lillian Angel - 1
Lubomir Rintel - 1
Matthias Saou - 1
Michal Nowak - 1
Peter Lemenkov - 1
Rex Dieter - 1
Simon Wesp - 1
Thomas Sailer - 1
Tom "spot" Callaway - 1
Review Requests: 60
Merge Reviews: 2
Total reviews modified: 63
Thanks,
/B
--
Brian Pepple <bpepple(a)fedoraproject.org>
https://fedoraproject.org/wiki/User:Bpepple
gpg --keyserver pgp.mit.edu --recv-keys 810CC15E
BD5E 6F9E 8688 E668 8F5B CBDE 326A E936 810C C15E
15 years, 1 month
pkgconfig dep problems in rawhide 20090421
by Michael Schwendt
Non-devel packages with problematic pkgconfig auto-dependencies
on -devel packages:
gtk-sharp2 (bz 477308)
Used to be assigned to avahi-ui-sharp - this needs confirmation
whether Mono requires .pc files to be present at run-time. If true,
avahi-ui-sharp => gtk-sharp2-devel pulls it several -devel packages,
and the .pc file could be moved into gtk-sharp2.
deskbar-applet (bz 477309)
libXaw (bz 489172)
gupnp-vala (bz 496790) NEW
15 years, 1 month
gnome keyring daemon from cron job
by Christoph Höger
Hi,
so I debugged my crontab problem and came down to the following issue:
invoking something like:
import gnomekeyring as gkey
if __name__ == "__main__":
gkey.get_default_keyring_sync()
print "got keyring"
from a normal shell works perfectly.
Invoked from a cron job (with DISPLAY=:0.0 set) returns a NoKeyring
Error.
This used to work under f9/10, but stopped working in rawhide.
Is this a bug or a feature?
thanks
christoph
15 years, 1 month