[gcc-python-plugin] docs: move section on callbacks into its own page
by David Hugh Malcolm
commit e680db63f1e4286bc488bee38a3aa1381a0b05a9
Author: David Malcolm <dmalcolm(a)redhat.com>
Date: Tue Jan 10 19:07:45 2012 -0500
docs: move section on callbacks into its own page
docs/basics.rst | 189 ++--------------------------------------------
docs/callbacks.rst | 212 ++++++++++++++++++++++++++++++++++++++++++++++++++++
docs/index.rst | 1 +
3 files changed, 219 insertions(+), 183 deletions(-)
---
diff --git a/docs/basics.rst b/docs/basics.rst
index 1ca3116..394dbc6 100644
--- a/docs/basics.rst
+++ b/docs/basics.rst
@@ -1,5 +1,5 @@
-.. Copyright 2011 David Malcolm <dmalcolm(a)redhat.com>
- Copyright 2011 Red Hat, Inc.
+.. Copyright 2011, 2012 David Malcolm <dmalcolm(a)redhat.com>
+ Copyright 2011, 2012 Red Hat, Inc.
This is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by
@@ -294,189 +294,12 @@ The other way to write scripts is to register callback functions
to be called when various events happen during compilation, such as using
:py:data:`gcc.PLUGIN_PASS_EXECUTION` to piggyback off of an existing GCC pass.
-.. py:function:: gcc.register_callback(event_id, function, [extraargs,] **kwargs)
-
- Wire up a python function as a callback. It will be called when the given
- event occurs during compilation. For some events, the callback will be
- called just once; for other events, the callback is called once per
- function within the source code being compiled. In the latter case, the
- plugin passes a :py:class:`gcc.Function` instance as a parameter to your
- callback, so that you can work on it::
-
- def my_pass_execution_callback(*args, **kwargs):
- print('my_pass_execution_callback was called: args=%r kwargs=%r'
- % (args, kwargs))
-
- import gcc
- gcc.register_callback(gcc.PLUGIN_PASS_EXECUTION,
- my_pass_execution_callback)
-
- You can pass additional arguments when registering the callback - they will
- be passed to the callback after any normal arguments. This is denoted in the
- descriptions of events below by `*extraargs`.
-
- You can also supply keyword arguments: they will be passed on as keyword
- arguments to the callback. This is denoted in the description of events
- below by `**kwargs`.
-
-The various events are exposed as constants within the `gcc` module and
-directly wrap GCC's plugin mechanism. The exact arguments you get aren't
-well-documented there, and may be subject to change. I've tried to document
-what I've seen in GCC 4.6 here, but it's worth experimenting and printing args
-and kwargs as shown above.
-
-Currently useful callback events
---------------------------------
-
-.. py:data:: gcc.PLUGIN_PASS_EXECUTION
-
- Called when GCC is about to run one of its passes.
-
- Arguments passed to the callback are:
-
- (`ps`, `fun`, `*extraargs`, `**kwargs`)
-
- where `ps` is a :py:class:`gcc.Pass` and `fun` is a :py:class:`gcc.Function`.
- Your callback will typically be called many times: there are many passes,
- and each can be invoked zero or more times per function (in the code being
- compiled)
-
- More precisely, some passes have a "gate check": the pass first checks a
- condition, and only executes if the condition is true.
-
- Any callback registered with `gcc.PLUGIN_PASS_EXECUTION` will get called
- if this condition succeeds.
-
- The actual work of the pass is done after the callbacks return.
-
- In pseudocode::
-
- if pass.has_gate_condition:
- if !pass.test_gate_condition():
- return
- invoke_all_callbacks()
- actually_do_the_pass()
-
- For passes working on individual functions, all of the above is done
- per-function.
-
- To connect to a specific pass, you can simply add a conditional based on the
- name of the pass::
-
- def my_callback(ps, fun):
- if ps.name != '*warn_function_return':
- # Not the pass we want
- return
- # Do something here
- print(fun.decl.name)
-
- gcc.register_callback(gcc.PLUGIN_PASS_EXECUTION,
- my_callback)
-
-
-.. py:data:: gcc.PLUGIN_PRE_GENERICIZE
-
- Arguments passed to the callback are:
-
- (`fndecl`, `*extraargs`, `**kwargs`)
-
- where `fndecl` is a :py:class:`gcc.Tree` representing a function declaration
- within the source code being compiled.
-
-.. py:data:: gcc.PLUGIN_FINISH_UNIT
-
- Called when GCC has finished compiling a particular translation unit.
-
- Arguments passed to the callback are:
-
- (`*extraargs`, `**kwargs`)
-
-.. Other callback events
- ---------------------
-
-.. (Commented out for now; probably should finish this and move it to a
- reference section)
-
-.. The following may need work before they're meaningfully usable from Python
- scripts:
-
- .. py:data:: gcc.PLUGIN_ATTRIBUTES
-
- Called from: init_attributes () at ../../gcc/attribs.c:187
- However, it seems at this point to have initialized these::
-
- static const struct attribute_spec *attribute_tables[4];
- static htab_t attribute_hash;
-
- .. py:data:: gcc.PLUGIN_PRAGMAS
-
- gcc_data=0x0
- Called from: c_common_init () at ../../gcc/c-family/c-opts.c:1052
-
- .. py:data:: gcc.PLUGIN_START_UNIT
-
- gcc_data=0x0
- Called from: compile_file () at ../../gcc/toplev.c:573
-
- .. py:data:: gcc.PLUGIN_PRE_GENERICIZE
-
- gcc_data is: tree fndecl;
- Called from: finish_function () at ../../gcc/c-decl.c:8323
-
- .. py:data:: gcc.PLUGIN_OVERRIDE_GATE
-
- gcc_data::
-
- &gate_status
- bool gate_status;
-
- Called from : execute_one_pass (pass=0x1011340) at ../../gcc/passes.c:1520
-
- .. py:data:: gcc.PLUGIN_ALL_IPA_PASSES_START
-
- gcc_data=0x0
- Called from: ipa_passes () at ../../gcc/cgraphunit.c:1779
-
- .. py:data:: gcc.PLUGIN_EARLY_GIMPLE_PASSES_START
-
- gcc_data=0x0
- Called from: execute_ipa_pass_list (pass=0x1011fa0) at ../../gcc/passes.c:1927
-
- .. py:data:: gcc.PLUGIN_EARLY_GIMPLE_PASSES_END
-
- gcc_data=0x0
- Called from: execute_ipa_pass_list (pass=0x1011fa0) at ../../gcc/passes.c:1930
-
- .. py:data:: gcc.PLUGIN_ALL_IPA_PASSES_END
-
- gcc_data=0x0
- Called from: ipa_passes () at ../../gcc/cgraphunit.c:1821
-
- .. py:data:: gcc.PLUGIN_ALL_PASSES_START
-
- gcc_data=0x0
- Called from: tree_rest_of_compilation (fndecl=0x7ffff16b1f00) at ../../gcc/tree-optimize.c:420
-
- .. py:data:: gcc.PLUGIN_ALL_PASSES_END
-
- gcc_data=0x0
- Called from: tree_rest_of_compilation (fndecl=0x7ffff16b1f00) at ../../gcc/tree-optimize.c:425
-
- .. py:data:: gcc.PLUGIN_FINISH
-
- gcc_data=0x0
- Called from: toplev_main (argc=17, argv=0x7fffffffdfc8) at ../../gcc/toplev.c:1970
-
- .. py:data:: gcc.PLUGIN_FINISH_TYPE
-
- gcc_data=tree
- Called from c_parser_declspecs (parser=0x7fffef559730, specs=0x15296d0, scspec_ok=1 '\001', typespec_ok=1 '\001', start_attr_ok=<optimized out>, la=cla_nonabstract_decl) at ../../gcc/c-parser.c:2111
+ .. literalinclude:: ../examples/show-passes.py
+ :lines: 19-
+ :language: python
- .. py:data:: gcc.PLUGIN_PRAGMA
+For more information, see :ref:`callbacks`
- gcc_data=0x0
- Called from: init_pragma at ../../gcc/c-family/c-pragma.c:1321
- to "Allow plugins to register their own pragmas."
Generating custom errors and warnings
=====================================
diff --git a/docs/callbacks.rst b/docs/callbacks.rst
new file mode 100644
index 0000000..21bc3bd
--- /dev/null
+++ b/docs/callbacks.rst
@@ -0,0 +1,212 @@
+.. Copyright 2012 David Malcolm <dmalcolm(a)redhat.com>
+ Copyright 2012 Red Hat, Inc.
+
+ This is free software: you can redistribute it and/or modify it
+ under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see
+ <http://www.gnu.org/licenses/>.
+
+.. _callbacks:
+
+Working with callbacks
+======================
+
+One way to work with GCC from the Python plugin is via callbacks. It's possible
+to register callback functions, which will be called when various events happen
+during compilation.
+
+For example, it's possible to piggyback off of an existing GCC pass by using
+:py:data:`gcc.PLUGIN_PASS_EXECUTION` to piggyback off of an existing GCC pass.
+
+.. py:function:: gcc.register_callback(event_id, function, [extraargs,] **kwargs)
+
+ Wire up a python function as a callback. It will be called when the given
+ event occurs during compilation. For some events, the callback will be
+ called just once; for other events, the callback is called once per
+ function within the source code being compiled. In the latter case, the
+ plugin passes a :py:class:`gcc.Function` instance as a parameter to your
+ callback, so that you can work on it::
+
+ def my_pass_execution_callback(*args, **kwargs):
+ print('my_pass_execution_callback was called: args=%r kwargs=%r'
+ % (args, kwargs))
+
+ import gcc
+ gcc.register_callback(gcc.PLUGIN_PASS_EXECUTION,
+ my_pass_execution_callback)
+
+ You can pass additional arguments when registering the callback - they will
+ be passed to the callback after any normal arguments. This is denoted in the
+ descriptions of events below by `*extraargs`.
+
+ You can also supply keyword arguments: they will be passed on as keyword
+ arguments to the callback. This is denoted in the description of events
+ below by `**kwargs`.
+
+The various events are exposed as constants within the `gcc` module and
+directly wrap GCC's plugin mechanism. The exact arguments you get aren't
+well-documented there, and may be subject to change. I've tried to document
+what I've seen in GCC 4.6 here, but it's worth experimenting and printing args
+and kwargs as shown above.
+
+Callback events
+---------------
+
+.. py:data:: gcc.PLUGIN_PASS_EXECUTION
+
+ Called when GCC is about to run one of its passes.
+
+ Arguments passed to the callback are:
+
+ (`ps`, `fun`, `*extraargs`, `**kwargs`)
+
+ where `ps` is a :py:class:`gcc.Pass` and `fun` is a :py:class:`gcc.Function`.
+ Your callback will typically be called many times: there are many passes,
+ and each can be invoked zero or more times per function (in the code being
+ compiled)
+
+ More precisely, some passes have a "gate check": the pass first checks a
+ condition, and only executes if the condition is true.
+
+ Any callback registered with `gcc.PLUGIN_PASS_EXECUTION` will get called
+ if this condition succeeds.
+
+ The actual work of the pass is done after the callbacks return.
+
+ In pseudocode::
+
+ if pass.has_gate_condition:
+ if !pass.test_gate_condition():
+ return
+ invoke_all_callbacks()
+ actually_do_the_pass()
+
+ For passes working on individual functions, all of the above is done
+ per-function.
+
+ To connect to a specific pass, you can simply add a conditional based on the
+ name of the pass::
+
+ def my_callback(ps, fun):
+ if ps.name != '*warn_function_return':
+ # Not the pass we want
+ return
+ # Do something here
+ print(fun.decl.name)
+
+ gcc.register_callback(gcc.PLUGIN_PASS_EXECUTION,
+ my_callback)
+
+
+.. py:data:: gcc.PLUGIN_PRE_GENERICIZE
+
+ Arguments passed to the callback are:
+
+ (`fndecl`, `*extraargs`, `**kwargs`)
+
+ where `fndecl` is a :py:class:`gcc.Tree` representing a function declaration
+ within the source code being compiled.
+
+.. py:data:: gcc.PLUGIN_FINISH_UNIT
+
+ Called when GCC has finished compiling a particular translation unit.
+
+ Arguments passed to the callback are:
+
+ (`*extraargs`, `**kwargs`)
+
+.. Other callback events
+ ---------------------
+
+.. (Commented out for now; probably should finish this and move it to a
+ reference section)
+
+.. The following may need work before they're meaningfully usable from Python
+ scripts:
+
+ .. py:data:: gcc.PLUGIN_ATTRIBUTES
+
+ Called from: init_attributes () at ../../gcc/attribs.c:187
+ However, it seems at this point to have initialized these::
+
+ static const struct attribute_spec *attribute_tables[4];
+ static htab_t attribute_hash;
+
+ .. py:data:: gcc.PLUGIN_PRAGMAS
+
+ gcc_data=0x0
+ Called from: c_common_init () at ../../gcc/c-family/c-opts.c:1052
+
+ .. py:data:: gcc.PLUGIN_START_UNIT
+
+ gcc_data=0x0
+ Called from: compile_file () at ../../gcc/toplev.c:573
+
+ .. py:data:: gcc.PLUGIN_PRE_GENERICIZE
+
+ gcc_data is: tree fndecl;
+ Called from: finish_function () at ../../gcc/c-decl.c:8323
+
+ .. py:data:: gcc.PLUGIN_OVERRIDE_GATE
+
+ gcc_data::
+
+ &gate_status
+ bool gate_status;
+
+ Called from : execute_one_pass (pass=0x1011340) at ../../gcc/passes.c:1520
+
+ .. py:data:: gcc.PLUGIN_ALL_IPA_PASSES_START
+
+ gcc_data=0x0
+ Called from: ipa_passes () at ../../gcc/cgraphunit.c:1779
+
+ .. py:data:: gcc.PLUGIN_EARLY_GIMPLE_PASSES_START
+
+ gcc_data=0x0
+ Called from: execute_ipa_pass_list (pass=0x1011fa0) at ../../gcc/passes.c:1927
+
+ .. py:data:: gcc.PLUGIN_EARLY_GIMPLE_PASSES_END
+
+ gcc_data=0x0
+ Called from: execute_ipa_pass_list (pass=0x1011fa0) at ../../gcc/passes.c:1930
+
+ .. py:data:: gcc.PLUGIN_ALL_IPA_PASSES_END
+
+ gcc_data=0x0
+ Called from: ipa_passes () at ../../gcc/cgraphunit.c:1821
+
+ .. py:data:: gcc.PLUGIN_ALL_PASSES_START
+
+ gcc_data=0x0
+ Called from: tree_rest_of_compilation (fndecl=0x7ffff16b1f00) at ../../gcc/tree-optimize.c:420
+
+ .. py:data:: gcc.PLUGIN_ALL_PASSES_END
+
+ gcc_data=0x0
+ Called from: tree_rest_of_compilation (fndecl=0x7ffff16b1f00) at ../../gcc/tree-optimize.c:425
+
+ .. py:data:: gcc.PLUGIN_FINISH
+
+ gcc_data=0x0
+ Called from: toplev_main (argc=17, argv=0x7fffffffdfc8) at ../../gcc/toplev.c:1970
+
+ .. py:data:: gcc.PLUGIN_FINISH_TYPE
+
+ gcc_data=tree
+ Called from c_parser_declspecs (parser=0x7fffef559730, specs=0x15296d0, scspec_ok=1 '\001', typespec_ok=1 '\001', start_attr_ok=<optimized out>, la=cla_nonabstract_decl) at ../../gcc/c-parser.c:2111
+
+ .. py:data:: gcc.PLUGIN_PRAGMA
+
+ gcc_data=0x0
+ Called from: init_pragma at ../../gcc/c-family/c-pragma.c:1321
+ to "Allow plugins to register their own pragmas."
diff --git a/docs/index.rst b/docs/index.rst
index ad34079..b12fc3b 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -36,6 +36,7 @@ Contents:
tree.rst
gimple.rst
passes.rst
+ callbacks.rst
attributes.rst
cpychecker.rst
success.rst
12 years, 4 months
[gcc-python-plugin] add notes to self on making a release
by David Hugh Malcolm
commit 06152cc6c51c7a443b2c7d7dfc8dbf53aee72df4
Author: David Malcolm <dmalcolm(a)redhat.com>
Date: Tue Jan 10 15:02:50 2012 -0500
add notes to self on making a release
Makefile | 62 +++++++++++++++++++++++++++++++++++++++++++++++++++++++-------
1 files changed, 55 insertions(+), 7 deletions(-)
---
diff --git a/Makefile b/Makefile
index dce2b0d..6a8d54d 100644
--- a/Makefile
+++ b/Makefile
@@ -202,16 +202,64 @@ docs/passes.svg: plugin generate-passes-svg.py
# Utility target, to help me to make releases
# - creates a tag in git, and pushes it
# - creates a tarball
-# - publishes it below:
-# https://fedorahosted.org/releases/g/c/gcc-python-plugin/
#
# The following assumes that VERSION has been set e.g.
-# $ make release VERSION=0.4
-#
-# git push --tags
-# scp gcc-python-plugin-$(VERSION).tar.gz dmalcolm@fedorahosted.org:gcc-python-plugin
+# $ make tarball VERSION=0.4
-release:
+tarball:
+ -git tag -d v$(VERSION)
git tag -a v$(VERSION) -m"$(VERSION)"
git archive --format=tar --prefix=gcc-python-plugin-$(VERSION)/ v$(VERSION) | gzip > gcc-python-plugin-$(VERSION).tar.gz
+ sha256sum gcc-python-plugin-$(VERSION).tar.gz
cp gcc-python-plugin-$(VERSION).tar.gz ~/rpmbuild/SOURCES/
+
+# Notes to self on making a release
+# ---------------------------------
+#
+# Before tagging:
+#
+# * update the version/release in docs/conf.py
+#
+# * update the version in gcc-python-plugin.spec
+#
+# * add release notes to docs
+#
+# Test the candidate tarball via a scratch SRPM build in Koji (this
+# achieves test coverage against python 2 and 3, on both i686 and x86_64)
+#
+# $ make koji VERSION=fixme
+#
+# After successful testing of a candidate tarball:
+#
+# * push the tag:
+#
+# $ git push --tags
+#
+# * upload it to https://fedorahosted.org/releases/g/c/gcc-python-plugin/
+# via:
+#
+# $ scp gcc-python-plugin-$(VERSION).tar.gz dmalcolm@fedorahosted.org:gcc-python-plugin
+#
+# * add version to Trac: https://fedorahosted.org/gcc-python-plugin/admin/ticket/versions
+#
+# * update release info at https://fedorahosted.org/gcc-python-plugin/wiki#Code
+#
+# * send release announcement:
+#
+# To: gcc(a)gcc.gnu.org, gcc-python-plugin(a)lists.fedorahosted.org, python-announce-list(a)python.org
+# Subject: ANN: gcc-python-plugin $(VERSION)
+# (etc)
+
+# Utility target, for building test rpms:
+srpm:
+ rpmbuild -bs gcc-python-plugin.spec
+
+# Perform a test rpm build locally:
+rpm:
+ rpmbuild -ba gcc-python-plugin.spec
+
+# Perform a test (scratch) build in Koji:
+# f16: gcc 4.6
+# f17: gcc 4.7
+koji: srpm
+ koji build --scratch f16 ~/rpmbuild/SRPMS/gcc-python-plugin-$(VERSION)-1.fc15.src.rpm
12 years, 4 months
[gcc-python-plugin] Created tag v0.8
by David Hugh Malcolm
The unsigned tag 'v0.8' was created.
Tagger: David Malcolm <dmalcolm(a)redhat.com>
Date: Tue Jan 10 14:42:59 2012 -0500
0.8
Changes since the last tag 'v0.7':
David Malcolm (96):
add release notes for 0.7
cpychecker: Py_BuildValue("O") can accept a ptr to any PyObject subclass
run-test-suite.py: support test cases for source languages other than C
docs: add more cross-references
C++ support
docs: minor fixes
cpychecker: fix handling of "static PyObject*" locals
cpychecker: implement PyErr_SetNone
docs: add LibreOffice example to Success Stories
docs: add more cross-references
docs: add pycups to success stories
docs: add notes on using gcc-with-cpychecker from distutils
docs: add python-krbV to success stories
implement verification of PyMethodDef tables
remove redundant test
cpychecker: implement PyArg_Parse
cpychecker: implement Py_AtExit
cpychecker: implement PyErr_WarnEx
cpychecker: implement PyLong_FromLongLong
cpychecker: implement PyModule_GetDict
cpychecker: implement PyObject_AsFileDescriptor
cpychecker: start adding explanations to NULL function argument warnings
cpychecker: introduce CPython API metadata
cpychecker: add metadata for the rest of the CPython API implementation
cpychecker: use fnmeta.name
cpychecker: use FnMeta when generating descriptive text
cpychecker: implement PyList_Size and improve field lookups
cpychecker: use State.read_field_by_name() in more places
cpychecker: add metadata for PyList_New
cpychecker: initial implementation of PyList_GetItem
cpychecker: implement PyEval_SaveThread and PyEval_RestoreThread
cpychecker: implement PyCObject_FromVoidPtr and PyTuple_Pack
cpychecker: implement PyCallable_Check and PyCObject_AsVoidPtr
cpychecker: implement PyString_Size
cpychecker: don't report reference leaks for traces that terminate the process
cpychecker: add shadows to descriptive text within HTML reports
cpychecker: introduce warn_about_NULL_without_exception()
cpychecker: tp_iternext callbacks are allowed to return NULL without setting an exception
cpychecker,docs: document e75656f113c7baf7a1d9b15f9c5f3455c6fcd93d
cpychecker: fix PyList_GetItem
cpychecker: implement PyObject_GenericGetAttr and PyObject_GenericSetAttr
cpychecker: implement PyUnicode_AsUTF8String
cpychecker: implement PyErr_NewException
cpychecker: implement AbstractValue.is_null_ptr
cpychecker: implement PyList_Append's behaviors with NULL arguments
cpychecker: implement PyModule_AddObject's behaviors with NULL arguments
cpychecker: clean up selftest for tp_iternext handling
cpychecker: fix traceback in tp_iternext handling
cpychecker: update explanation-making functions to take a FnMeta
cpychecker: use FnMeta in more places
cpychecker: handle PyArg_Parse code "O&" in reference-count checker
cpychecker: improve behavior on functions that are too complicated to fully handle
cpychecker: tone down the transition limit
cpychecker: optimize Trace.get_all_var_region_pairs()
cpychecker: add (disabled) code to profile the refcount checker
cpychecker: add bulletproofing to refcount modifications
cpychecker: implement PyObject_CallFunctionObjArgs and PyObject_CallMethodObjArgs
cpychecker: fix casting of pointer to a non-pointer value
cpychecker: implement PyObject_CallFunction
cpychecker: implement PyEval_CallObjectWithKeywords
cpychecker: implement PyCapsule_GetPointer
cpychecker: implement PyMapping_Size
support emitting unconditional warnings
move diagnostics code from gcc-python.c into a new gcc-python-diagnostics.c
move callback-handling code from gcc-python.c into a new gcc-python-callbacks.c file
fix usage of gcc.warning in test code
add support for metadata.ini files to the test suite
cpychecker: convert format string checker from permerror to warning
cpychecker: issue warnings rather than errors
docs: add PyObject_Call{Function,Method} to list of functions with format-string checking
docs: add note about checking of PyObject_Call{Function,Method}ObjArgs
exclude 6 more tests known to fail on 32-bit
cpychecker: protect against division-by-zero python tracebacks (ticket #25)
cpychecker: fix Python traceback (ticket #20)
docs: gcc.ComponentReference -> gcc.ComponentRef (ticket #9)
add "constant" and __repr__ to gcc.RealCst
cpychecker: support gcc.RealCst instances (ticket #23)
cpychecker: fix python traceback in analysis of PyTuple_Size where size is unknown (ticket #24)
cpychecker: fix python traceback analyzing PyList_Append(list, UnknownValue() ) (ticket #22)
cpychecker: fix python traceback analyzing PyList_Append() (ticket #19)
cpychecker: fix python traceback with negative shifts (ticket #14)
docs: "CaseLabelExpr" -> "gcc.CaseLabelExpr"
docs: revamp the gimple documentation
docs: clarify gcc.PointerType et al (ticket #27)
docs: document looking up a C struct by name (ticket #26)
docs: add a table of important gcc.Tree subclasses
add get_symbol() classmethod to various tree subclasses
hide references to gcc.VarDecl in output (ticket #12)
fix python 3 incompatibility in gcc_RealCst_get_constant
fix python 3 incompatibility in tests/plugin/expressions/get_symbol
python 3: fix up PyString_Type to PyBytes_Type in PyUnicode_AsUTF8String
python 3: skip various tests that don't work on Python 3
selftest fixes for 32-bit builds
pre-release version bumps to 0.8
docs: add 0.8 release notes
add David Narvaez to contributors.rst
David Narvaez (1):
run-test-suite.py: type IDs are hexadecimal
Tom Tromey (1):
redeclare decl_as_string as weak
12 years, 4 months
[gcc-python-plugin] add David Narvaez to contributors.rst
by David Hugh Malcolm
commit f6376da61857cab6f1c001c3b8a0be1508c967ce
Author: David Malcolm <dmalcolm(a)redhat.com>
Date: Tue Jan 10 14:42:43 2012 -0500
add David Narvaez to contributors.rst
contributors.rst | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)
---
diff --git a/contributors.rst b/contributors.rst
index 6f86a8b..17514bc 100644
--- a/contributors.rst
+++ b/contributors.rst
@@ -1,3 +1,4 @@
Alexandre Lissy
David Malcolm
+David Narvaez
Tom Tromey
12 years, 4 months
[gcc-python-plugin] docs: add 0.8 release notes
by David Hugh Malcolm
commit 8f3d448178a1c5274d66338ca9524a65f7cb8014
Author: David Malcolm <dmalcolm(a)redhat.com>
Date: Tue Jan 10 14:37:47 2012 -0500
docs: add 0.8 release notes
docs/0.8.rst | 213 ++++++++++++++++++++++++++++++++++++++++++++++++
docs/release-notes.rst | 5 +-
2 files changed, 216 insertions(+), 2 deletions(-)
---
diff --git a/docs/0.8.rst b/docs/0.8.rst
new file mode 100644
index 0000000..73f0d20
--- /dev/null
+++ b/docs/0.8.rst
@@ -0,0 +1,213 @@
+.. Copyright 2012 David Malcolm <dmalcolm(a)redhat.com>
+ Copyright 2012 Red Hat, Inc.
+
+ This is free software: you can redistribute it and/or modify it
+ under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful, but
+ WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see
+ <http://www.gnu.org/licenses/>.
+
+0.8
+~~~
+
+Thanks to David Narvaez and Tom Tromey for their code contributions to this
+release.
+
+Changes to the GCC Python Plugin
+================================
+
+Initial C++ support
+-------------------
+This release adds the beginnings of C++ support: :py:class:`gcc.FunctionDecl`
+instances now have a "fullname" attribute, along with "is_public",
+"is_private", "is_protected", "is_static" booleans.
+
+For example, given this code:
+
+ .. code-block:: c++
+
+ namespace Example {
+ struct Coord {
+ int x;
+ int y;
+ };
+
+ class Widget {
+ public:
+ void set_location(const struct Coord& coord);
+ };
+ };
+
+`set_location`'s fullname is::
+
+ 'void Example::Widget::set_location(const Example::Coord&)'
+
+This is only present when the plugin is invoked from the C++ frontend
+(`cc1plus`), gracefully handling the case when we're invoked from other
+language frontends.
+
+Similarly, :py:class:`gcc.MethodType` has gained an "argument_types" attribute.
+
+Unconditional warnings
+----------------------
+The :py:func:`gcc.warning()` function in previous versions of the plugin required an
+"option" argument, such as ``gcc.Option('-Wformat')``
+
+It's now possible to emit an unconditional warning, by supplying `None` for
+this argument, which is now the default value::
+
+ gcc.warning(func.start, 'this is an unconditional warning')
+
+.. code-block:: bash
+
+ $ ./gcc-with-python script.py input.c
+ input.c:25:1: warning: this is an unconditional warning [enabled by default]
+
+which will be an error if `-Werror` is supplied as a command-line argument to gcc:
+
+.. code-block:: bash
+
+ $ ./gcc-with-python script.py -Werror input.c
+ input.c:25:1: error: this is an unconditional warning [-Werror]
+
+Improvements to :doc:`gcc-with-cpychecker </cpychecker>`
+========================================================
+
+The :doc:`"libcpychecker" </cpychecker>` Python code is a large example of
+using the plugin: it extends GCC with code that tries to detect various bugs
+in CPython extension modules.
+
+As of this release, all of the errors emitted by the tool have been converted
+to warnings. This should make `gcc-with-cpychecker` more usable as a drop-in
+replacement for `gcc`: the first source file with a refcounting error should
+no longer terminate the build (unless the program uses ``-Werror``, of
+course).
+
+Verification of PyMethodDef tables
+----------------------------------
+
+This release adds checking of tables of PyMethodDef initialization values, used
+by Python extension modules for binding C functions to Python methods.
+
+The checker will verify that the signatures of the callbacks match the
+flags, and that the such tables are NULL terminated::
+
+ input.c:48:22: warning: flags do not match callback signature for 'test' within PyMethodDef table
+ input.c:48:22: note: expected ml_meth callback of type "PyObject (fn)(someobject *, PyObject *)" (2 arguments)
+ input.c:48:22: note: actual type of underlying callback: struct PyObject * <Tc58> (struct PyObject *, struct PyObject *, struct PyObject *) (3 arguments)
+ input.c:48:22: note: see http://docs.python.org/c-api/structures.html#PyMethodDef
+
+Coverage of the CPython API
+---------------------------
+
+When the checker warns about code that can erroneously pass ``NULL`` to
+various CPython API entrypoints which are known to implicitly dereference
+those arguments, the checker will now add an explanatory note about why it
+is complaining.
+
+For example::
+
+ input.c: In function 'test':
+ input.c:38:33: warning: calling PyString_AsString with NULL (gcc.VarDecl('repr_args')) as argument 1 at input.c:38
+ input.c:31:15: note: when PyObject_Repr() fails at: repr_args = PyObject_Repr(args);
+ input.c:38:33: note: PyString_AsString() invokes Py_TYPE() on the pointer via the PyString_Check() macro, thus accessing (NULL)->ob_type
+ input.c:27:1: note: graphical error report for function 'test' written out to 'input.c.test-refcount-errors.html'
+
+The checker will now verify the argument lists of invocations of
+`PyObject_CallFunctionObjArgs
+<http://docs.python.org/c-api/object.html#PyObject_CallFunctionObjArgs>`_ and
+`PyObject_CallMethodObjArgs
+<http://docs.python.org/c-api/object.html#PyObject_CallMethodObjArgs>`_,
+checking that all of the arguments are of the correct type
+(PyObject* or subclasses), and that the list is NULL-terminated::
+
+ input.c: In function 'test':
+ input.c:33:5: warning: argument 2 had type char[12] * but was expecting a PyObject* (or subclass)
+ input.c:33:5: warning: arguments to PyObject_CallFunctionObjArgs were not NULL-terminated
+
+This release also adds heuristics for the behavior of the following CPython API
+entrypoints:
+
+ * PyArg_Parse
+ * PyCObject_{As,From}VoidPtr
+ * PyCallable_Check
+ * PyCapsule_GetPointer
+ * PyErr_{NewException,SetNone,WarnEx}
+ * PyEval_CallObjectWithKeywords
+ * PyEval_{Save,Restore}Thread (and thus the Py_{BEGIN,END}_ALLOW_THREADS
+ macros)
+ * PyList_{GetItem,Size}
+ * PyLong_FromLongLong
+ * PyMapping_Size
+ * PyModule_GetDict
+ * PyObject_AsFileDescriptor
+ * PyObject_Call{Function,FunctionObjArgs,MethodObjArgs}
+ * PyObject_Generic{Get,Set}Attr
+ * PyString_Size
+ * PyTuple_Pack
+ * PyUnicode_AsUTF8String
+ * Py_AtExit
+
+Bug fixes
+---------
+
+* gcc-with-cpychecker will now try harder on functions that are too
+ complicated to fully handle. Previously, when a function was too
+ complicated for the reference-count tracker to fully analyze, it would give
+ up, performing no analysis. The checker will now try to obtain at least
+ some subset of the list of all traces through the function, and analyze
+ those. It will still note that the function was too complicated to fully
+ analyze.
+
+ Given that we do a depth-first traversal of the tree, and that "success"
+ transitions are typically visited before "failure" transitions, this means
+ that it should at least analyze the trace in which all functions calls
+ succeed, together with traces in which some of the later calls fail.
+
+* the reference-count checker now correctly handles "static" `PyObject*` local
+ variables: a `static PyObject *` local preserves its value from call to call,
+ and can thus permanently own a reference.
+
+ Fixes a false-positive seen in psycopg2-2.4.2
+ (`psycopg/psycopgmodule.c:psyco_GetDecimalType`)
+ where the refcount checker erroneously reported that a reference was leaked.
+
+* the checker for Py_BuildValue("O") (and "S" and "N") was being too strict,
+ requiring a (PyObject*). Although it's not explicitly documented, it's
+ clear that these can also accept pointers to any PyObject subclass.
+
+ Fixes a false positive seen when running gcc-with-cpychecker on
+ coverage-3.5.1b1.tar.gz, in which `coverage/tracer.c:Tracer_trace` passes a
+ PyFrameObject* as an argument to such a call.
+
+* the reference-count checker now correctly suppresses reports about "leaks"
+ for traces that call a function that never return (such as `abort()`).
+
+ Fixes a false positive seen in rpm-4.9.1.2 in a handler for fatal errors:
+ (in python/rpmts-py.c:die) where the checker erroneously reported that a
+ reference was leaked.
+
+* `tp_iternext` callbacks are allowed to return NULL without setting an
+ exception. The reference-count checker will now notice if a function is
+ used in such a role, and suppress warnings about such behavior.
+
+* fixed various Python tracebacks (tickets
+ `#14 <https://fedorahosted.org/gcc-python-plugin/ticket/14>`_,
+ `#19 <https://fedorahosted.org/gcc-python-plugin/ticket/19>`_,
+ `#20 <https://fedorahosted.org/gcc-python-plugin/ticket/20>`_,
+ `#22 <https://fedorahosted.org/gcc-python-plugin/ticket/22>`_,
+ `#23 <https://fedorahosted.org/gcc-python-plugin/ticket/23>`_,
+ `#24 <https://fedorahosted.org/gcc-python-plugin/ticket/24>`_,
+ `#25 <https://fedorahosted.org/gcc-python-plugin/ticket/25>`_)
+
+* various other fixes
+
+
diff --git a/docs/release-notes.rst b/docs/release-notes.rst
index 3822838..e18c307 100644
--- a/docs/release-notes.rst
+++ b/docs/release-notes.rst
@@ -1,5 +1,5 @@
-.. Copyright 2011 David Malcolm <dmalcolm(a)redhat.com>
- Copyright 2011 Red Hat, Inc.
+.. Copyright 2011, 2012 David Malcolm <dmalcolm(a)redhat.com>
+ Copyright 2011, 2012 Red Hat, Inc.
This is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by
@@ -20,4 +20,5 @@ Release Notes
.. toctree::
+ 0.8.rst
0.7.rst
12 years, 4 months
[gcc-python-plugin] pre-release version bumps to 0.8
by David Hugh Malcolm
commit f6e5926708557eb6d55b68add143d4baaa8c3810
Author: David Malcolm <dmalcolm(a)redhat.com>
Date: Tue Jan 10 14:11:50 2012 -0500
pre-release version bumps to 0.8
docs/conf.py | 8 ++++----
gcc-python-plugin.spec | 5 ++++-
2 files changed, 8 insertions(+), 5 deletions(-)
---
diff --git a/docs/conf.py b/docs/conf.py
index 1b797ae..6a65104 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -1,5 +1,5 @@
-# Copyright 2011 David Malcolm <dmalcolm(a)redhat.com>
-# Copyright 2011 Red Hat, Inc.
+# Copyright 2011, 2012 David Malcolm <dmalcolm(a)redhat.com>
+# Copyright 2011, 2012 Red Hat, Inc.
#
# This is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by
@@ -65,9 +65,9 @@ copyright = u'2011, David Malcolm'
# built documents.
#
# The short X.Y version.
-version = '0.1'
+version = '0.8'
# The full version, including alpha/beta/rc tags.
-release = '0.1'
+release = '0.8'
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
diff --git a/gcc-python-plugin.spec b/gcc-python-plugin.spec
index 3a64249..8e33205 100644
--- a/gcc-python-plugin.spec
+++ b/gcc-python-plugin.spec
@@ -1,5 +1,5 @@
Name: gcc-python-plugin
-Version: 0.6
+Version: 0.8
Release: 1%{?dist}
Summary: GCC plugin that embeds Python
@@ -333,6 +333,9 @@ CheckPlugin \
%doc examples
%changelog
+* Tue Jan 10 2012 David Malcolm <dmalcolm(a)redhat.com> - 0.8-1
+- 0.8
+
* Tue Aug 2 2011 David Malcolm <dmalcolm(a)redhat.com> - 0.6-1
- 0.6
12 years, 4 months
[gcc-python-plugin] selftest fixes for 32-bit builds
by David Hugh Malcolm
commit 10a532059b4c50215b0590ffe7ecf378c653606f
Author: David Malcolm <dmalcolm(a)redhat.com>
Date: Tue Jan 10 13:33:39 2012 -0500
selftest fixes for 32-bit builds
run-test-suite.py | 8 ++++++++
1 files changed, 8 insertions(+), 0 deletions(-)
---
diff --git a/run-test-suite.py b/run-test-suite.py
index e3d4ee6..cd36a4a 100644
--- a/run-test-suite.py
+++ b/run-test-suite.py
@@ -81,6 +81,8 @@ class TestStream:
if six.MAXSIZE == 0x7fffffff:
expdata = expdata.replace('"Py_ssize_t *" (pointing to 64 bits)',
'"Py_ssize_t *" (pointing to 32 bits)')
+ expdata = expdata.replace('0x8000000000000000', '0x80000000')
+ expdata = expdata.replace('0x7fffffffffffffff', '0x7fffffff')
self.expdata = expdata
else:
self.expdata = ''
@@ -136,6 +138,12 @@ class TestStream:
line = re.sub(r'frozenset\(\[(.*)\]\)',
r'frozenset({\1})',
line)
+
+ # Avoid further 32-bit vs 64-bit differences due to int vs long
+ # overflow:
+ line = re.sub('0x7fffffffL', '0x7fffffff', line)
+ line = re.sub('0xffffffffL', '0xffffffff', line)
+
result += line + '\n'
return result
12 years, 4 months
[gcc-python-plugin] python 3: skip various tests that don't work on Python 3
by David Hugh Malcolm
commit 981a5e9bdc0ce580edff803f238bee63da08449f
Author: David Malcolm <dmalcolm(a)redhat.com>
Date: Tue Jan 10 06:27:45 2012 -0500
python 3: skip various tests that don't work on Python 3
run-test-suite.py | 11 +++++++++++
1 files changed, 11 insertions(+), 0 deletions(-)
---
diff --git a/run-test-suite.py b/run-test-suite.py
index e065f09..e3d4ee6 100644
--- a/run-test-suite.py
+++ b/run-test-suite.py
@@ -343,10 +343,18 @@ if six.PY3:
# Similarly for the PyString_ API:
exclude_tests_below('tests/cpychecker/refcounts/PyString_AsString')
exclude_tests_below('tests/cpychecker/refcounts/PyString_FromStringAndSize')
+ exclude_tests_below('tests/cpychecker/refcounts/PyString_Size')
+
+ # The PyCObject_ API was removed in 3.2:
+ exclude_tests_below('tests/cpychecker/refcounts/PyCObject_FromVoidPtr')
# The following tests happen to use PyInt or PyString APIs and thus we
# exclude them for now:
+ exclude_test('tests/cpychecker/refcounts/function-that-exits') # PyString
+ exclude_test('tests/cpychecker/refcounts/GIL/correct') # PyString
exclude_test('tests/cpychecker/refcounts/PyArg_ParseTuple/correct_O_bang') # PyString
+ exclude_test('tests/cpychecker/refcounts/PyObject_CallMethodObjArgs/correct') # PyString
+ exclude_test('tests/cpychecker/refcounts/PyObject_CallMethodObjArgs/incorrect') # PyString
exclude_test('tests/cpychecker/refcounts/PyStructSequence/correct') # PyInt
exclude_test('tests/cpychecker/refcounts/PySys_SetObject/correct') # PyString
exclude_test('tests/cpychecker/refcounts/subclass/handling') # PyString
@@ -355,6 +363,9 @@ if six.PY3:
# this test for Python 2:
exclude_test('tests/cpychecker/refcounts/module_handling')
+ # Uses METH_OLDARGS:
+ exclude_test('tests/cpychecker/refcounts/PyArg_Parse/correct_simple')
+
# Certain tests don't work for debug builds of Python:
if hasattr(sys, 'gettotalrefcount'):
exclude_test('tests/cpychecker/refcounts/PyDict_SetItem/correct')
12 years, 4 months
[gcc-python-plugin] python 3: fix up PyString_Type to PyBytes_Type in PyUnicode_AsUTF8String
by David Hugh Malcolm
commit 52e638470921479649c80331a8be8bb5880cbcde
Author: David Malcolm <dmalcolm(a)redhat.com>
Date: Tue Jan 10 06:11:09 2012 -0500
python 3: fix up PyString_Type to PyBytes_Type in PyUnicode_AsUTF8String
Introduce CPython.object_ctor_bytes() for API entrypoints that vary beteen
Python 2 and Python 3; fix up PyUnicode_AsUTF8String to use it.
Also, fix up "PyString_Type" to "PyBytes_Type" in expected output when
running under python 3.
Fixes tests/cpychecker/refcounts/PyUnicode_AsUTF8String/basic under Python 3
libcpychecker/refcounts.py | 19 ++++++++++++++++---
run-test-suite.py | 6 ++++--
2 files changed, 20 insertions(+), 5 deletions(-)
---
diff --git a/libcpychecker/refcounts.py b/libcpychecker/refcounts.py
index f7cd299..ff52f8b 100644
--- a/libcpychecker/refcounts.py
+++ b/libcpychecker/refcounts.py
@@ -414,6 +414,21 @@ class CPython(Facet):
t_failure.dest.cpython.set_exception('PyExc_MemoryError', stmt.loc)
return (nonnull, t_success, t_failure)
+ def object_ctor_bytes(self, stmt):
+ """
+ As per self.object_ctor(stmt, typename, typeobjname), returning:
+ (r_newobj, t_success, t_failure)
+ where the API entrypoint returns:
+ * a PyStringObject in Python 2
+ * a PyBytesObject in Python 3
+ """
+ if is_py3k():
+ typename, typeobjname = 'PyBytesObject', 'PyBytes_Type'
+ else:
+ typename, typeobjname = 'PyStringObject', 'PyString_Type'
+ return self.object_ctor(stmt,
+ typename, typeobjname)
+
def steal_reference(self, region):
log('steal_reference(%r)', region)
check_isinstance(region, Region)
@@ -2587,9 +2602,7 @@ class CPython(Facet):
self.state.raise_any_null_ptr_func_arg(stmt, 0, v_unicode,
why=invokes_Py_TYPE_via_macro(fnmeta,
'PyUnicode_Check'))
- r_newobj, t_success, t_failure = self.object_ctor(stmt,
- 'PyStringObject',
- 'PyString_Type')
+ r_newobj, t_success, t_failure = self.object_ctor_bytes(stmt)
return [t_success, t_failure]
########################################################################
diff --git a/run-test-suite.py b/run-test-suite.py
index f434859..e065f09 100644
--- a/run-test-suite.py
+++ b/run-test-suite.py
@@ -72,8 +72,10 @@ class TestStream:
# replace long literals with int literals:
expdata = re.sub('([0-9]+)L', '\g<1>', expdata)
expdata = re.sub('(0x[0-9a-f]+)L', '\g<1>', expdata)
- expdata = expdata.replace('struct PyStringObject',
- 'struct PyBytesObject')
+ expdata = expdata.replace('PyStringObject',
+ 'PyBytesObject')
+ expdata = expdata.replace('PyString_Type',
+ 'PyBytes_Type')
# The expected data is for 64-bit builds of Python
# Fix it up for 32-bit builds as necessary:
if six.MAXSIZE == 0x7fffffff:
12 years, 4 months
[gcc-python-plugin] fix python 3 incompatibility in tests/plugin/expressions/get_symbol
by David Hugh Malcolm
commit 0f48db0078466c499f8565923c2426321f60e1e4
Author: David Malcolm <dmalcolm(a)redhat.com>
Date: Tue Jan 10 05:06:44 2012 -0500
fix python 3 incompatibility in tests/plugin/expressions/get_symbol
tests/plugin/expressions/get_symbol/script.py | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
---
diff --git a/tests/plugin/expressions/get_symbol/script.py b/tests/plugin/expressions/get_symbol/script.py
index 5a53b36..cc6542d 100644
--- a/tests/plugin/expressions/get_symbol/script.py
+++ b/tests/plugin/expressions/get_symbol/script.py
@@ -34,7 +34,7 @@ for name in sorted(dir(gcc)):
t.add_row((':py:class:`gcc.%s`' % name,
'`%s`' % sym.strip()))
-from StringIO import StringIO
+from six import StringIO
s = StringIO()
t.write(s)
12 years, 4 months