git commit 932744b
and 74c0392) resolved the "can not fetch multiple bags" issue, but I
think we still have an issue when eager fetching any type of
entity-related collection (xxxToMany mapping), right?
Granted, doing "for (next : List<Entity>) {
next.getSomeCollection().size() }" suffers the N+1 query issue
(actually, 'm*N+1' where 'm' is the number of collections being
fetched), but at least it would return the correct results (1 row with
all collections properly fetched, instead of the 48 mazz is seeing).
In the future, I think we could implement a more sophisticated mechanism
that removes the performance penalty:
entity ObjectA { @OneToMany Set<ObjectB> myObjectBs; }
entity ObjectB { @ManyToOne ObjectA theObjectA; }
List<ObjectA> resultObjects = criteriaBasedFetch(ObjectA.class, ...);
List<Integer> resultPKs = getPrimaryKeys(resultObjects);
List<ObjectB> relatedObjects = criteriaBasedFetch(ObjectB.class,
"theObjectA", resultPKs);
wireTogether(resultObjects, relatedObjects, "myObjectBs");
Using this bulk loading mechanism, and then reflection to wire the
objects together after the query returns, the performance impact reduces
to 'm + 1' (again, where 'm' is the number of collections being fetched).
-joseph
On 10/26/2010 12:47 PM, John Sanda wrote:
The reason I added the isPersistentBag logic is because hibernate
does
not allow you to simultaneously fetch multiple bags. See
http://opensource.atlassian.com/projects/hibernate/browse/HHH-1413. And
if I recall from when I encountered this, it seemed to be the case that
we have a number of collections that are stored as lists which results
in the bag semantics where we should be using sets.
- John
On 10/26/10 12:05 PM, Joseph Marques wrote:
> OK, I think the root is this issue is that we're not generating the
> appropriate SQL. We know that if we're left fetch joining to objects
> whose entity relationship is xxxToOne, then the join is safe because
> we'll get the same number of rows with the join than without. However,
> for xxxToMany relationships, the idea was that we would:
>
> 1) get the List<Entity> back
> 2) iterate over the list and eager fetch data while the xaction was
> still open -- for (next : List<Entity>) { next.getSomeCollection().size() }
>
> You can see this logic in the CriteriaQueryGenerator:
>
> for (String fetchJoin : getFetchFields(criteria)) {
> if (isPersistentBag(fetchJoin)) {
> addPersistentBag(fetchJoin);
> } else {
> results.append("LEFT JOIN FETCH");
> results.append(alias).append('.').append(fetchJoin);
> results.append(NL);
> }
> }
>
> So, it appears that sticking to the strict definition of isPersistentBag
> is not sufficient here, because there are still other cases that can
> return duplicates that aren't persistent bags. I would say that
> changing the if-condition check to look for either the OneToMany or
> ManyToMany annotation on the fetched relationship would be a good
> catch-all, but I'm pretty sure there was a reason we used
> isPersistentBag in the first place. Looking at SCM history, it shows
> that jsanda was responsible for this - I'm going to ping him, to see
> what he can add to this thread.
>
> On 10/26/2010 11:46 AM, John Mazzitelli wrote:
>> I just noticed that if I add
>>
>> q.setMaxResults(100)
>>
>> on my JPA queries, I get 1 row back always (which is good). Using
>> Hibernate API, that still doesn't work, but I think this talks about
>> that issue:
>>
>>
https://forums.hibernate.org/viewtopic.php?f=1&t=958745&start=0
>>
>>
>> On 10/26/2010 11:16 AM, Joseph Marques wrote:
>>> On 10/26/2010 10:07 AM, John Mazzitelli wrote:
>>>> Just did your quick tests - its interesting. The first query (the
>>>> original with all the joins) returned 48 rows as I said.
>>> 48 is the cross of the related rows your fetching: 1 x 16 x 1 x 3
>>>> If I just do a
>>>> single LEFT JOIN FETCH it depends what I'm fetching that determines
the
>>>> rows it returns:
>>>>
>>>> If I join resourceConfigurationDefinition, I get 1 row (good)
>>> Yup, one resource configuration definition for every type in the system.
>>>> If I join metricDefinitions, I get 16 rows (bad)
>>> There are 16 metrics for the Linux platform type.
>>>> If I join evenDefinition, I get 1 row (good)
>>> There is, by convention, either zero or one event definition for a type.
>>>> If I join operationDefinitions, I get 3 rows (bad)
>>> The Linux platform type has 3 operations: discovery, viewProcessList,
>>> clearYumMetadataCache
>>>> Here's the 5 queries I tested with:
>>>>
>>>> sqlStrings[0] = "SELECT resourcetype " + //
>>>> "FROM ResourceType resourcetype " + //
>>>> "LEFT JOIN FETCH
resourcetype.resourceConfigurationDefinition " + //
>>>> "LEFT JOIN FETCH resourcetype.metricDefinitions " +
//
>>>> "LEFT JOIN FETCH resourcetype.eventDefinitions " + //
>>>> "LEFT JOIN FETCH resourcetype.operationDefinitions " +
//
>>>> "WHERE ( resourcetype.id in ( :ids ) ) ";
>>>>
>>>> sqlStrings[1] = "SELECT resourcetype " + //
>>>> "FROM ResourceType resourcetype " + //
>>>> "LEFT JOIN FETCH
resourcetype.resourceConfigurationDefinition " + //
>>>> "WHERE ( resourcetype.id in ( :ids ) ) ";
>>>>
>>>> sqlStrings[2] = "SELECT resourcetype " + //
>>>> "FROM ResourceType resourcetype " + //
>>>> "LEFT JOIN FETCH resourcetype.metricDefinitions " +
//
>>>> "WHERE ( resourcetype.id in ( :ids ) ) ";
>>>>
>>>> sqlStrings[3] = "SELECT resourcetype " + //
>>>> "FROM ResourceType resourcetype " + //
>>>> "LEFT JOIN FETCH resourcetype.eventDefinitions " + //
>>>> "WHERE ( resourcetype.id in ( :ids ) ) ";
>>>>
>>>> sqlStrings[4] = "SELECT resourcetype " + //
>>>> "FROM ResourceType resourcetype " + //
>>>> "LEFT JOIN FETCH resourcetype.operationDefinitions " +
//
>>>> "WHERE ( resourcetype.id in ( :ids ) ) ";
>>>>
>>>>
>>>> On 10/25/2010 03:09 PM, Joseph Marques wrote:
>>>>> Yes, I've seen Hibernate do this from time to time when
left join
>>>>> fetching.
>>>>>
>>>>> Do a little investigation for me - how many rows do you get back if
you
>>>>> only fetch eventDefinitions versus if you only fetch
>>>>> operationDefinitions or resourceConfigurationDefinitions? Also, what
is
>>>>> the generated SQL for the query (if there is a join to a collection
>>>>> [xxxToMany mapping] anywhere, that would account for the duplicate
rows
>>>>> of the projected data.)
>>>>>
>>>>> On 10/23/2010 03:08 PM, John Mazzitelli wrote:
>>>>>> I don't get it. I run this code, and each System.out.println
shows me
>>>>>> that the results have 48 rows - but each query should only
produce 1 row.
>>>>>>
>>>>>> Note: if I go to admin/test/hibernate.jsp, and try this JPQL
string, it
>>>>>> actually only produces 1 row (the correct results!). I don't
know what
>>>>>> its doing differently, but here's the code that I tested with
- there
>>>>>> are actually 4 different ways I try it (two using JPA API, two
using
>>>>>> Hibernate API - for each I try using a collection of 1 ID
integer, and I
>>>>>> try using a single parameter value of Integer type).
>>>>>>
>>>>>> String sqlString = "SELECT resourcetype " + //
>>>>>> "FROM ResourceType resourcetype " + //
>>>>>> "LEFT JOIN FETCH
resourcetype.resourceConfigurationDefinition " + //
>>>>>> "LEFT JOIN FETCH resourcetype.metricDefinitions
" + //
>>>>>> "LEFT JOIN FETCH resourcetype.eventDefinitions
" + //
>>>>>> "LEFT JOIN FETCH
resourcetype.operationDefinitions " + //
>>>>>> "WHERE ( resourcetype.id in ( :ids ) ) ";
>>>>>>
>>>>>> ArrayList<Integer> idList = new
ArrayList<Integer>();
>>>>>> idList.add(new Integer(10014));
>>>>>>
>>>>>> Query q;
>>>>>> List<?> results;
>>>>>>
>>>>>> // JPA - parameter is an ArrayList of 1 ID
>>>>>> q = entityManager.createQuery(sqlString);
>>>>>> q.setParameter("ids", idList);
>>>>>> results = q.getResultList();
>>>>>> System.out.println("1~~~~~~~~~size=" +
results.size());
>>>>>>
>>>>>> // JPA - parameter is a single Integer value
>>>>>> q = entityManager.createQuery(sqlString);
>>>>>> q.setParameter("ids", new Integer(10014));
>>>>>> results = q.getResultList();
>>>>>> System.out.println("2~~~~~~~~~size=" +
results.size());
>>>>>>
>>>>>> Session session;
>>>>>> org.hibernate.Query hq;
>>>>>> session = (Session) entityManager.getDelegate();
>>>>>>
>>>>>> // Hibernate - parameter is an ArrayList of 1 ID
>>>>>> hq = session.createQuery(sqlString);
>>>>>> hq.setParameterList("ids", idList);
>>>>>> results = hq.list();
>>>>>> System.out.println("3~~~~~~~~~size=" +
results.size());
>>>>>>
>>>>>> // Hibernate - parameter is a single Integer value
>>>>>> hq = session.createQuery(sqlString);
>>>>>> hq.setParameter("ids", new Integer(10014));
>>>>>> results = hq.list();
>>>>>> System.out.println("4~~~~~~~~~size=" +
results.size());
>>>>>> _______________________________________________
>>>>>> rhq-devel mailing list
>>>>>> rhq-devel(a)lists.fedorahosted.org
>>>>>>
https://fedorahosted.org/mailman/listinfo/rhq-devel
>>>>> _______________________________________________
>>>>> rhq-devel mailing list
>>>>> rhq-devel(a)lists.fedorahosted.org
>>>>>
https://fedorahosted.org/mailman/listinfo/rhq-devel
>>>> _______________________________________________
>>>> rhq-devel mailing list
>>>> rhq-devel(a)lists.fedorahosted.org
>>>>
https://fedorahosted.org/mailman/listinfo/rhq-devel
>>> _______________________________________________
>>> rhq-devel mailing list
>>> rhq-devel(a)lists.fedorahosted.org
>>>
https://fedorahosted.org/mailman/listinfo/rhq-devel
>> _______________________________________________
>> rhq-devel mailing list
>> rhq-devel(a)lists.fedorahosted.org
>>
https://fedorahosted.org/mailman/listinfo/rhq-devel
> _______________________________________________
> rhq-devel mailing list
> rhq-devel(a)lists.fedorahosted.org
>
https://fedorahosted.org/mailman/listinfo/rhq-devel
_______________________________________________
rhq-devel mailing list
rhq-devel(a)lists.fedorahosted.org
https://fedorahosted.org/mailman/listinfo/rhq-devel