Hi, in the API a lot of IDs get passed around are UUIDs. The point is that as long as you are not the entity generating the UUIDs the fact that these are UUIDs have no real significance to you. I suggest removing the validation of UUIDs from the receiving end. There is no real reason to make sure these are real UUIDs. It's another restriction we can remove from the interface simplifying the code and the interface.
Just to be clear I'm not saying that we should stop using UUIDs. For example, vdsm will keep generating task IDs as UUIDs. But the documentation will state that it could be *any* string value. If for some reason we choose to change the format of task IDs. There will be no need to change the interface.
The same goes for VM IDs. Currently the engine uses UUIDs but there is no reason for VDSM to enforce this and limit the engine from ever changing it in the future and using other string values.
On Thu, Aug 30, 2012 at 05:19:46PM -0400, Saggi Mizrahi wrote:
Hi, in the API a lot of IDs get passed around are UUIDs. The point is that as long as you are not the entity generating the UUIDs the fact that these are UUIDs have no real significance to you. I suggest removing the validation of UUIDs from the receiving end. There is no real reason to make sure these are real UUIDs. It's another restriction we can remove from the interface simplifying the code and the interface.
Just to be clear I'm not saying that we should stop using UUIDs. For example, vdsm will keep generating task IDs as UUIDs. But the documentation will state that it could be *any* string value. If for some reason we choose to change the format of task IDs. There will be no need to change the interface.
IMHO it is worth having strict UUIDs in preference to arbitrary strings, since their fixed size lets you deal with them more efficiently and predictably.
The same goes for VM IDs. Currently the engine uses UUIDs but there is no reason for VDSM to enforce this and limit the engine from ever changing it in the future and using other string values.
I'm not sure this is correct. IIUC the vmId value is used set the <uuid> element in the libvirt VM XML configuration, and libvirt will strictly validate for a well formed UUID string.
Daniel
On Thu, Aug 30, 2012 at 02:23:54PM -0700, Daniel P. Berrange wrote:
On Thu, Aug 30, 2012 at 05:19:46PM -0400, Saggi Mizrahi wrote:
Hi, in the API a lot of IDs get passed around are UUIDs. The point is that as long as you are not the entity generating the UUIDs the fact that these are UUIDs have no real significance to you. I suggest removing the validation of UUIDs from the receiving end. There is no real reason to make sure these are real UUIDs. It's another restriction we can remove from the interface simplifying the code and the interface.
Just to be clear I'm not saying that we should stop using UUIDs. For example, vdsm will keep generating task IDs as UUIDs. But the documentation will state that it could be *any* string value. If for some reason we choose to change the format of task IDs. There will be no need to change the interface.
IMHO it is worth having strict UUIDs in preference to arbitrary strings, since their fixed size lets you deal with them more efficiently and predictably.
The same goes for VM IDs. Currently the engine uses UUIDs but there is no reason for VDSM to enforce this and limit the engine from ever changing it in the future and using other string values.
I'm not sure this is correct. IIUC the vmId value is used set the <uuid> element in the libvirt VM XML configuration, and libvirt will strictly validate for a well formed UUID string.
Dan, you are correct. vmId==libvirt's domain uuid.
----- Original Message -----
From: "Saggi Mizrahi" smizrahi@redhat.com To: "arch" arch@ovirt.org, "VDSM Project Development" vdsm-devel@lists.fedorahosted.org Sent: Friday, August 31, 2012 12:19:46 AM Subject: [vdsm] [RFC] Implied UUIDs in API
Hi, in the API a lot of IDs get passed around are UUIDs. The point is that as long as you are not the entity generating the UUIDs the fact that these are UUIDs have no real significance to you. I suggest removing the validation of UUIDs from the receiving end. There is no real reason to make sure these are real UUIDs. It's another restriction we can remove from the interface simplifying the code and the interface.
Just to be clear I'm not saying that we should stop using UUIDs. For example, vdsm will keep generating task IDs as UUIDs. But the documentation will state that it could be *any* string value. If for some reason we choose to change the format of task IDs. There will be no need to change the interface.
The same goes for VM IDs. Currently the engine uses UUIDs but there is no reason for VDSM to enforce this and limit the engine from ever changing it in the future and using other string values.
I agree that UUID is just a method of generating unique strings, there is no reason to validate the value nor the format.
Thanks, Alon.
On 08/31/2012 12:33 AM, Alon Bar-Lev wrote:
----- Original Message -----
From: "Saggi Mizrahi" smizrahi@redhat.com To: "arch" arch@ovirt.org, "VDSM Project Development" vdsm-devel@lists.fedorahosted.org Sent: Friday, August 31, 2012 12:19:46 AM Subject: [vdsm] [RFC] Implied UUIDs in API
Hi, in the API a lot of IDs get passed around are UUIDs. The point is that as long as you are not the entity generating the UUIDs the fact that these are UUIDs have no real significance to you. I suggest removing the validation of UUIDs from the receiving end. There is no real reason to make sure these are real UUIDs. It's another restriction we can remove from the interface simplifying the code and the interface.
Just to be clear I'm not saying that we should stop using UUIDs. For example, vdsm will keep generating task IDs as UUIDs. But the documentation will state that it could be *any* string value. If for some reason we choose to change the format of task IDs. There will be no need to change the interface.
The same goes for VM IDs. Currently the engine uses UUIDs but there is no reason for VDSM to enforce this and limit the engine from ever changing it in the future and using other string values.
I agree that UUID is just a method of generating unique strings, there is no reason to validate the value nor the format.
I'm with daniel on this one - knowing a field is a uuid makes it easier to deal with in the db and code around it compared to a string (stored binary instead of string representation, etc.)
----- Original Message -----
From: "Itamar Heim" iheim@redhat.com To: "Alon Bar-Lev" alonbl@redhat.com Cc: "Saggi Mizrahi" smizrahi@redhat.com, "arch" arch@ovirt.org, "VDSM Project Development" vdsm-devel@lists.fedorahosted.org Sent: Friday, August 31, 2012 12:23:38 PM Subject: Re: [vdsm] [RFC] Implied UUIDs in API
On 08/31/2012 12:33 AM, Alon Bar-Lev wrote:
----- Original Message -----
From: "Saggi Mizrahi" smizrahi@redhat.com To: "arch" arch@ovirt.org, "VDSM Project Development" vdsm-devel@lists.fedorahosted.org Sent: Friday, August 31, 2012 12:19:46 AM Subject: [vdsm] [RFC] Implied UUIDs in API
Hi, in the API a lot of IDs get passed around are UUIDs. The point is that as long as you are not the entity generating the UUIDs the fact that these are UUIDs have no real significance to you. I suggest removing the validation of UUIDs from the receiving end. There is no real reason to make sure these are real UUIDs. It's another restriction we can remove from the interface simplifying the code and the interface.
Just to be clear I'm not saying that we should stop using UUIDs. For example, vdsm will keep generating task IDs as UUIDs. But the documentation will state that it could be *any* string value. If for some reason we choose to change the format of task IDs. There will be no need to change the interface.
The same goes for VM IDs. Currently the engine uses UUIDs but there is no reason for VDSM to enforce this and limit the engine from ever changing it in the future and using other string values.
I agree that UUID is just a method of generating unique strings, there is no reason to validate the value nor the format.
I'm with daniel on this one - knowing a field is a uuid makes it easier to deal with in the db and code around it compared to a string (stored binary instead of string representation, etc.)
Why to store binary?
Alon.
On 08/31/2012 11:27 AM, Alon Bar-Lev wrote:
----- Original Message -----
From: "Itamar Heim" iheim@redhat.com To: "Alon Bar-Lev" alonbl@redhat.com Cc: "Saggi Mizrahi" smizrahi@redhat.com, "arch" arch@ovirt.org, "VDSM Project Development" vdsm-devel@lists.fedorahosted.org Sent: Friday, August 31, 2012 12:23:38 PM Subject: Re: [vdsm] [RFC] Implied UUIDs in API
On 08/31/2012 12:33 AM, Alon Bar-Lev wrote:
----- Original Message -----
From: "Saggi Mizrahi" smizrahi@redhat.com To: "arch" arch@ovirt.org, "VDSM Project Development" vdsm-devel@lists.fedorahosted.org Sent: Friday, August 31, 2012 12:19:46 AM Subject: [vdsm] [RFC] Implied UUIDs in API
Hi, in the API a lot of IDs get passed around are UUIDs. The point is that as long as you are not the entity generating the UUIDs the fact that these are UUIDs have no real significance to you. I suggest removing the validation of UUIDs from the receiving end. There is no real reason to make sure these are real UUIDs. It's another restriction we can remove from the interface simplifying the code and the interface.
Just to be clear I'm not saying that we should stop using UUIDs. For example, vdsm will keep generating task IDs as UUIDs. But the documentation will state that it could be *any* string value. If for some reason we choose to change the format of task IDs. There will be no need to change the interface.
The same goes for VM IDs. Currently the engine uses UUIDs but there is no reason for VDSM to enforce this and limit the engine from ever changing it in the future and using other string values.
I agree that UUID is just a method of generating unique strings, there is no reason to validate the value nor the format.
I'm with daniel on this one - knowing a field is a uuid makes it easier to deal with in the db and code around it compared to a string (stored binary instead of string representation, etc.)
Why to store binary?
An UUID stored in its binary format takes 16 bytes. Stored as an string it takes 36 bytes, and more than 72 bytes in memory in the engine. The amount of CPU needed to create/compare/etc is proportional.
The engine takes advantage of this and uses an specialized UUID class and a specialized database type to manage them. If we change to arbitrary strings then a *lot* of changes have to be done to the engine.
----- Original Message -----
From: "Juan Hernandez" jhernand@redhat.com To: "Alon Bar-Lev" alonbl@redhat.com Cc: "Itamar Heim" iheim@redhat.com, "arch" arch@ovirt.org, "VDSM Project Development" vdsm-devel@lists.fedorahosted.org Sent: Friday, August 31, 2012 12:36:10 PM Subject: Re: [vdsm] [RFC] Implied UUIDs in API
On 08/31/2012 11:27 AM, Alon Bar-Lev wrote:
----- Original Message -----
From: "Itamar Heim" iheim@redhat.com To: "Alon Bar-Lev" alonbl@redhat.com Cc: "Saggi Mizrahi" smizrahi@redhat.com, "arch" arch@ovirt.org, "VDSM Project Development" vdsm-devel@lists.fedorahosted.org Sent: Friday, August 31, 2012 12:23:38 PM Subject: Re: [vdsm] [RFC] Implied UUIDs in API
On 08/31/2012 12:33 AM, Alon Bar-Lev wrote:
----- Original Message -----
From: "Saggi Mizrahi" smizrahi@redhat.com To: "arch" arch@ovirt.org, "VDSM Project Development" vdsm-devel@lists.fedorahosted.org Sent: Friday, August 31, 2012 12:19:46 AM Subject: [vdsm] [RFC] Implied UUIDs in API
Hi, in the API a lot of IDs get passed around are UUIDs. The point is that as long as you are not the entity generating the UUIDs the fact that these are UUIDs have no real significance to you. I suggest removing the validation of UUIDs from the receiving end. There is no real reason to make sure these are real UUIDs. It's another restriction we can remove from the interface simplifying the code and the interface.
Just to be clear I'm not saying that we should stop using UUIDs. For example, vdsm will keep generating task IDs as UUIDs. But the documentation will state that it could be *any* string value. If for some reason we choose to change the format of task IDs. There will be no need to change the interface.
The same goes for VM IDs. Currently the engine uses UUIDs but there is no reason for VDSM to enforce this and limit the engine from ever changing it in the future and using other string values.
I agree that UUID is just a method of generating unique strings, there is no reason to validate the value nor the format.
I'm with daniel on this one - knowing a field is a uuid makes it easier to deal with in the db and code around it compared to a string (stored binary instead of string representation, etc.)
Why to store binary?
An UUID stored in its binary format takes 16 bytes. Stored as an string it takes 36 bytes, and more than 72 bytes in memory in the engine. The amount of CPU needed to create/compare/etc is proportional.
The engine takes advantage of this and uses an specialized UUID class and a specialized database type to manage them. If we change to arbitrary strings then a *lot* of changes have to be done to the engine.
We are trying to reduce types of vdsm to simplify the VDSM-next API.
I guess it will derive a lot of changes in the engine anyway...
32->72 bytes in memory of JVM is not something that I care, JVM is not optimized for memory use in any sense.
I am not sure that compare in database of binary or string has significant impact, if there is a proper indexing, and if foreign key is used to cascade.
Regards, Alon.
On 08/31/2012 03:36 PM, Alon Bar-Lev wrote:
----- Original Message -----
From: "Juan Hernandez" jhernand@redhat.com To: "Alon Bar-Lev" alonbl@redhat.com Cc: "Itamar Heim" iheim@redhat.com, "arch" arch@ovirt.org, "VDSM Project Development" vdsm-devel@lists.fedorahosted.org Sent: Friday, August 31, 2012 12:36:10 PM Subject: Re: [vdsm] [RFC] Implied UUIDs in API
On 08/31/2012 11:27 AM, Alon Bar-Lev wrote:
----- Original Message -----
From: "Itamar Heim" iheim@redhat.com To: "Alon Bar-Lev" alonbl@redhat.com Cc: "Saggi Mizrahi" smizrahi@redhat.com, "arch" arch@ovirt.org, "VDSM Project Development" vdsm-devel@lists.fedorahosted.org Sent: Friday, August 31, 2012 12:23:38 PM Subject: Re: [vdsm] [RFC] Implied UUIDs in API
On 08/31/2012 12:33 AM, Alon Bar-Lev wrote:
----- Original Message -----
From: "Saggi Mizrahi" smizrahi@redhat.com To: "arch" arch@ovirt.org, "VDSM Project Development" vdsm-devel@lists.fedorahosted.org Sent: Friday, August 31, 2012 12:19:46 AM Subject: [vdsm] [RFC] Implied UUIDs in API
Hi, in the API a lot of IDs get passed around are UUIDs. The point is that as long as you are not the entity generating the UUIDs the fact that these are UUIDs have no real significance to you. I suggest removing the validation of UUIDs from the receiving end. There is no real reason to make sure these are real UUIDs. It's another restriction we can remove from the interface simplifying the code and the interface.
Just to be clear I'm not saying that we should stop using UUIDs. For example, vdsm will keep generating task IDs as UUIDs. But the documentation will state that it could be *any* string value. If for some reason we choose to change the format of task IDs. There will be no need to change the interface.
The same goes for VM IDs. Currently the engine uses UUIDs but there is no reason for VDSM to enforce this and limit the engine from ever changing it in the future and using other string values.
I agree that UUID is just a method of generating unique strings, there is no reason to validate the value nor the format.
I'm with daniel on this one - knowing a field is a uuid makes it easier to deal with in the db and code around it compared to a string (stored binary instead of string representation, etc.)
Why to store binary?
An UUID stored in its binary format takes 16 bytes. Stored as an string it takes 36 bytes, and more than 72 bytes in memory in the engine. The amount of CPU needed to create/compare/etc is proportional.
The engine takes advantage of this and uses an specialized UUID class and a specialized database type to manage them. If we change to arbitrary strings then a *lot* of changes have to be done to the engine.
We are trying to reduce types of vdsm to simplify the VDSM-next API.
I guess it will derive a lot of changes in the engine anyway...
32->72 bytes in memory of JVM is not something that I care, JVM is not optimized for memory use in any sense.
that doesn't mean you should abuse it. it's not a single item. its all the items.
I am not sure that compare in database of binary or string has significant impact, if there is a proper indexing, and if foreign key is used to cascade.
Regards, Alon.
----- Original Message -----
From: "Itamar Heim" iheim@redhat.com To: "Alon Bar-Lev" alonbl@redhat.com Cc: "Juan Hernandez" jhernand@redhat.com, "arch" arch@ovirt.org, "VDSM Project Development" vdsm-devel@lists.fedorahosted.org Sent: Friday, August 31, 2012 4:22:14 PM Subject: Re: [vdsm] [RFC] Implied UUIDs in API
On 08/31/2012 03:36 PM, Alon Bar-Lev wrote:
----- Original Message -----
From: "Juan Hernandez" jhernand@redhat.com To: "Alon Bar-Lev" alonbl@redhat.com Cc: "Itamar Heim" iheim@redhat.com, "arch" arch@ovirt.org, "VDSM Project Development" vdsm-devel@lists.fedorahosted.org Sent: Friday, August 31, 2012 12:36:10 PM Subject: Re: [vdsm] [RFC] Implied UUIDs in API
On 08/31/2012 11:27 AM, Alon Bar-Lev wrote:
----- Original Message -----
From: "Itamar Heim" iheim@redhat.com To: "Alon Bar-Lev" alonbl@redhat.com Cc: "Saggi Mizrahi" smizrahi@redhat.com, "arch" arch@ovirt.org, "VDSM Project Development" vdsm-devel@lists.fedorahosted.org Sent: Friday, August 31, 2012 12:23:38 PM Subject: Re: [vdsm] [RFC] Implied UUIDs in API
On 08/31/2012 12:33 AM, Alon Bar-Lev wrote:
----- Original Message ----- > From: "Saggi Mizrahi" smizrahi@redhat.com > To: "arch" arch@ovirt.org, "VDSM Project Development" > vdsm-devel@lists.fedorahosted.org > Sent: Friday, August 31, 2012 12:19:46 AM > Subject: [vdsm] [RFC] Implied UUIDs in API > > Hi, in the API a lot of IDs get passed around are UUIDs. > The point is that as long as you are not the entity generating > the > UUIDs the fact that these are UUIDs have no real significance > to > you. > I suggest removing the validation of UUIDs from the receiving > end. > There is no real reason to make sure these are real UUIDs. > It's another restriction we can remove from the interface > simplifying > the code and the interface. > > Just to be clear I'm not saying that we should stop using > UUIDs. > For example, vdsm will keep generating task IDs as UUIDs. But > the > documentation will state that it could be *any* string value. > If for some reason we choose to change the format of task IDs. > There > will be no need to change the interface. > > The same goes for VM IDs. Currently the engine uses UUIDs but > there > is no reason for VDSM to enforce this and limit the engine > from > ever > changing it in the future and using other string values.
I agree that UUID is just a method of generating unique strings, there is no reason to validate the value nor the format.
I'm with daniel on this one - knowing a field is a uuid makes it easier to deal with in the db and code around it compared to a string (stored binary instead of string representation, etc.)
Why to store binary?
An UUID stored in its binary format takes 16 bytes. Stored as an string it takes 36 bytes, and more than 72 bytes in memory in the engine. The amount of CPU needed to create/compare/etc is proportional.
The engine takes advantage of this and uses an specialized UUID class and a specialized database type to manage them. If we change to arbitrary strings then a *lot* of changes have to be done to the engine.
We are trying to reduce types of vdsm to simplify the VDSM-next API.
I guess it will derive a lot of changes in the engine anyway...
32->72 bytes in memory of JVM is not something that I care, JVM is not optimized for memory use in any sense.
that doesn't mean you should abuse it. it's not a single item. its all the items.
I guess you want me to analyse our current implementation and find optimizations that can be done... or in different words, find entities we abuse now... :)
I am not sure that compare in database of binary or string has significant impact, if there is a proper indexing, and if foreign key is used to cascade.
Regards, Alon.
In my opinion the implications on the engine database storage are not relevant, the engine can still choose to store everything as UUID if it finds it more easier. The only change would be that VDMS wouldn't care if they are UUIDs or not. They can still *be* UUIDs. The only problem is that if other managers start creating entities you will have to be able to gracefully handle that in API responses.
I also wonder why libvirt chose to validate UUIDs. Because as I said, as long as you don't generate them the fact that the are *possibly* unique is of no consequence to you. Seeing as the field comes to libvirt from an XML stream there is no real issue with parsing and putting it in a properly null terminated buffer.
The main reason I want to rid away the UUID restriction is that I think that it will help us have namespacing.
As VDSM's API becomes open and the ovirt-engine less centralized it will be harder to know which VDSM side resources are owned by which managing entity. Being able to mangle in you data (eg. OVIRT_ENGINE_0003-321321-321321-3213) will make it easier to know which VMs, domains, storage connections, are owned by which entity without actually going into implementing access controls on VDSM resources.
Further more, subsystems that might generate IDs in a different way could be fitted more easily.
----- Original Message -----
From: "Alon Bar-Lev" alonbl@redhat.com To: "Itamar Heim" iheim@redhat.com Cc: "Juan Hernandez" jhernand@redhat.com, "arch" arch@ovirt.org, "VDSM Project Development" vdsm-devel@lists.fedorahosted.org Sent: Friday, August 31, 2012 9:52:33 AM Subject: Re: [vdsm] [RFC] Implied UUIDs in API
----- Original Message -----
From: "Itamar Heim" iheim@redhat.com To: "Alon Bar-Lev" alonbl@redhat.com Cc: "Juan Hernandez" jhernand@redhat.com, "arch" arch@ovirt.org, "VDSM Project Development" vdsm-devel@lists.fedorahosted.org Sent: Friday, August 31, 2012 4:22:14 PM Subject: Re: [vdsm] [RFC] Implied UUIDs in API
On 08/31/2012 03:36 PM, Alon Bar-Lev wrote:
----- Original Message -----
From: "Juan Hernandez" jhernand@redhat.com To: "Alon Bar-Lev" alonbl@redhat.com Cc: "Itamar Heim" iheim@redhat.com, "arch" arch@ovirt.org, "VDSM Project Development" vdsm-devel@lists.fedorahosted.org Sent: Friday, August 31, 2012 12:36:10 PM Subject: Re: [vdsm] [RFC] Implied UUIDs in API
On 08/31/2012 11:27 AM, Alon Bar-Lev wrote:
----- Original Message -----
From: "Itamar Heim" iheim@redhat.com To: "Alon Bar-Lev" alonbl@redhat.com Cc: "Saggi Mizrahi" smizrahi@redhat.com, "arch" arch@ovirt.org, "VDSM Project Development" vdsm-devel@lists.fedorahosted.org Sent: Friday, August 31, 2012 12:23:38 PM Subject: Re: [vdsm] [RFC] Implied UUIDs in API
On 08/31/2012 12:33 AM, Alon Bar-Lev wrote: > > > ----- Original Message ----- >> From: "Saggi Mizrahi" smizrahi@redhat.com >> To: "arch" arch@ovirt.org, "VDSM Project Development" >> vdsm-devel@lists.fedorahosted.org >> Sent: Friday, August 31, 2012 12:19:46 AM >> Subject: [vdsm] [RFC] Implied UUIDs in API >> >> Hi, in the API a lot of IDs get passed around are UUIDs. >> The point is that as long as you are not the entity >> generating >> the >> UUIDs the fact that these are UUIDs have no real >> significance >> to >> you. >> I suggest removing the validation of UUIDs from the >> receiving >> end. >> There is no real reason to make sure these are real UUIDs. >> It's another restriction we can remove from the interface >> simplifying >> the code and the interface. >> >> Just to be clear I'm not saying that we should stop using >> UUIDs. >> For example, vdsm will keep generating task IDs as UUIDs. >> But >> the >> documentation will state that it could be *any* string >> value. >> If for some reason we choose to change the format of task >> IDs. >> There >> will be no need to change the interface. >> >> The same goes for VM IDs. Currently the engine uses UUIDs >> but >> there >> is no reason for VDSM to enforce this and limit the engine >> from >> ever >> changing it in the future and using other string values. > > I agree that UUID is just a method of generating unique > strings, > there is no reason to validate the value nor the format.
I'm with daniel on this one - knowing a field is a uuid makes it easier to deal with in the db and code around it compared to a string (stored binary instead of string representation, etc.)
Why to store binary?
An UUID stored in its binary format takes 16 bytes. Stored as an string it takes 36 bytes, and more than 72 bytes in memory in the engine. The amount of CPU needed to create/compare/etc is proportional.
The engine takes advantage of this and uses an specialized UUID class and a specialized database type to manage them. If we change to arbitrary strings then a *lot* of changes have to be done to the engine.
We are trying to reduce types of vdsm to simplify the VDSM-next API.
I guess it will derive a lot of changes in the engine anyway...
32->72 bytes in memory of JVM is not something that I care, JVM is not optimized for memory use in any sense.
that doesn't mean you should abuse it. it's not a single item. its all the items.
I guess you want me to analyse our current implementation and find optimizations that can be done... or in different words, find entities we abuse now... :)
I am not sure that compare in database of binary or string has significant impact, if there is a proper indexing, and if foreign key is used to cascade.
Regards, Alon.
vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
----- Original Message -----
In my opinion the implications on the engine database storage are not relevant, the engine can still choose to store everything as UUID if it finds it more easier. The only change would be that VDMS wouldn't care if they are UUIDs or not. They can still *be* UUIDs. The only problem is that if other managers start creating entities you will have to be able to gracefully handle that in API responses.
Wrt db - nothing needs to change. The value is determined by engine today and can go on being a UUID even if VDSM doesn't validate it to be one.
I also wonder why libvirt chose to validate UUIDs. Because as I said, as long as you don't generate them the fact that the are *possibly* unique is of no consequence to you. Seeing as the field comes to libvirt from an XML stream there is no real issue with parsing and putting it in a properly null terminated buffer.
For fields on which underlying components apply limitations such as the VM ID passed to libvirt I think we should validate but note in the documentation that there is no guarantee that it is a UUID as implementation may change.
Wrt Daniel's comment about fixed length - I agree it is easier to deal with fixed length strings (we also rely on this in sanlock iirc) so I would keep a length limitation on the string.
The main reason I want to rid away the UUID restriction is that I think that it will help us have namespacing.
As VDSM's API becomes open and the ovirt-engine less centralized it will be harder to know which VDSM side resources are owned by which managing entity. Being able to mangle in you data (eg. OVIRT_ENGINE_0003-321321-321321-3213) will make it easier to know which VMs, domains, storage connections, are owned by which entity without actually going into implementing access controls on VDSM resources.
Further more, subsystems that might generate IDs in a different way could be fitted more easily.
----- Original Message -----
From: "Alon Bar-Lev" alonbl@redhat.com To: "Itamar Heim" iheim@redhat.com Cc: "Juan Hernandez" jhernand@redhat.com, "arch" arch@ovirt.org, "VDSM Project Development" vdsm-devel@lists.fedorahosted.org Sent: Friday, August 31, 2012 9:52:33 AM Subject: Re: [vdsm] [RFC] Implied UUIDs in API
----- Original Message -----
From: "Itamar Heim" iheim@redhat.com To: "Alon Bar-Lev" alonbl@redhat.com Cc: "Juan Hernandez" jhernand@redhat.com, "arch" arch@ovirt.org, "VDSM Project Development" vdsm-devel@lists.fedorahosted.org Sent: Friday, August 31, 2012 4:22:14 PM Subject: Re: [vdsm] [RFC] Implied UUIDs in API
On 08/31/2012 03:36 PM, Alon Bar-Lev wrote:
----- Original Message -----
From: "Juan Hernandez" jhernand@redhat.com To: "Alon Bar-Lev" alonbl@redhat.com Cc: "Itamar Heim" iheim@redhat.com, "arch" arch@ovirt.org, "VDSM Project Development" vdsm-devel@lists.fedorahosted.org Sent: Friday, August 31, 2012 12:36:10 PM Subject: Re: [vdsm] [RFC] Implied UUIDs in API
On 08/31/2012 11:27 AM, Alon Bar-Lev wrote:
----- Original Message ----- > From: "Itamar Heim" iheim@redhat.com > To: "Alon Bar-Lev" alonbl@redhat.com > Cc: "Saggi Mizrahi" smizrahi@redhat.com, "arch" > arch@ovirt.org, "VDSM Project Development" > vdsm-devel@lists.fedorahosted.org > Sent: Friday, August 31, 2012 12:23:38 PM > Subject: Re: [vdsm] [RFC] Implied UUIDs in API > > On 08/31/2012 12:33 AM, Alon Bar-Lev wrote: >> >> >> ----- Original Message ----- >>> From: "Saggi Mizrahi" smizrahi@redhat.com >>> To: "arch" arch@ovirt.org, "VDSM Project Development" >>> vdsm-devel@lists.fedorahosted.org >>> Sent: Friday, August 31, 2012 12:19:46 AM >>> Subject: [vdsm] [RFC] Implied UUIDs in API >>> >>> Hi, in the API a lot of IDs get passed around are UUIDs. >>> The point is that as long as you are not the entity >>> generating >>> the >>> UUIDs the fact that these are UUIDs have no real >>> significance >>> to >>> you. >>> I suggest removing the validation of UUIDs from the >>> receiving >>> end. >>> There is no real reason to make sure these are real UUIDs. >>> It's another restriction we can remove from the interface >>> simplifying >>> the code and the interface. >>> >>> Just to be clear I'm not saying that we should stop using >>> UUIDs. >>> For example, vdsm will keep generating task IDs as UUIDs. >>> But >>> the >>> documentation will state that it could be *any* string >>> value. >>> If for some reason we choose to change the format of task >>> IDs. >>> There >>> will be no need to change the interface. >>> >>> The same goes for VM IDs. Currently the engine uses UUIDs >>> but >>> there >>> is no reason for VDSM to enforce this and limit the engine >>> from >>> ever >>> changing it in the future and using other string values. >> >> I agree that UUID is just a method of generating unique >> strings, >> there is no reason to validate the value nor the format. > > I'm with daniel on this one - knowing a field is a uuid > makes > it > easier > to deal with in the db and code around it compared to a > string > (stored > binary instead of string representation, etc.) >
Why to store binary?
An UUID stored in its binary format takes 16 bytes. Stored as an string it takes 36 bytes, and more than 72 bytes in memory in the engine. The amount of CPU needed to create/compare/etc is proportional.
The engine takes advantage of this and uses an specialized UUID class and a specialized database type to manage them. If we change to arbitrary strings then a *lot* of changes have to be done to the engine.
We are trying to reduce types of vdsm to simplify the VDSM-next API.
I guess it will derive a lot of changes in the engine anyway...
32->72 bytes in memory of JVM is not something that I care, JVM is not optimized for memory use in any sense.
that doesn't mean you should abuse it. it's not a single item. its all the items.
I guess you want me to analyse our current implementation and find optimizations that can be done... or in different words, find entities we abuse now... :)
I am not sure that compare in database of binary or string has significant impact, if there is a proper indexing, and if foreign key is used to cascade.
Regards, Alon.
vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
On 09/02/2012 10:32 AM, Ayal Baron wrote:
----- Original Message -----
In my opinion the implications on the engine database storage are not relevant, the engine can still choose to store everything as UUID if it finds it more easier. The only change would be that VDMS wouldn't care if they are UUIDs or not. They can still *be* UUIDs. The only problem is that if other managers start creating entities you will have to be able to gracefully handle that in API responses.
Wrt db - nothing needs to change. The value is determined by engine today and can go on being a UUID even if VDSM doesn't validate it to be one.
I also wonder why libvirt chose to validate UUIDs. Because as I said, as long as you don't generate them the fact that the are *possibly* unique is of no consequence to you. Seeing as the field comes to libvirt from an XML stream there is no real issue with parsing and putting it in a properly null terminated buffer.
For fields on which underlying components apply limitations such as the VM ID passed to libvirt I think we should validate but note in the documentation that there is no guarantee that it is a UUID as implementation may change.
Wrt Daniel's comment about fixed length - I agree it is easier to deal with fixed length strings (we also rely on this in sanlock iirc) so I would keep a length limitation on the string.
for identities the caller pass, I think vdsm can use string and engine can use uuid as that string. but for identities generated by vdsm (task id's, maybe tomorrow image id's, etc.) - there is a huge different between "not validating" and formally using uuid's as the expected type to not throw engine expecting a uuid for these fields.
The main reason I want to rid away the UUID restriction is that I think that it will help us have namespacing.
As VDSM's API becomes open and the ovirt-engine less centralized it will be harder to know which VDSM side resources are owned by which managing entity. Being able to mangle in you data (eg. OVIRT_ENGINE_0003-321321-321321-3213) will make it easier to know which VMs, domains, storage connections, are owned by which entity without actually going into implementing access controls on VDSM resources.
Further more, subsystems that might generate IDs in a different way could be fitted more easily.
----- Original Message -----
From: "Alon Bar-Lev" alonbl@redhat.com To: "Itamar Heim" iheim@redhat.com Cc: "Juan Hernandez" jhernand@redhat.com, "arch" arch@ovirt.org, "VDSM Project Development" vdsm-devel@lists.fedorahosted.org Sent: Friday, August 31, 2012 9:52:33 AM Subject: Re: [vdsm] [RFC] Implied UUIDs in API
----- Original Message -----
From: "Itamar Heim" iheim@redhat.com To: "Alon Bar-Lev" alonbl@redhat.com Cc: "Juan Hernandez" jhernand@redhat.com, "arch" arch@ovirt.org, "VDSM Project Development" vdsm-devel@lists.fedorahosted.org Sent: Friday, August 31, 2012 4:22:14 PM Subject: Re: [vdsm] [RFC] Implied UUIDs in API
On 08/31/2012 03:36 PM, Alon Bar-Lev wrote:
----- Original Message -----
From: "Juan Hernandez" jhernand@redhat.com To: "Alon Bar-Lev" alonbl@redhat.com Cc: "Itamar Heim" iheim@redhat.com, "arch" arch@ovirt.org, "VDSM Project Development" vdsm-devel@lists.fedorahosted.org Sent: Friday, August 31, 2012 12:36:10 PM Subject: Re: [vdsm] [RFC] Implied UUIDs in API
On 08/31/2012 11:27 AM, Alon Bar-Lev wrote: > > > ----- Original Message ----- >> From: "Itamar Heim" iheim@redhat.com >> To: "Alon Bar-Lev" alonbl@redhat.com >> Cc: "Saggi Mizrahi" smizrahi@redhat.com, "arch" >> arch@ovirt.org, "VDSM Project Development" >> vdsm-devel@lists.fedorahosted.org >> Sent: Friday, August 31, 2012 12:23:38 PM >> Subject: Re: [vdsm] [RFC] Implied UUIDs in API >> >> On 08/31/2012 12:33 AM, Alon Bar-Lev wrote: >>> >>> >>> ----- Original Message ----- >>>> From: "Saggi Mizrahi" smizrahi@redhat.com >>>> To: "arch" arch@ovirt.org, "VDSM Project Development" >>>> vdsm-devel@lists.fedorahosted.org >>>> Sent: Friday, August 31, 2012 12:19:46 AM >>>> Subject: [vdsm] [RFC] Implied UUIDs in API >>>> >>>> Hi, in the API a lot of IDs get passed around are UUIDs. >>>> The point is that as long as you are not the entity >>>> generating >>>> the >>>> UUIDs the fact that these are UUIDs have no real >>>> significance >>>> to >>>> you. >>>> I suggest removing the validation of UUIDs from the >>>> receiving >>>> end. >>>> There is no real reason to make sure these are real UUIDs. >>>> It's another restriction we can remove from the interface >>>> simplifying >>>> the code and the interface. >>>> >>>> Just to be clear I'm not saying that we should stop using >>>> UUIDs. >>>> For example, vdsm will keep generating task IDs as UUIDs. >>>> But >>>> the >>>> documentation will state that it could be *any* string >>>> value. >>>> If for some reason we choose to change the format of task >>>> IDs. >>>> There >>>> will be no need to change the interface. >>>> >>>> The same goes for VM IDs. Currently the engine uses UUIDs >>>> but >>>> there >>>> is no reason for VDSM to enforce this and limit the engine >>>> from >>>> ever >>>> changing it in the future and using other string values. >>> >>> I agree that UUID is just a method of generating unique >>> strings, >>> there is no reason to validate the value nor the format. >> >> I'm with daniel on this one - knowing a field is a uuid >> makes >> it >> easier >> to deal with in the db and code around it compared to a >> string >> (stored >> binary instead of string representation, etc.) >> > > Why to store binary?
An UUID stored in its binary format takes 16 bytes. Stored as an string it takes 36 bytes, and more than 72 bytes in memory in the engine. The amount of CPU needed to create/compare/etc is proportional.
The engine takes advantage of this and uses an specialized UUID class and a specialized database type to manage them. If we change to arbitrary strings then a *lot* of changes have to be done to the engine.
We are trying to reduce types of vdsm to simplify the VDSM-next API.
I guess it will derive a lot of changes in the engine anyway...
32->72 bytes in memory of JVM is not something that I care, JVM is not optimized for memory use in any sense.
that doesn't mean you should abuse it. it's not a single item. its all the items.
I guess you want me to analyse our current implementation and find optimizations that can be done... or in different words, find entities we abuse now... :)
I am not sure that compare in database of binary or string has significant impact, if there is a proper indexing, and if foreign key is used to cascade.
Regards, Alon.
vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
vdsm-devel mailing list vdsm-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
Arch mailing list Arch@ovirt.org http://lists.ovirt.org/mailman/listinfo/arch
+1
Long ago it was not enforced and then changed. The motivation for the enforcing as I recall was that for VMs this was also the guest system ID. I don't know if this is the case now days.
Regards, Simon
Sent from my iPhone, please excuse any typos.
ב-31 באוג 2012, בשעה 00:19, Saggi Mizrahi smizrahi@redhat.com כתב/ה:
Hi, in the API a lot of IDs get passed around are UUIDs. The point is that as long as you are not the entity generating the UUIDs the fact that these are UUIDs have no real significance to you. I suggest removing the validation of UUIDs from the receiving end. There is no real reason to make sure these are real UUIDs. It's another restriction we can remove from the interface simplifying the code and the interface.
Just to be clear I'm not saying that we should stop using UUIDs. For example, vdsm will keep generating task IDs as UUIDs. But the documentation will state that it could be *any* string value. If for some reason we choose to change the format of task IDs. There will be no need to change the interface.
The same goes for VM IDs. Currently the engine uses UUIDs but there is no reason for VDSM to enforce this and limit the engine from ever changing it in the future and using other string values. _______________________________________________ Arch mailing list Arch@ovirt.org http://lists.ovirt.org/mailman/listinfo/arch
vdsm-devel@lists.fedorahosted.org