On 11/26/2016 09:09 AM, Dusty Mabe wrote:

It looks like all composes but rawhide composes have been failing since 11/21. Do we know what the problem is here?

Similarly there haven't been any new f25 ostrees since 11/16. I don't know if the two are related or not (probably). Anybody know what is going on or do I need to start digging?

Ok, Thanks to kevin for helping me start the debugging process.

Here is the pungi compose from last nights run that failed:

https://kojipkgs.fedoraproject.org/compose/twoweek/Fedora-Atomic-25-20161129...

Here is the runroot task that failed:

http://koji.fedoraproject.org/koji/taskinfo?taskID=16665772

Here is the runroot.log from the failure:

https://kojipkgs.fedoraproject.org//work/tasks/5772/16665772/runroot.log

Which shows us:

``` + lorax --product=Fedora-Atomic --version=25 --release=25 --source=file:///mnt/koji/compose/twoweek/Fedora-Atomic-25-20161129.0/work/x86_64/repo --variant=Cloud --nomacboot --noupgrade --isfinal --buildarch=x86_64 --volid=Fedora-C-dvd-x86_64-25 --installpkgs=fedora-productimg-cloud /mnt/koji/compose/twoweek/Fedora-Atomic-25-20161129.0/work/x86_64/buildinstall/Cloud 2016-11-29 07:48:07,878: Added 'lorax-repo-0': file:///mnt/koji/compose/twoweek/Fedora-Atomic-25-20161129.0/work/x86_64/repo 2016-11-29 07:48:07,878: Fetching metadata... Traceback (most recent call last): File "/usr/sbin/lorax", line 273, in <module> main() File "/usr/sbin/lorax", line 133, in main remove_temp=True, verify=opts.verify) File "/usr/lib/python3.5/site-packages/pylorax/__init__.py", line 215, in run os.makedirs(self.outputdir) File "/usr/lib64/python3.5/os.py", line 231, in makedirs makedirs(head, mode, exist_ok) File "/usr/lib64/python3.5/os.py", line 241, in makedirs mkdir(name, mode) PermissionError: [Errno 13] Permission denied: '/mnt/koji/compose/twoweek/Fedora-Atomic-25-20161129.0/work/x86_64/buildinstall' ```

So basically the failure is because it's not allowed to create the directory at /mnt/koji/compose/twoweek/Fedora-Atomic-25-20161129.0/work/x86_64/buildinstall

Can we investigate this further?

Dusty