12:16:06 <E-P> #startmeeting Mer QA meeting 3/5/2012
12:16:06 <MerBot> Meeting started Thu May  3 12:16:06 2012 UTC.  The chair is E-P. Information about MeetBot at http://wiki.merproject.org/wiki/Meetings.
12:16:06 <MerBot> Useful Commands: #action #agreed #help #info #idea #link #topic.
12:16:20 <E-P> #topic Current status
12:16:25 <lbt> (I think you can feed it the logs later)
12:16:27 <timoph> (sorry. couldn't resist)
12:16:40 <lbt> the problem atm is that although copyproj works, it doesn't handle events correctly
12:16:47 <lbt> so that's still a WIP
12:17:04 <lbt> I doubt much will happen until post Tizen conf
12:17:24 <E-P> ok, sounds good
12:18:03 <lbt> yes, it'll let us make images
12:20:14 <E-P> anything else?
12:20:40 <E-P> lets move on
12:20:57 <lbt> ok
12:21:03 <E-P> #topic QA process
12:21:46 <E-P> I draw first draft about the Mer QA, http://wiki.merproject.org/wiki/Quality/Test_processes#Mer_QA_process
12:21:59 <E-P> lbt: is there some important parts missing?
12:22:57 <lbt> it's not 100% correct architecturally
12:23:30 <lbt> but from a flow pov it's fine
12:23:40 <lbt> 4/5 is clearly not there yet
12:24:13 <lbt> and I think we have 4 sent to vendors+maintainers
12:24:43 <lbt> we previously discussed having 6/7 be a replication of 4/5 but delayed
12:25:10 <lbt> so if a NO occured at 4/5 then the 6/7 subscribers would have lower test load
12:25:33 <lbt> there's also a time limit on 4/5/6/7 for vendors ... 48h we mentioned
12:25:55 <lbt> (you see why this uses a business process workflow tool :) )
12:27:14 <lbt> Vendor QA looks good
12:27:21 <E-P> lbt: yes, I didn't draw the delayed notifications
12:27:23 * timoph needs to go
12:27:29 <E-P> timoph: bye
12:27:34 <lbt> timoph: o/
12:27:35 <timoph> o/
12:27:54 <E-P> from the vendor QA picture the qa-reports is missing
12:28:03 <E-P> if the vendor wants to use it
12:28:21 <lbt> yes, there are other things we'll add in more detail - such as updates to bugzilla when the patch is accepted
12:28:45 <E-P> nod
12:28:48 <lbt> currently the patch is manually accepted after Mer QA step 6
12:28:57 <lbt> and that's worth noting I think
12:29:43 <lbt> 6 is actually more "update gerrit with results and notify someone to accept/reject"
12:30:07 <lbt> (tecnically the maintainer can accept a package BOSS rejects)
12:30:24 <E-P> I will add those to the description
12:30:58 <lbt> *nod* so the "do stuff on accept" is actually a discrete process
12:31:47 <E-P> that process is mainly for package change, the release testing process is almost the same
12:31:48 <lbt> oh, we did do "add notation to mentioned bugs" at a step concurrent with 2
12:32:45 <lbt> I'm not sure there is a release test
12:33:28 <lbt> I think the notification should be something like "in pre-release mode"
12:33:30 <lbt> but hmm
12:33:31 <lbt> no
12:33:40 <lbt> you're right
12:33:53 <lbt> when we do a pre-release we do a changelog
12:34:02 <lbt> that's a delta on package versions
12:34:04 <E-P> in the release testing, the qa process is triggered by someone
12:34:15 <lbt> that could/should drive the execution planner
12:34:24 <E-P> yes, giving the list of delta packages
12:34:59 <E-P> jumping directly to point 4
12:35:01 <lbt> so vendor QA would be triggered in the same way but told "this is a snap/pre//release delta" or
12:35:36 <lbt> wouldn't 2/3 still be needed
12:35:51 <lbt> we're not testing images, we're testing releases
12:36:09 <lbt> the vendor has a product release process which is a little different
12:37:22 <E-P> does the vendor need own OBS too, if the vendor has to compile the package for his/her device?
12:37:34 <lbt> we assume they'll have one
12:38:12 <lbt> it is possible vendors will recompile all of Mer "to be sure"
12:38:24 <lbt> we'd ask that this testing is done on Mer rpms
12:38:26 <E-P> should that be on the vendor's picture, before step 2
12:38:41 <lbt> I think we need another process there
12:38:50 <lbt> ah
12:39:06 <lbt> I see what you mean
12:40:10 <E-P> the vendor might not use the Mer rpms
12:40:13 <E-P> or cannot
12:40:22 <lbt> mmm
12:40:32 <lbt> 1.1 would be BOSS triggers vendor OBS to use new package
12:40:39 <lbt> 1.2 BOSS waits for rebuild
12:41:14 <lbt> yeah - we'd need to have some idea of what they're doing in that respect
12:41:29 <lbt> and we'd assess their report based on our knowledge of it
12:41:41 <E-P> where is the src of the changed package located, in gerrit and mer obs?
12:42:08 <lbt> it should go via MDS
12:42:23 <lbt> honestly don't know
12:42:39 <lbt> one job I have is to build a reference implementation of all this :D
12:42:52 <E-P> how it is done at the moment? the BOSS takes the src and creates a new package in OBS?
12:43:25 <lbt> something like that
12:43:41 <lbt> we may use copypac from Mer CI -> vendor OBS
12:44:25 <E-P> and then giving the correct repository url to image creator
12:44:31 <lbt> yeah - that would mimic how ci does it
12:45:12 <lbt> I'd say boss gets packages from Mer CI (not using MDS?)
12:47:11 <E-P> that will be one risk point, we will be wiser when we have first versions of image builder and test execution planner
12:47:24 <lbt> oh yes
12:48:45 <E-P> I was planning to setup OTS and nemo VM
12:49:00 <E-P> then we would have something to work with
12:49:17 <lbt> *nod*
12:50:13 <lbt> (BTW, for the future, I'm hoping to get OTS to break down into functional process units that can be workflowed by BOSS)
12:51:27 <E-P> can you explain that a bit more?
12:52:25 <lbt> OTS uses much the same tech as BOSS to execute various steps - but AFAIK it's hardcoded
12:53:02 <lbt> so I wanted to loosen the various steps to allow more flexible combinations
12:53:19 <lbt> and to permit vendors to perform additional steps
12:53:35 <lbt> it would also cut down on our maintenance burden
12:54:11 <lbt> nb ... I consider this at the stage of "architectural blueprint" ... the real world may have different ideas :)
12:54:26 <E-P> *nod*
12:54:45 <E-P> there are many things what is needed for OTS
12:55:27 <E-P> anything to add to the mer qa process?
12:56:16 <lbt> nope
12:56:38 <E-P> ok, I will wrap up our discussion then
12:58:20 <lbt> yep - it's been good to expand on it a little
12:59:20 <E-P> #info First Mer QA process, http://wiki.merproject.org/wiki/Quality/Test_processes#Mer_QA_process
12:59:23 <E-P> #info Mer QA: Notification system missing
12:59:28 <E-P> #info Mer QA: having 6/7 be a replication of 4/5 but delayed
12:59:28 <E-P> #info Mer QA: if a NO occured at 4/5 then the 6/7 subscribers would have lower test load
12:59:31 <E-P> #info Mer QA: a time limit on 4/5/6/7 for vendors ... 48h we mentioned
12:59:56 <E-P> #info Mer QA: Other things needed to add in more detail - such as updates to bugzilla when the patch is accepted
12:59:59 <E-P> #info Mer QA: Currently the patch is manually accepted after Mer QA step 6
13:00:02 <E-P> #info Mer QA: 6 is actually more "update gerrit with results and notify someone to accept/reject"
13:00:31 <E-P> #info Mer/Vendor QA: how to deliver correct version of packages to vendor, how the vendor compiles the packages
13:00:46 <E-P> #info Changes to OTS, break down into functional process units that can be workflowed by BOSS
13:01:27 <E-P> did I miss something?
13:02:25 <E-P> I will update the pictures, and continue working with the test selector
13:03:14 <E-P> #action E-P Update the QA pictures
13:03:36 <E-P> thanks lbt for participating
13:04:08 <lbt> np
13:04:11 <iekku> i hope i have some time to try to test
13:04:22 <lbt> this looks useful for setting goals+tasks too
13:04:42 <iekku> so i can also check if the wiki-documentation is good
13:05:04 <E-P> iekku: that would be nice, usually it is lacking
13:05:05 <lbt> who manages OTS ?
13:05:41 <E-P> lbt: no one i guess, i was planning to move it to the test-execution-tools gitorious project
13:06:16 <E-P> I have to check that together with l3i
13:06:31 <E-P> if they are still using it or developing it
13:06:51 <lbt> we should definitely find out if there is an upstream
13:07:01 <lbt> if not we can pull it in to Mer
13:07:11 <E-P> meego.gitorius was/is the upstream
13:07:30 <lbt> I'm torn - I hope there is no upstream so we can make it part of the BOSS code and manage it easier
13:07:41 <E-P> :)
13:07:48 <E-P> I will find that out
13:08:14 <E-P> #action E-P find out who is maintaining the OTS and is there any upstream project
13:08:30 <E-P> ok, have a great evening everyone
13:08:38 <E-P> #endmeeting