09:04:04 #startmeeting CalDAV/CardDAV Community Contributions Kick-off Meeting 09:04:04 Meeting started Mon Sep 12 09:04:04 2016 UTC. The chair is chriadam. Information about MeetBot at http://wiki.merproject.org/wiki/Meetings. 09:04:04 Useful Commands: #action #agreed #help #info #idea #link #topic. 09:05:04 #link https://sailfishos.org/wiki/CalDAV_and_CardDAV_Community_Contributions 09:05:16 #topic Agenda 09:06:05 Hi everyone, the agenda is listed in the Wiki page 09:06:46 The idea of this meeting is to get people to introduce themselves, then I can explain a bit about what the project needs and why, and then we can try to get volunteers for stuff. 09:06:59 In some cases, we already have people who have volunteered (via email) and that's great! 09:07:16 :) 09:07:20 #topic Introductions 09:07:32 Hello ! 09:07:37 hello 09:07:38 Please introduce yourself with #info at the start, along with your name / irc nick and how you hope to contribute 09:08:03 #info Lucien Xu, Sfiet_Konstantin, I would like to take a look at unit-tests and code-related tasks 09:08:07 #info Chris Adams / chriadam - Sailor with Jolla, have contributed to CardDAV/CalDAV and other sync plugins. 09:08:28 #info Lewis Rockliffe - just observing 09:08:57 #info Nadir Hanid, nh1402, just observing. 09:09:03 #info William, I am complete newby at Mer, but I would like to get into CPP stuff, so probably simple bugfixing first 09:09:11 #info David Greaves / lbt - Sailor and Mer guy. Can setup test infra if needed 09:09:53 #info Damien Caliste, I would like to help contributing to unit tests and related bugs. 09:11:02 we'll wait 2 more minutes then continue to next steps. I know that a couple of people won't be able to be at this meeting due to travel or other commitments. 09:11:29 for those who did show up - thanks very much! hopefully this meeting will be a productive one. 09:13:01 #topic Overview 09:13:16 The wiki page has a pretty comprehensive summary, but the short version is: 09:13:41 The plugins are currently only manually tested, and then only against 3 servers (Fruux, Memotoo and Yahoo) 09:13:55 Going forward, we want to address these issues by: 09:14:22 a) increasing the number of services we test against (especially OwnCloud, Radicale, Baikal, COZY, Synology) 09:14:50 b) increasing the number of people who run manual tests when a change is made (to increase chance of finding regressions) 09:15:05 c) automating those tests to a larger extent 09:15:21 d) writing unit tests to comprehensively test some things which are amenable to unit testing. 09:15:56 From Jolla's perspective, we obviously would benefit from this in that our sync would be more reliable 09:17:04 From Community perspective, this will provide more reliable sync, but also hopefully give experience and learning about how SFOS works and how to work with Jolla going forward 09:17:46 because, historically, it's true that much of Jolla's processes etc were internal (SDK, Bugtracker, comms etc) 09:18:14 So hopefully through this project, we can push to reduce a lot of the friction which the community faces when trying to contribute to the SFOS project. 09:18:23 That's my hope, anyway. 09:18:31 Does anyone have any input / thoughts on this at this stage? 09:18:44 sounds like a good plan chriadam 09:19:15 Sounds great to me, and CalDAV/CardDAV is a great area for improvements. 09:19:26 caldav/cardav has been a sore point for a long time 09:20:23 Any specific issues with the focus being on testing more than feature development, at this stage? E.g., going forward we want to support things like Event Invitations, and also Todo's, but we figure at this stage that quality assurance is a better target in the short term. 09:21:27 ok 09:21:36 I agree, that QA is more urgent. It's better to build on safe fondations ! And, learning the code on QA will produce better merge request when adding new functionalities. 09:21:47 dcaliste: +1 09:21:48 that's my feeling, too. 09:21:49 great 09:21:58 I would improve usability (testing) on several platforms, especially ownCloud/NextCloud. There are quite nasty issues with both calendar and contacts. 09:22:26 #info First target is to improve stability and quality through improved testing (and related bug-fixing) 09:22:26 lbt: question for you: how easy it is to setup a jenkins instance ? 09:22:35 * SfietKonstantinW like CI 09:22:46 CI is a good idea, but probably a bit further down the track, IMO 09:23:14 chriadam: CI is good even for test coverage 09:23:20 so that we know which area to improve 09:23:22 SfietKonstantinW: if we're doing CI we'll stick with OBS and the other Mer automation stuff 09:23:49 SfietKonstantinW: there's testrunner-lite etc which run tests and only promote if they all pass 09:23:55 ha 09:23:58 but it's a promotion gate rather than a merge/commit gate 09:24:11 I was more about just getting information 09:24:37 oh, yeah - I'm not sure what's involved with enabling the CI gates for a particular project. 09:24:54 Sorry to be late ... 09:24:55 but let's cross that bridge once we have some tests to run ;-) 09:24:59 dr_gogeta86: no problem :-) 09:25:15 Ok, any last comments on the Overview stuff, or shall I continue? 09:25:17 chriadam: true 09:25:23 lbt count on me for infra 09:25:26 we can run tests locally :) 09:25:31 chriadam: please proceed 09:25:45 #topic Test Servers - The Plan 09:26:21 We have had people from community (e.g. Fabien/occirol) who have volunteered to set up instances of various services for us to test against. 09:27:01 is better to provide a reproducible clean rooms for that IMHO 09:27:03 occirol is traveling at the moment, he gets back late this week, so perhaps early next week would be a good idea for lbt, myself, occirol and dr_gogeta86 to have a quick chat on IRC about how to proceed there? 09:27:17 the idea is to have some VMs in the Mer infra against which we can run tests 09:27:59 dr_gogeta86: I agree - but need some way to allow the developer/tester to "click a button" to reset the VM to the "ready to begin test" state 09:28:13 any ideas about how to do this properly would be appreciated 09:28:31 that should be something we can handle 09:28:55 so, at this stage, the plan is: Early next week (~20th of Sept) have discussion and split up services. At least OwnCloud + Radicale + Baikal we have volunteers for. 09:29:22 the by the end of this month, should have at least those services running and able to be tested against in the Mer infra, I hope. 09:29:28 * lbt would like to add egroupware if that's OK 09:29:32 lbt: does that sound reasonable timeframe? 09:29:39 lbt: sure, if we find someone who can set it up 09:29:46 chriadam, we need also Bedework and Apple Calendar server 09:29:51 chriadam: I think the end of this month is tight 09:30:04 lbt: ok. shuffle back by 2 weeks? 09:30:04 it's 2 weeks and there's a fair bit of work and coordination 09:30:15 and I'm really busy 09:30:22 or is that still overly optimistic? mid oct? 09:30:35 yeah - sorry but this is going to be a bit slow 09:31:04 what I think we can do is decide on a single service and work on that 09:31:43 ok. I would vote for an OwnCloud instance (but if I understand Fabien correctly, he is willing to set everything up etc, so long as he has access to a VM) 09:31:44 lets see which is the best from a collab PoV (ie most available/responosive admin; easiest install etc) 09:32:04 OK 09:32:05 +1 for ownCloud 09:32:32 from Mer infra POV I guess it's "provide a VM which is sandboxed from everything else, which Fabien can get access to, which has an externally-visible address..." 09:33:12 yes 09:33:59 ok. so what is a realistic timeframe, do you think? have the discussion early next week, and then have the first server instance set up by... when? 09:34:31 lets aim for end sept for the first one 09:34:50 great. we can play it by ear - no huge rush, just want to have a concrete goal. if it slips, it slips. 09:34:57 docker images could be good for local testing 09:35:41 r0kk3rz: ah, so we can just point the plugin to 127.0.0.1:8080 or whatever? good idea 09:35:41 owncloud or nextcloud ? 09:35:52 owncloud has bigger installed base IMO 09:35:59 for now... 09:36:03 ok 09:36:11 right now owncloud and nextcloud "is the same" in practice 09:36:19 most of these services probably already provide docker images too 09:36:46 #info Discussion between lbt/chriadam/occirol/dr_gogeta86 around 20th Sept to discuss setting up test server instances in Mer infra 09:37:06 #info Goal to have first test server instance up and running (and usable for testing) by 30th Sept. 09:37:10 * lbt would certainly like a nextcloud instance for OSS community reasons 09:38:17 is someone willing to look into what's required to get a useful docker image running to perform tests against with buteo-sync-plugin-caldav, end-to-edn? 09:38:55 I'm not sure about using docker on the infra btw 09:39:06 ah 09:39:09 I need to understand the security/isolation side 09:39:14 true 09:39:19 it doesn't prevent local use fwiw 09:39:31 I'm very happy to be persuaded though 09:39:53 I don't know enough about it. maybe something to discuss further, in the discussion next week. 09:39:57 on the infra I'd probably use kvm VMs and provide a way to do a reset 09:40:18 yeah, sounds good. 09:40:27 ok, let's proceed to next topic: 09:40:38 #topic Simple System Test Scripts - Any Volunteers? 09:40:41 Some background: 09:41:13 once we have the capability to reset these services, then it'd be nice to have some simple "system test" scripts which verify basic functionality 09:41:36 e.g., add an account, perform a sync, dump the contents of the calendar/contacts database, ensure that it contains the appropriate content 09:42:27 this isn't C++ related, really, just some simple scripting 09:42:42 is anyone willing to write the first one of these scripts? 09:42:56 the idea being, as a developer I want to be able to run: 09:43:16 ./sanity_check.sh some.server.url 09:43:38 and have it do the test and say "yes, we got what we expected" or "you've broken the plugin" 09:43:41 vagrant might also be a good vm based env builder 09:44:54 to start with, the script can even ignore account-related stuff 09:45:50 If no-one is keen to do that, let's continue ;-) 09:45:58 :D 09:46:07 #topic Any Volunteers For MER#1623 ? 09:46:09 you don't have an assembly that loves scripting it seems 09:46:37 (it's a high friction item, I agree, because of the accounts + buteo complications) 09:46:38 #link https://bugs.merproject.org/show_bug.cgi?id=1623 09:46:38 Mer bug 1623 in buteo-sync-plugin-caldav "CalDAV import fails due to being unable to parse iCal data" [Task,New] 09:46:42 I've quickly looked into #1623. Is it correct that: 09:46:57 - the plugin is using mkcal to obtain ical data ? 09:47:06 - mkcal itself is using libical ? 09:47:22 mkcal doesn't, but kcalcore does 09:47:41 so, mkcal is a storage backend for kcalcore (and mkcal provides some extension apis and whatnot, but ignore that for this) 09:47:58 libical is used by Gnome IMHO, so such issues should have been corrected already, I guees ! 09:48:32 Currently libical is version 1.0.1 in Mer, maybe it would be interesting to try to link against 2.0.0 and test. 09:48:33 you retrieve a KCalCore::Event (or really, an Incidence), and then pass it to libical (via a complex pathway of calls in kcalcore) 09:48:39 dcaliste: right, I agree. 09:48:52 dcaliste: but it might be "higher up" 09:49:03 #info https://github.com/owncloud/vm 09:49:08 e.g., in the plugin when we parse the data from the server response, do we unescape things correctly? 09:49:53 I can do this and see if it helps. If not I can try to track further. Currently, it was just reading code in diagonal. 09:50:27 dcaliste: see src/reader.cpp Reader::readResponse() 09:50:42 Do you have any "real" ical files that triggers #1623 or I can create some from the bug report ? 09:50:58 dcaliste: I believe that the one in the bug report does trigger it 09:50:59 I mean adding @ and breaking lines by hand. 09:51:11 I can't remember if I tested it personally o rnot 09:51:29 dcaliste: thanks for volunteering :-) 09:51:33 Ok, I'll try with the one of the bug report. 09:51:48 #agreed dcaliste to investigate MER#1623 09:52:11 as always, if you have any questions please feel free to pin gme on irc or email me 09:52:18 next topic! 09:52:32 #topic Any Volunteers For MER#1647 ? 09:52:56 this is another case where updating to latest libical may indeed solve it 09:53:05 but it's also a relatively simple case to work around. 09:53:19 1) check to see that the VCALENDAR definition contains just one event 09:53:38 2) if so, check to see if the UID is improperly located in the VCALENDAR definition instead of the VEVENT definition 09:53:50 chriadam: yes, I'll ask if I don't understand something, but first for me will be to reproduce the bug in a simple C++ executable linking on the buteo plugin. I'll share sources when ready if someone else want to try. 09:53:51 3) if so, move it into the VEVENT block 09:54:16 I can look into this bug. 09:54:18 William__: is this something that you'd be interested to investigate? 09:54:20 great! 09:54:25 :) 09:54:30 William__: do you have an SDK set up? 09:54:31 #link https://bugs.merproject.org/show_bug.cgi?id=1647 09:54:31 Mer bug 1647 in buteo-sync-plugin-caldav "CalDAV sync fails due to UID specified outside of VEVENT definition" [Task,New] 09:54:35 Hi, sorry I'm so so late, but I wasn't able to connect from my phone until now. I've been reading the logs through. I'd be willing to code the script-sanity thing but I'd need some time now since I'm quite busy right now 09:54:48 elfio_mvl: great! that'd be a huge help 09:54:49 no rush 09:55:25 I have an SDK, yes, but that's pretty much it. I will need some time to make things work. 09:55:27 #agreed elfio will contribute a simple system-testing script 09:55:38 I've some python knowledge with request library which I think could be enough, but honestly I haven't cose anything similar ever :p 09:55:57 William__: cool. Have you tried building a Mer platform component (like one of the buteo sync plugins) before? 09:56:12 William__: if not, either I or dcaliste can help you with that 09:56:23 No. So now is the time. The instructions are good on the wiki though, I went through them. 09:56:27 chriadam: sorry for not answer the email. Very busy lately. I hope to be more free in a week or so 09:56:36 elfio_mvl: no problems :-) 09:57:08 chriadam, thanks! I will email you if I bump into something. 09:57:23 William__: Well, hopefully the instructions work. I'll discuss this more at the end, but the SDK is going to be updated soon, and the instructions in the Wiki will be updated based on a QA run of the "new" SDK soon. 09:57:46 #agreed William__ will look into MER#1647 09:58:07 actually, that's a nice segue into our next topic: 09:58:16 #topic SDK Issues - Roadmap For Better Platform SDK (October) 09:58:55 lbt can tell you more, but basically Sailors internally use a version of the Mer Platform SDK which is slightly different (it has packages from the internal Jolla OBS instead of the Mer OBS, for example). 09:59:35 yep - but.... I'm working hard on getting the internal build replicated on the public OBS 09:59:38 the plan is to make that one available for externals also, in the nearish future, assuming that QA doesn't find issues with that (e.g., brokenness due to packages being unavailable, or something) 10:00:51 in the meantime, I *believe* that building Buteo plugins shouldn't be adversely affected - they don't have any fancy dependencies - but just in case, the wiki instructions might be slightly different to realiyt. 10:01:13 in short, if you hit any issues, yell at me and I'll do my best to figure out what's going wrong and either fix it or tell you how to work around it. 10:01:42 as I said, building buteo plugins should be mostly simple (e.g., plenty of people have used the HADK to build them without any issues) 10:01:43 ok, good to know 10:01:50 but that's that. 10:02:05 any other questions about SDK related things? dcaliste what has your experience there been? 10:02:36 the platform sdk install is a bit clunky 10:03:04 I've tried only to build QMF related stuff currently and comm-historyd. 10:03:22 For QMF, no issues, a mb2 build works out of the box. 10:03:34 r0kk3rz: dcaliste: are you using the Mer Platform SDK, or the HADK? (or sshing into the Application Developer SDK vm?) 10:04:04 For comm-historyd, I had to cheat a bit with requires in the spec file and comment certain parts of code because dependencies were too recent for SDK. 10:04:22 I ssh into the Application Developer SDK vm 10:04:27 ah great 10:04:31 that'd be the "preferred" way, IMO. 10:04:48 I'm testing ATM for the caldav plugin, it's compiling out of the box. 10:05:10 dcaliste: can you email me with step-by-step instructions to do that (starting from a blank host window, bringing up the sdk, sshing in, entering mer-sdk-chroot, building) and I'll add those to the wiki 10:05:22 if you could, that'd be much appreciated 10:05:34 chriadam: ive used platform sdk and hadk, both are a little hazardous 10:05:50 r0kk3rz: what hazards did you found ? 10:06:03 I never had any issues either with both SDK and HADK 10:06:10 chriadam: I'll email you (in my afternoon) a step step process, so you can paste it on the wiki 10:06:17 dcaliste: no rush 10:06:19 at all 10:06:25 and tyvm 10:06:42 Well, I may forget if I postpone it later ! 10:06:50 haha, fair enough 10:07:09 ok, next topic: 10:07:21 #topic Unit Tests - How? 10:07:44 Before contributors can meaningfully contribute unit tests, there probably needs to be some framework set up to allow that easily 10:08:11 e.g., using QtTest framework 10:08:28 some refactoring may be needed to allow the class methods to be called from the unit test 10:08:59 some work is required here. If someone from community wants to look at that, they're more than welcome, otherwise I plan on doing that in the next couple of weeks some time. 10:09:34 once we have a unit test framework, then community members can start to contribute self-contained tests, and XML test data to use in unit tests. But until then, it's difficult. 10:09:51 Thanks chriadam for doing all of this with community. Sorry, I ned to leave. 10:09:56 so, my opinion is: unless someone is super keen to dive in with that stuff now, probably best to wait until I do the setup 10:09:59 dcaliste: no problem, thank you 10:10:25 OK, that's all I had for the agenda. Any other business / topics / questions? 10:10:29 #topic Any Other Business ? 10:11:15 hello, I am too late, sorry. I believe there is problem with old .vcs export in mer-kcalcore, which I was trying to fix. It would be good to not use old .vcs as exchange format on Jolla. 10:12:08 vam52: Hi, yep I completely agree. I forgot to mention in the email to you, but we should use the newer KCalCore::ICalFormat to create the interchange file 10:12:36 there's code to do that in the caldav plugin, for example, but we don't use that in the jolla-calendar transfer stuff... but we should, yes. 10:13:06 #info vam52 mentioned that the interchange format used for Bluetooth transfers / Sharing of Calendar Events is vCalendar not iCalendar - should update 10:13:30 vam52: I'll add a note to that bug about that - if you're willing to see if that works? 10:14:13 ok, I'll summarise the meeting and then close it, it's been a long one, thanks for sticking with it ;-) 10:14:17 actually I believe I fixed vCal export and it should work now 10:14:17 #topic Summary 10:14:31 vam52: oh! ok, great! I'll talk to you more about this after the meeting 10:14:49 1) Early next week we'll discuss with the volunteers to set up the server instances 10:15:01 2) By the end of September we should have one test server instance running 10:15:07 3) dcaliste will take MER#1623 10:15:14 4) William__ will take MER#1647 10:15:46 4) An updated platform SDK is in the works, so beware some wiki instructions may not work currently. dcaliste will send chriadam instructions for using the Application SDK to build buteo-plugins, to add to wiki 10:15:58 err 5^ 10:16:24 6) elfio will write a simple system test script to perform basic sanity checking / regression detection 10:17:03 I think that about covers the agenda topic resolutions 10:17:17 with that - thanks to everyone for coming and for helping out 10:17:21 it really is greatly appreciated 10:17:46 there are four or five other people who want to contribute who were not here also, and hopefully we can engage with them and get them concrete tasks to do, in the near future 10:17:53 (especially once we have the test servers set up) 10:18:05 #endmeeting