09:01:08 <chriadam> #startmeeting CalDAV/CardDAV Contributor Meeting
09:01:08 <merbot> Meeting started Mon Nov  7 09:01:08 2016 UTC.  The chair is chriadam. Information about MeetBot at http://wiki.merproject.org/wiki/Meetings.
09:01:08 <merbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
09:01:16 <chriadam> #topic Introductions
09:01:34 <chriadam> Please introduce yourself, with #info Name / who you are :-)
09:01:45 <chriadam> #info Chris Adams, sailor working on sync domain amongst other things
09:03:21 <dcaliste> #info Damien Caliste, community, trying to help when I can and learn new things
09:03:57 <tadzik> #info tadzik, community member
09:04:26 <blam_> #info Bea Lam, sailor also working in sync domain sometimes, hi :)
09:04:45 <chriadam> will wait another 5 mins for folks to join, then we'll start on agenda :-)
09:05:29 <kimmoli> #info Kimmo Lindholm, just following OOI
09:08:32 <chriadam> ok, thanks everyone, let's get started :-)
09:08:41 <chriadam> #topic Follow-up from last meeting
09:09:02 <chriadam> 1) we have unit tests for the CardDAV plugin's server-response handling
09:09:19 <chriadam> 2) we have unit tests for the CalDAV plugin's server-response handling
09:09:45 <chriadam> 3) we now have test servers up and running within the Mer infra (e.g. http://8.1.tst.merproject.org/)
09:10:12 <chriadam> Big thanks to dr_gogeta86 and dcaliste as well as lbt and larstiq for their great work on the above items
09:10:38 <lbt> #info David Greaves - mer guy and sailor (and late)
09:10:55 <chriadam> jlo_ agreed to help with triaging CalDAV and CardDAV issues on TJC, so looking forward to the influx of bug reports on mer bugzilla soon ;-)
09:11:07 <chriadam> hi lbt :-)
09:11:15 <chriadam> #topic Test servers in Mer infra - current status
09:11:36 <larstiq> chriadam: 8.1 is not the only available server
09:11:45 <larstiq> ah, missed the "e.g."
09:11:46 <larstiq> right
09:12:13 <chriadam> indeed, there should be several different versions of owncloud and also cozy
09:12:27 <chriadam> if you wish to help out with manual testing against these services please let me know
09:12:44 <chriadam> if you want to add more services, please play with it locally: clone https://git.merproject.org/dr_gogeta86/caldav-test-farm and "docker-compose up"
09:12:51 <chriadam> Instructions for adding new services are: https://sailfishos.org/wiki/CalDAV_and_CardDAV_Community_Contributions#Adding_A_New_Service
09:13:19 <chriadam> you can do that locally, and test with your own device, and then create a merge request on the git.merproject.org :-)
09:13:34 <chriadam> Going forward, we need to work on "provisioning" (e.g., restart services, with certain calendars + events pre-installed in the database), volunteers to investigate?
09:13:53 <chriadam> Other future steps include exposing a "reset to original provisioning state" button or REST API, but that's longer term goal.
09:14:00 <chriadam> For now, would be good if someone could help write scripts to do the cleanup+provisioning - which brings us to the next topic...
09:14:30 <chriadam> (but before I go there: larstiq/lbt: any comments on the server?  aside from "please don't mess with it if you haven't spoken to me yet")
09:14:39 <dcaliste> It can be possible to write something from NotebookSyncAgent::sendLocalChanges().
09:15:10 <dcaliste> I've did that in a test to send calendar data to a Radicale server from a read response.
09:15:18 <lbt> chriadam: I'd say talk to us before starting any projects to make sure the approach is agreeable
09:15:25 <dcaliste> It's quite simple, but it's compiled, not a script.
09:16:07 <chriadam> lbt: yep, makes sense - although people can try stuff on their own local docker env, and create an MR for something they think is worthwhile -- we can then review and give feedback / accept / reject based on our requirements etc
09:16:14 <chriadam> I'd assume ^
09:16:56 <chriadam> dcaliste: this is a provisioning thing?  cool - actually that's a really nice idea - let's discuss that in the next topic
09:16:56 <lbt> yep - it's mainly if they're considering something new - eg any web UI or stuff
09:17:11 <chriadam> lbt: makes sense :-)
09:17:11 <chriadam> thanks
09:17:22 <chriadam> #topic Script to clean the server content via curl PROPFIND+DELETE
09:17:37 <chriadam> So this is part of "provisioning" topic - although the opposite here - resetting back to default
09:18:24 <chriadam> as dcaliste mentioned, perhaps we could have some "predefined" code which could be triggered to do the reset or some specific provisioning
09:18:39 <William_> #info William, community newbie. Sorry, I'm late.
09:18:39 <chriadam> dcaliste: any idea how to selectively trigger the appropriate provisioning code?  env var, or ?
09:18:47 <chriadam> hi William_ :-)
09:19:04 <dr_gogeta86> Hi ... sorry to be late ...
09:19:17 <chriadam> dr_gogeta86: Hi :-)
09:19:21 <dcaliste> Currently, I don't know, I'll play with the docker env when I have time.
09:19:31 <chriadam> dcaliste: yep, makes sense.
09:19:40 <dr_gogeta86> radicale is coming ...
09:20:01 <dcaliste> The NotebookSyncAgent::sendLocalChanges() stuff can be used to add, modify or delete.
09:20:02 <chriadam> concurrently, it'd be good if someone could investigate a bash-script to do the same thing (e.g. PROPFIND+DELETE to delete everything, or PROPFIND+PUT to provision)
09:20:29 <chriadam> do we have any volunteers to investigate creating such a script?  If not, I volunteer to do so.
09:21:14 <chriadam> will wait 15 seconds for another volunteer else I'll take that one :-)
09:21:43 <dr_gogeta86> chriadam, better destroy the env in order to achive clean room
09:21:50 <dr_gogeta86> pls trackdown this https://github.com/nextcloud/calendar/issues/63
09:22:32 <chriadam> dr_gogeta86: I agree, but until we can expose such a "switch" via REST API or web ui, I think we should try to unblock ourselves for manual testing as much as possible - and a "reset to clean state" script would go a long way toward that, I think
09:23:22 <chriadam> saw that webcal calendars one... let's cover that at the end of the meeting as that's a new topic
09:23:37 <chriadam> #action chriadam to investigate PROPFIND+DELETE script to reset server state
09:23:54 <chriadam> #action dcaliste to investigate "compiled" provisioning options via sendLocalChanges() code
09:24:12 <chriadam> #topic Manual testing - volunteers?
09:24:13 <dr_gogeta86> https://francescou.github.io/docker-compose-ui/api.html
09:24:23 <dr_gogeta86> just tailor it ... and you are done
09:24:26 <chriadam> Until I've written that propfind script, probably can't have manual testing starting yet, I guess
09:24:44 <chriadam> dr_gogeta86: ooh, interesting!  lbt, thoughts?
09:25:03 <dr_gogeta86> needs securing ...
09:25:06 <chriadam> urgh
09:25:10 <chriadam> ok, longer-term thing then
09:25:13 <dr_gogeta86> and mer infra access limitations
09:25:23 <dcaliste> maybe, for manual testing, a HowTo should be added (if not yet) to the wiki.
09:25:28 <dr_gogeta86> ok
09:25:33 <dr_gogeta86> my fault dcaliste
09:25:33 <chriadam> dcaliste: good point, I will do that
09:25:52 <chriadam> #action chriadam to add some notes about manual testing with the new Mer test instances, to the wiki
09:25:53 <lbt> (back - doorbell)
09:26:33 <chriadam> ok, next topic:
09:26:40 <chriadam> #topic Any Volunteers For MER#1625 ? https://bugs.merproject.org/show_bug.cgi?id=1625
09:26:40 <merbot> Mer bug 1625 in buteo-sync-plugin-caldav "CalDAV sync can fail due to attempted PUT to read-only shared calendar" [Task,New]
09:26:45 <lbt> chriadam: thoughts : authentication  :)
09:26:45 <chriadam> actually, dcaliste already has a PR for this
09:27:03 <dcaliste> #1625 status: corrected, merge request available for review, https://git.merproject.org/mer-core/buteo-sync-plugin-caldav/merge_requests/8
09:27:04 <chriadam> lbt: indeed.  let's leave that docker api stuff for later, and not schedule it for now.
09:27:10 <lbt> +1
09:27:28 <dcaliste> I've added a commit to ignore 403 answers.
09:27:39 <dcaliste> And another one to rollback on 403 errors.
09:28:01 <chriadam> dcaliste: one thing is: how can we test this one? is there a way to add a read-only collection to the server, simply?
09:28:32 <dcaliste> Well, for Radicale, I just tuned the access settings locally to test…
09:29:06 <chriadam> makes sense.  maybe this possibility (of doing it programmatically as part of provisioning for a specific manual test) is something we can look at later on
09:29:16 <dcaliste> Indeed, I even force 403 answers for specific UID from Radical, just to test !
09:29:21 <chriadam> I will add a note about that to the bug report, but it shouldn't block us from getting that PR merged IMO
09:29:44 <dr_gogeta86> dcaliste, just open an issue on mer git with all your conf and i'll add it to docker compose
09:30:06 <chriadam> thanks!
09:30:09 <chriadam> next topic:
09:30:15 <chriadam> #topic Any Volunteers For MER#1689 ? https://bugs.merproject.org/show_bug.cgi?id=1689
09:30:15 <merbot> Mer bug 1689 in buteo-sync-plugin-caldav "CalDAV plugin can raise CredentialsNeedUpdate notification due to Captive Portal" [Task,New]
09:30:15 <dcaliste> dr_gogeta86: ok, I'll do this at the end of the week.
09:30:56 <chriadam> This is related to captive-portals.  I don't have time to look into this one at the moment, unfortunately.  Are there any volunteers for that?  It's a bit tricky, as you'll need access to the wifi gateway which requires captive portal login
09:31:13 <dr_gogeta86> chriadam, just create a proxy with captive portal
09:31:15 <dr_gogeta86> :-D
09:31:19 <dr_gogeta86> there are many
09:31:33 <chriadam> dr_gogeta86: aha, good suggestion!  thanks!
09:31:48 <kimmoli> does that work with cookies after captive portal login?
09:31:58 <dcaliste> dr_gogeta86: I'm dumb on network infrastructure, may you give some link ?
09:32:24 <dcaliste> The harder part for me would be to reproduce the conditions of the captive portal.
09:32:30 <chriadam> not sure how QNetworkAccessManager's default cookie jar works with respect to proxies
09:32:39 <dcaliste> After the correction, will look like for bug #1625.
09:32:39 <merbot> Mer bug 1625 in buteo-sync-plugin-caldav "CalDAV sync can fail due to attempted PUT to read-only shared calendar" [Task,New] https://bugs.merproject.org/show_bug.cgi?id=1625
09:32:39 <dr_gogeta86> kimmoli, usually is the default gw who switch something at l2/l3
09:32:55 <dr_gogeta86> kimmoli, sometims
09:32:57 <dr_gogeta86> *sometimes
09:33:43 <kimmoli> we tested that with maira, webview cookies needed to be stolen to a common jar.
09:33:56 <dr_gogeta86> cheap and dirty ... if you can't reach ipv4.jolla.com ... shutup!
09:34:03 <chriadam> This one is probably lower priority issue, but of course good if someone can investigate.  but if no volunteers, no problem, can leave in backlog and we will try again next meeting.
09:34:12 <dr_gogeta86> chriadam, can involve hw
09:34:30 <chriadam> dr_gogeta86: unless that host happens to be down for whatever reason... and unnecessary traffic etc... but yes that would be one possible workaround
09:34:37 <dr_gogeta86> but is this the time to tackle all wpa-enterprise issues
09:34:56 <dr_gogeta86> i know we are talking about a service ...
09:35:09 <chriadam> wpa-enterprise as a topic is more broad, we should collect the requirements for that one more cohesively
09:35:09 <dr_gogeta86> called *DAV
09:35:18 <chriadam> not part of this meeting I think
09:35:33 <dr_gogeta86> but is something coming from other fronts
09:35:50 <chriadam> let's move on.  of course if someone wants to take this task in the meantime they'r emore than welcome :-)
09:35:54 <chriadam> #topic Any Volunteers For MER#1569 ? https://bugs.merproject.org/show_bug.cgi?id=1569
09:35:54 <merbot> Mer bug 1569 in buteo-sync-plugin-caldav "CalDAV sync plugin doesn't handle shared event occurrences correctly" [Task,New]
09:36:30 <chriadam> This one is where a parent series is in one calendar (A), and it has an exception occurrence which is in calendar A but also shared to calendar B
09:36:50 <chriadam> in this case, we handle it wrongly, since when we attempt to sync calendar B, we find an exception occurrence with no "parent" series event
09:36:59 * lbt wonders whether the bugs should require a test-case and acceptance criteria?
09:37:15 <larstiq> chriadam: "exception occurence" is something like, "every week except this week"?
09:37:30 <dcaliste> I need to read documentation about shared calendar to understand the issue properly. After that I can investigate.
09:37:39 <dcaliste> If noone else volunteer.
09:37:44 <chriadam> larstiq: not necessarily, it's just a particular instance of the series with some information different.  e.g., some weekly series event is 6pm, but one instance is 7pm
09:37:44 <dcaliste> *no one
09:38:04 <larstiq> chriadam: right
09:38:22 <chriadam> dcaliste: that would be great!  no rush, this is an old bug and we need to get it right, because lots of opportunity for conflicts/lost updates if we do it wrong ;-)
09:38:37 <chriadam> #action dcaliste to investigate MER#1569
09:38:38 <chriadam> thanks
09:38:41 <chriadam> ok, moving on:
09:38:42 <dcaliste> I understand and I agree.
09:38:52 <chriadam> #topic Any Volunteers For MER#1568 ? https://bugs.merproject.org/show_bug.cgi?id=1568
09:38:52 <merbot> Mer bug 1568 in nemo-qml-plugin-calendar "Allow import of VERSION:1.0 vCalendar files" [Task,New]
09:39:05 <chriadam> This is somewhat related to MER#1623 which dcaliste previously investigated
09:39:22 <chriadam> I will volunteer to take this one, if no one else wants to volunteer :-)
09:39:53 <dcaliste> Yes, the correction imho should be in reader.cpp to add a fallback to vcalendar parser in case ical parser report issues.
09:40:16 <chriadam> dcaliste: great, makes sense, thanks
09:40:27 <chriadam> ok, waiting 10 seconds for volunteers else I will take this one
09:40:35 <dcaliste> As you prefer Chriadam, you or me, depends if you want to concentrate on something else.
09:41:07 <chriadam> dcaliste: I don't mind taking this one, you took the shared occurrence one :-)
09:41:19 <chriadam> #action chriadam to investigate MER#1568
09:41:27 <dcaliste> ok, fine with me, thanks.
09:41:32 <chriadam> #topic Any Volunteers To Te MER#1647 ? https://bugs.merproject.org/show_bug.cgi?id=1647
09:41:32 <merbot> Mer bug 1647 in buteo-sync-plugin-caldav "CalDAV sync fails due to UID specified outside of VEVENT definition" [Task,New]
09:41:36 <chriadam> actually William_ has updated this one
09:41:50 <chriadam> and once I have merged dcaliste's caldav unit testing PR, William_ can make his PR
09:41:51 <William_> Yes, this should be fixed
09:41:54 <chriadam> since he has a unit test for this case
09:42:01 <William_> exactly
09:42:14 <chriadam> thanks for doing that William_, I will ping you once I've merged that unit test one - should be tomorrow.
09:42:24 <William_> great!
09:43:11 <chriadam> William_: please feel free to look through the backlog at: https://bugs.merproject.org/buglist.cgi?quicksearch=caldav or https://bugs.merproject.org/buglist.cgi?quicksearch=carddav if you want to investigate something
09:43:16 <chriadam> new
09:43:20 <chriadam> ok, moving on:
09:43:26 <chriadam> #topic Any Other Business?
09:43:35 <chriadam> that was the last topic I had in the agenda
09:43:55 <chriadam> I know that there was an issue with WebCal (Apple-specific extension to CalDAV to allow subscriptions)
09:44:10 <lbt> chriadam: testing in general?
09:44:14 <dcaliste> #1646 status: got material to reproduce the bug locally, thanks to Benjamin sending the raw server response to me, still under investigation
09:44:41 <chriadam> great!
09:45:03 <chriadam> lbt: I haven't yet had time to really start testing against the instances in the Mer infra yet
09:45:14 <chriadam> I'm hoping to get a chance this week
09:45:33 <chriadam> oh, I forgot to mention:
09:45:54 <chriadam> elfio has started writing a bash script to perform a simple "smoke test" / system test
09:46:01 <lbt> chriadam: I was thinking about making the test suites public
09:46:14 <chriadam> lbt: ah, the manual test cases?
09:46:16 <lbt> and a way to extend them - eg to cover #1568 etc
09:46:19 <lbt> yes
09:46:32 <chriadam> lbt: good point - I will have to speak to jwalden about that
09:46:33 <lbt> and making sure that bugs like those mentioned actually get tests too
09:46:56 <lbt> eg what's the acceptance criteria for a patch to solve #1569
09:47:06 <chriadam> long term, I'm hoping that elfio's script can form the basis for these sorts of test cases going forward (i.e. where it cannot be unit tested, but requires system test style testing)
09:47:33 <chriadam> lbt: that's a very good questoin
09:47:39 * lbt thinks about fuzzing a calendar :)
09:48:16 <chriadam> we should try to codify our "definition of done" with respect to this work.  I don't want to reduce our agility (or raise the bar for contributions too high) but definitely we want fixes to include unit tests or system tests where possible.
09:48:54 <dcaliste> About TJC triaging, I've added comments in some questions tagged with 'calDAV' to ask users their log when possible. Most of the recent questions
09:48:55 <dcaliste> are related to bug #1646 IMHO but not sure.
09:48:55 <lbt> I'd be OK if a patch can't be accepted until there's a test
09:48:56 <merbot> Mer bug 1646 in buteo-sync-plugin-caldav "CalDAV sync fails due to annually-recurring event" [Task,New] https://bugs.merproject.org/show_bug.cgi?id=1646
09:49:10 <lbt> but it's ok to split the task to allow someone else to do the test bit
09:49:37 <larstiq> lbt: some leeway is needed for hard to test things
09:49:41 <chriadam> I will come up with a set of "manual tests" which must pass (or be modified if they enforce incorrec tbehaviour) for changes to be accepted.  but that will take some time for me to formulate properly.  in the meantime I think we should just do what testing seems appropriate.
09:49:46 <lbt> larstiq: true
09:50:12 <chriadam> #action chriadam to formulate a set of manual-test-cases to be used as acceptance-tests
09:50:16 <lbt> chriadam: yes - this is more about saying we need to do it - and also allows contribution in this area
09:50:29 <chriadam> I can' tgive an ETA on when I'll finish that.   and yes, contributions would definitely be appreciated
09:50:30 <lbt> it can allow non-coders to get in on it
09:50:43 <chriadam> hopefully elfio's work there will also greatly help, long term, in automating these manual tests
09:51:08 <chriadam> but yes, if non-programmers can help with the definition (and performing!) a series of manual tests, that would be greatly appreciated.
09:51:14 <chriadam> something to discuss more fully next time I think...
09:51:20 <chriadam> next meeting*
09:51:23 <dcaliste> chriadam, that's good step, having a set of tests to be always checked at each MR.
09:51:45 <chriadam> oh, that reminds me: is 0900 UTC still good for everyone, or does DST make that a bad time now?
09:52:09 <dcaliste> That particular time is still fine with me.
09:52:17 <William_> good for me
09:52:59 <lbt> wfm if my calendar reminder goes off :)
09:53:09 <chriadam> if you aren't at this meeting because this time is bad for you, please email me :-D
09:53:21 <chriadam> if you're reading this from the meeting log :-)
09:54:12 <chriadam> otherwise let's plan to have next meeting on Monday 5th December at 0900 UTC (hopefully that doesn't clash with Sailfish OS Community meeting?  I will double check and send email to mailing list anyway)
09:54:22 <chriadam> ok, anything else to cover?
09:54:25 <dcaliste> May I mention these two TJC issues ?
09:54:41 <dcaliste> - https://together.jolla.com/question/150401/zombie-appointments-return-from-the-undead/, related to Google sync and sociald
09:54:41 <dcaliste> - https://together.jolla.com/question/142197/caldav-calendars-get-lost-when-the-sync-server-isnt-available/
09:54:56 <chriadam> we definitely need to address the second one
09:55:08 <chriadam> that one was raised during a community webinar recently also
09:55:14 <chriadam> I forgot to create a MER# about that
09:55:17 <dcaliste> chriadam, should I create MER bug for them so we can discuss them next meeting ?
09:55:23 <chriadam> dcaliste: yes please
09:55:26 <chriadam> let me check the first one
09:56:12 <lbt> maybe topic for next time: does mer bz need anything changing to help this kind of work?
09:56:46 <chriadam> lbt: for me, mer bz is fine as-is.  dcaliste do you disagree?
09:57:00 <chriadam> dcaliste: I think I fixed that zombie event issue
09:57:03 <chriadam> with commit...
09:57:23 <lbt> good - no desire to tinker with working things :)
09:57:26 <dcaliste> its fine with me, tagging is enough to follow what happen on bz.
09:57:54 <chriadam> https://git.merproject.org/mer-core/buteo-sync-plugin-caldav/commit/57396878ee5c202abc70f8686202ec017e56cc2b
09:58:13 <chriadam> but more investigation/testing may be required.
09:59:00 <chriadam> and https://git.merproject.org/mer-core/buteo-sync-plugins-social/commit/e3e1715090e1cd1741cb70d9ae8535f3690764da
09:59:04 <chriadam> for sociald/google
09:59:42 <chriadam> I will check that TJC thread tomorrow
09:59:45 <chriadam> and comment on it
09:59:56 <chriadam> thanks for pointing out
09:59:58 <dcaliste> chriadam: ok, thanks for zombie issue.
10:00:05 <dcaliste> *for the
10:00:15 <chriadam> ok, if there's nothing else, will end the meeting in 5...
10:00:25 <chriadam> 4...
10:00:32 <chriadam> 3...
10:00:41 <chriadam> 2...
10:00:46 <chriadam> 1...
10:00:49 <chriadam> #endmeeting