Friday, 2019-11-08

rinigusmorning! I am debugging suspend on xz2. testing now work any active radios (wifi/bt/cellular/nfc off in settings)08:18
rinigusin these conditions, most of the suspend attempts fail, at rate 10 failures per second. overall time in sleep is 10-20%08:19
rinigusrather common snpshot in dmesg is at
rinigusas a last active wakeup, I got this night mostly mce_mux and usb_moded_input08:20
rinigusat the same time, phone was without any usb connections08:21
rinigushow can I check what triggered mce_mux specifically?08:21
riniguswhen looking at wakeup_source_activate using tracing, I get many hits on mce_mux, usb_moded_input, and even more frequently c440000.qcom,spmi:qcom,pmi8998@2:qpnp,fg08:23
rinigusI suspect that the latter is power supply, but I will ask 'bout it08:24
Mister_Magisterrinigus: if i could disable wakelock from that qcom thing there would be nothing to wake my phone up :P08:27
rinigusMister_Magister: I see. any tips on what's causing mce_mux? do we have some mce-based wakelock debugging available?08:28
rinigusMister_Magister: btw, is suspend working fine on pro1?08:29
Mister_Magisteri guess it's working fine08:29
Mister_Magisterrinigus: i have no idea what you are doing so nah not really08:29
T42Fatrank was added by: Fatrank08:29
Mister_Magisterbut you could ask  spiiroin about mce i believe08:30
rinigusMister_Magister, re guess: you could check that. but if it is able to stay idle on battery multiple days it should be ok08:33
rinigusspiiroin: would you mind to help with suspend debugging? in particular, how can I distinguish what caused mce_mux to appear? (see my description from todays above)08:34
Mister_Magisterrinigus: i can't really tell if battery drain is bad or battery is bad as i have prototype unit…08:35
Mister_Magisterbut it wasn't really running fast when i forgot about it it lasted 3 days i believe08:35
Mister_Magisterrinigus: battery drain was better on 3.0 than 3.1 ;-;08:35
rinigusMister_Magister: you could easily get stats on suspend using mcetools (see its help, grep suspend). if you don't use the phone, expect to get suspended time as a majority (90% or so).08:37
Mister_Magisterrinigus: currently smd_dev irq seems to wake my phone up08:38
Mister_Magisterrinigus:  for 12k uptime i have 10k suspend08:38
Mister_Magisterthat didn't change08:38
rinigusMister_Magister: then you are fine08:38
Mister_Magisterbut somehow battery drain is 2/3x worse and i'm not the only one who noticed it08:38
Mister_Magistermultiple people who use my ports and not only my ports noticed higher battery drain. Even my mom!08:39
r0kk3rzthats fairky subjective though08:47
Mister_Magisterthat's not subjective at all08:47
Mister_Magisterimagine your phone lasting 3 days instead of 3 weeks after you updated it08:47
rinigusI am not going into 3.0 vs 3.1 debate. just my logs over the year don't show any significant difference on onyx when I glance over them.08:48
Mister_Magisterand it's not only my port08:48
Mister_Magisterwell facts are facts08:48
rinigus... but tips on mce debugging are welcome08:48
Mister_Magisterr0kk3rz: yes facts08:49
r0kk3rzif its that bad it should be fairly obvious08:49
Mister_Magisterbut well it is not08:49
r0kk3rz3.1 hasnt been out that long08:50
Mister_Magistercouple months?08:50
r0kk3rzoh thats right, we're up to 3.2 now08:52
T42<adampigg> 3.2 seems much better for battery on mido09:02
Mister_Magisterwill have to update then and check09:03
rinigusspiiroin: I will look into suspend issue with more verbose logs, as set by mcetool to debug level. that should give me some overview. will get back to you when will know more11:55
T42<nitin03 %lastname%> ril is running
T42<nitin03 %lastname%> logcat returns this
jusaelros34: what do you mean by "wrong directory"? dbus packaging has had that binary in /lib/dbus-1/dbus-daemon-launch-helper since initial packaging (in 2013)?14:00
jusaheh ok servicehelper in system.conf indeed does list that as /usr/libexec14:02
T42<elros34> jusa: looks like something changed in dbus, because when I strace dbus-daemon it expects it in /usr/libexec/14:02
jusaI'd wager last dbus update has something to do with that.. I'll take a look14:02
T42<elros34> thanks, this breaks patchmanager in 3.2.014:03
jusaaha.. yep, update to 1.13.12 has changed from autotools to cmake for building and at that point --libexecdir=/%{_lib}/dbus-1 was dropped :(14:04
Mister_Magisterhi jusa how ya doin14:11
spiiroinrinigus: mce_mux is "multiplexed" wakelock that by nature is "mixed bag of beans" -> some visibility (and expect lots of noise) for ex: systemctl stop mce ; mce -T -lmce-wakelock.c:*14:17
rinigusspiiroin: yeah, as for mce_mux - I figured out that much from reading the code.14:18
rinigusspiiroin: would that mce -T ... command log into terminal?14:19
spiirointhat usb_moded_input might actually be more significant ... if there is lots of udev noise usb-moded needs to wade through -> can block suspend longer than what the intent is14:19
rinigusspiiroin: from reading usb_moded code and your docs. could it be that its reading wrong input to record usb insertion?14:20
spiiroinrinigus: yes, -T -> stderr, with slightly different output format & not subjected to journald line count limitations14:21
rinigusspiiroin: ok, then I will have to redirect it to file to ensure it will not be waking up for typing14:22
spiiroinrinigus: more like: the wakelocking in usb-moded was a quick hack to ensure that connect and/or disconnect actually get handled on - mm maybe l500d - before device suspends14:22
spiiroinso I would not be surprised if it causes problems sometimes in some devices14:23
rinigusspiiroin: so, to proceed - I will gather the logs regarding wakelocks on mce and report back. hopefully, it will tell us where to go from that.14:25
spiiroinrinigus: you might want to put usb-moded to verbose mode14:25
rinigusspiiroin: when using tracing, I had lots of hits from what appears to be fuel gauge as a wakeup_source_activate14:26
rinigusspiiroin: how to put it into verbose?14:26
rinigus(sorry for cross-posting, just wanted to mention)14:27
spiiroinrinigus: edit /lib/systemd/system/usb-moded.service -> add "-Dl" to ExecStart line14:27
spiiroin... come to think about it. I might have disabled usb-moded wakelock logging ...14:28
rinigusspiiroin: thanks! will do. will be able to get on with it tonight ...14:28
rinigusspiiroin: was it tooo verbose?14:29
spiiroinsemi botched it in some refactoring too, the define is still in usb-moded.c ... ;-/14:31
spiiroinrinigus: also: iirc usb moded uses two kinds of wakelocks: one that is explicitly obtained/released, and then ones that use timeout i.e. kernel cancels them automatically14:33
spiirointhe latter ones are used when there are dbus signals to broadcast etc -> guarantee that other ends gets them before device suspends14:33
spiirointhat is the one that might be too long14:34
rinigusspiiroin: so, I will have to recompile usb-moded with wakelock verbose in usb_moded_common14:34
rinigusspiiroin: but in my case its already going for suspend and then cancels it as a new wakelock arrives. as far as I understand the logs14:35
spiirointhat is a bit odd, I'd expect that it could happen occasionally. but constantly = ?14:36
spiiroinshould the kernel even attempt to suspend?14:36
rinigusspiiroin: its as if there are no wakelocks active and, while it suspends, some arrive. we are talking about 10 times per second, for hours14:37
rinigusspiiroin: hence my suspicion on whether usb-moded is even reading a correct place to get its data from. note that phone is idle, nothing inserted, nothing communicating14:38
rinigus... but in this respect, verbose mce may point us into correct direction14:40
rinigus... or verbose usb-moded14:40
rinigusspiiroin: ... or it could be wlan driver that didn't get suspended with rfkill (when I switch it off).14:41
rinigus(too many options at this moment)14:42
rinigusspiiroin: however, what I do know is that AOSP9 works fine. so, it should be possible for us as well14:44
spiiroinuserspace managing to insert wakelocks in mid kernel side suspend attempt... sounds unlikely - could be some kernel hiccup too14:44
spiiroinrinigus: actually, that mce_mux could be related to battery tracking - which also reads udev events similarly to what usb-moded does14:45
spiiroinenable extra verbose udev logging: printf "[BatteryUDevPropertyBlacklist]\nxxx=yyy\n" > /etc/mce/90-xxx.ini14:47
spiirointhen run mce so that they get printed out: mce -Tq -lmodules/battery-udev.c:*14:47
spiiroinrinigus: this was new port? you /might/ need to blacklist some power_supply devices if they produce false positives for mce battery tracking ->
rinigusspiiroin: thanks! will do14:49
rinigusspiiroin: yes, its a new port - sony xz2.14:50
spiiroinok, and as a clarification: blacklisting = helps with battery state evaluation, does not change the situation regarding udev wakeups / input processing14:51
* spiiroin needs to go - back online later14:51
rinigusspiiroin: thanks! I will deal with the logs later tonigh14:52
jusaelros34: could you check if patchmanager works after changing in /usr/share/dbus-1/system.conf servicehelper to /lib/dbus-1/dbus-daemon-launch-helper ? just to make sure nothing else is needed14:59
T42<elros34> I made symlink yesterday and it was ok but I will check that conf now15:05
T42<elros34> jusa: I changed system.conf and rebooted. Now it works15:14
jusaok thanks15:16
deathmistam I missing something or why aren't there SDK version 2.3.15 (SFOS 3.2) platform SDK chroot tarballs?15:46
rinigusspiiroin : here is mce log that was obtained with verbose udev logging and wakelock logging16:26
rinigusspiiroin: mce log:
rinigusspiiroin: journal:
rinigusspiiroin: now mce log is 26K lines, pasted just top with the end of the paste continuing for a while16:27
rinigusspiiroin: this phone has some smarter (or 'smarter') charging. it actually allows to suspend while charging, in sfos and aosp16:27
rinigusI do wonder what's increasing this SEQNUM in mce logs...16:29
deathmistoh and also mal: how was the internal discussion regarding nemo-qml-plugin-systemsettings? I (and others that would need such changes) definitely can't just keep using a "fix" like this indefinitely :p16:38
malsorry, haven't had time yet16:39
deathmistalright, guess I'16:43
deathmistll hold onto this then for a while, do you know anything about the platform SDK chroot for SFOS 3.2 situation?16:44
rinigusspiiroin: the logs were obtained with the stock usb-moded. I haven't recompiled it for verbosity of wakelock. let me know if its needed16:49
maldeathmist: what do you mean? the target is in targets.json16:51
maldeathmist: maybe the latest hasn't been updated16:52
spiiroinrinigus: that sequence number is just a increasing event counter. as that is usually reported as the only thing that changes -> the same udev data is sent over and over and over again -> does not sound normal17:06
rinigusspiiroin: maybe that's what it does. question is now whether we can somehow filter it out on android level?17:32
rinigusspiiroin: as far as I understand, we have 1 wakelock created by mce for each of these sequence increases.17:32
rinigusspiiroin: probably one more for usb-moded as well.17:33
rinigusspiiroin: so, if we cannot filter event out, maybe its possible to avoid wakelocks?17:33
rinigus... I can also look for the code that generates this SEQNUM. maybe there is a way to stop it17:35
rinigusspiiroin: this is what's going on during one suspend attempt - which fails:
T42<DylanVanAssche> Is the build target already available on OBS?18:43
rinigus... on top - dmesg / below udevadm monitor --env. haven't made diffs yet between battery updates18:43
rinigus... during 0.1 s while suspending we have 13 udev + kernel events that were in udevadm monitor log18:50
T42<nitin03 %lastname%> any way to debug why wifi not working?19:04
T42<hacker12455> did you add the auto wlan start service?19:04
T42<nitin03 %lastname%> yes19:05
T42<hacker12455> modprobe wlan does not do anything?19:05
T42<nitin03 %lastname%> no19:05
T42<hacker12455> check dmesg for firmware loading errors maybe?19:05
T42<nitin03 %lastname%>
T42<nitin03 %lastname%> from logcat I got this19:10
T42<nitin03 %lastname%> 07-25 15:53:07.463  4252  4252 I android.hardware.wifi@1.0-service: Wifi Hal is booting up...19:10
T42<nitin03 %lastname%> 07-25 15:53:07.463  4252  4252 I ServiceManagement: Removing namespace from process name android.hardware.wifi@1.0-service to wifi@1.0-service.19:10
*** OhYash1 is now known as ohyash19:19
rinigusspiiroin: testing now without this wakelock (this and its unlock below commented out)19:41
piggzmal: any builds of qtquick.controls 2 available on sfos?20:58
rinigusspiiroin: after dropping wakelock in usb_moded and mce (modules/battery-udev.c), I am still getting lots of suspend failures. now the last wakelock is `c440000.qcom,spmi:qcom,pmi8998@2:qpnp,fg`21:49
rinigusspiiroin: with kernel tracing, the same wakelock is reported as a main one as well. in trace, there is no mce nor usb-moded wakelocks, as expected after commenting them out21:50
rinigusthat wakelock is probably from , but I will have to confirm it21:52
piggzmal: i found an error in the QML formatter!22:02
piggzit converts this:22:02
piggz        for (i = 0,len = str.length; i < len; i++) {22:02
piggz        for (i = 0len = str.length; i < len; i++) {22:03
deathmist@piggz shouldn't "i = 0," be "i = 0;" (notice the comma to semi-colon :p)22:05
piggzdeathmist: no, 2 variables are set, not just one22:05
deathmistah, I see now. I've not done something like that ever before :)22:06
deathmistpiggz: any reason you'd want to define "len" inside the for loop initialization part instead of above the for loop? I'm not sure if that is even possible as I've never seen that before anywhere22:15
piggzdeathmist: its not something id normally do, im porting an app and thats in the original code22:16
deathmistoh okay. now that I think of it, probable reason for that would be scope22:20

Generated by 2.17.1 by Marius Gedminas - find it at!