Charlie Harvey

Solved: Icedove/Davical CalDAV events disappear after reloading

A while back I upgraded Icedove/Thunderbird on some of my machines. Since then I have been able to add events to my calendar, but they have been disappearing on reload. It has been doing my nut in a bit, so I have written up the issue in case others hit the same problem.

tl;dr

icedove logo pic

The problem was that the new Iceowl/Lightning doesn’t like ASCII control characters in events. Searching the database for events that contained control characters and deleting those events sorted the problem out.

My setup

I use DAViCal as my CalDAV server, backed by a PostgreSQL database. It works wonderfully and this is the first problem I have hit in the 5 years or so I have been using it.

For the client I use the stock Debian version of Icedove with Iceowl. This is the Debian de-branded version of Mozilla’s Thunderbird and Lightning. Again, it is a nice bit of software.

Troubleshooting process

I started out by recreating the calendar from scratch. Then I tried deleting my other calendars. No dice. I googled and duckduckgo-ed various permutations of CALDAV event uploading problem. Still nothing. Adding Lightning and DAViCal to my query again with no success.

Next I looked at the DAViCal server logs. Weirdly these showed the events being created as expected. Checking the database revealed that the events did indeed exist. Crazy.

Since this wasn’t an obvious problem, I looked in the Icedove error console to see if there were any clues there. Here I found an error that was reported twice: Error: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [nsISAXXMLReader.onDataAvailable]" nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame :: resource://calendar/modules/calUtils.jsm -> file:///usr/lib/icedove/extensions/%7Bxxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx%7D/calendar-js/calDavRequestHandlers.js :: mg_onDataAvailable :: line 825" data: no] Source File: resource://calendar/modules/calUtils.jsm -> file:///usr/lib/icedove/extensions/%7Bxxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx%7D/calendar-js/calDavRequestHandlers.js

I had a look at line 825 of calDavRequestHandlers.js to see if that meant anything to me. It is located in a js function called onDataAvailable: 822 onDataAvailable: function mg_onDataAvailable(request, context, inputStr eam, offset, count) { 823 if (this.reader) { 824 // No reader means request error 825 this.reader.onDataAvailable(request, context, inputStream, off set, count); 826 } 827 },

I figured that it might take a while to work out how the implementation worked, so I enabled logging of calendar events in the hope that might provide a clue. That can be enabled by going to Edit Preferences, going to the general tab and toggling calendar.debug.log and calendar.debug.log.verbose to true.

Whilst it was interesting to see what CalDAV actually does, it didn't get me any further.

More duckduckgo-ing found me a page which contained a single line from RalfK that set me on the right track:

Maybe. But actually the parser is dying on an DC3-character.

A DC3-character is a device control character ofthen used to end transmissions. In the ASCII terminology with which I am familiar, it is the same as an ETX.

I could search the postgres database for it. I logged in to postgres, and used a regex search to find out if there were any events I had created with DC-3s in them. As you can see there was indeed! davical=# select count(caldav_data) from caldav_data where user_no=1003 and caldav_data ~ '\c3' ; 0

I deleted the event and now only got the NS_ERROR_UNEXPECTED once. davical=# delete from caldav_data where user_no=1003 and caldav_data ~ '\c3' ; DELETE 1

I was a bit disappointed that that wasn’t the end of the pproblem. I figured it was likely that there was some other control character in one of my events so I started working my way through all the ASCII control characters. Not fun, but eventually productive when I found a TAB in an event. davical=# select caldav_data from caldav_data where user_no=1003 and caldav_data ~ '\c1' ;
davical=# select caldav_data from caldav_data where user_no=1003 and caldav_data ~ '\c2' ;
davical=# select caldav_data from caldav_data where user_no=1003 and caldav_data ~ '\c4' ;
davical=# select caldav_data from caldav_data where user_no=1003 and caldav_data ~ '\c5' ;
davical=# select caldav_data from caldav_data where user_no=1003 and caldav_data ~ '\c6' ;
davical=# select caldav_data from caldav_data where user_no=1003 and caldav_data ~ '\c7' ;
davical=# select caldav_data from caldav_data where user_no=1003 and caldav_data ~ '\c8' ;
davical=# select count(caldav_data) from caldav_data where user_no=1003 and caldav_data ~ '\c9' ; 1 davical=# delete from caldav_data where user_no=1003 and caldav_data ~ '\c9' ; DELETE 1

At this point I restarted icedove and everything magically worked. Now the only thing I had to do was to delete all the events I had added with names like "work, you bastard". A story for another day perhaps…


Comments

  • Be respectful. You may want to read the comment guidelines before posting.
  • You can use Markdown syntax to format your comments. You can only use level 5 and 6 headings.
  • You can add class="your language" to code blocks to help highlight.js highlight them correctly.

Privacy note: This form will forward your IP address, user agent and referrer to the Akismet, StopForumSpam and Botscout spam filtering services. I don’t log these details. Those services will. I do log everything you type into the form. Full privacy statement.