Google Calendar sync: 409 errors when syncing from a CalDAV server to gcal and source event changes

[telotortium]

When running vdirsyncer sync to regularly sync events from a CalDAV server to a Google Calendar, and the event on the CalDAV server changes after the initial sync, I will see an exception due to a HTTP 409 from Google Calendar.

Seems very similar to #613.

• Your vdirsyncer version: 0.18.0
• Your Python version: Python 3.9
• Your operating system: macOS 10.12 x86_64

config file:

[general]
status_path = "~/.cache/vdirsyncer/status/"

[storage my_caldav_calendar]
type = "caldav"
url = "[REDACTED]"
username = "[REDACTED]"
password = "[REDACTED]"
read_only = true
item_types = ["VEVENT"]

[storage my_remote_gcal]
type = "google_calendar"
token_file = "~/.cache/vdirsyncer/my_remote_gcal.token"
client_id = "[REDACTED]"
client_secret = "[REDACTED]"
item_types = ["VEVENT"]

[pair local_to_remote]
a = "my_caldav_calendar"
b = "my_remote_gcal"

# Robert work (new)
collections = [["lark_to_gcal", "[my-local-calendar-id]", "[my-google-calendar-id]@group.calendar.google.com"]]

conflict_resolution = "a wins"
partial_sync = "ignore"

vdirsyncer -vdebug sync:

Doing conflict resolution for item [my-event-id]...
Copying (updating) item [my-event-id] to gcal_robert_work/[my-google-calendar-id]@group.calendar.google.com
debug: Already normalized: '/caldav/v2/[my-google-calendar-id]@group.calendar.google.com/events/[my-event-id].ics'
debug: ====================
debug: PUT https://apidata.googleusercontent.com/caldav/v2/[my-google-calendar-id]@group.calendar.google.com/events/[my-event-id].ics
debug: {'User-Agent': 'my-google-app-id.apps.googleusercontent.com', 'Content-Type': 'text/calendar', 'If-Match': '"63778220399"'}
debug: [ICAL-EVENT-DETAILS-REDACTED]
debug: Sending request...
debug: 409
debug: {'Pragma': 'no-cache', 'Content-Type': 'text/xml; charset=UTF-8', 'Date': 'Tue, 18 Jan 2022 21:12:10 GMT', 'Expires': 'Mon, 01 Jan 1990 00:00:00 GMT', 'Cache-Control': 'no-cache, no-store, max-age=0, must-revalidate', 'Vary': 'Origin, X-Origin, Referer', 'Content-Encoding': 'gzip', 'Server': 'ESF', 'X-XSS-Protection': '0', 'X-Frame-Options': 'SAMEORIGIN', 'X-Content-Type-Options': 'nosniff', 'Alt-Svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"', 'Transfer-Encoding': 'chunked'}
debug: b'<?xml version="1.0" encoding="UTF-8"?>\n<D:error xmlns:D="DAV:"/>\n'
error: Unknown error occurred for lark_to_gcal_robert_work/lark_to_gcal: 409 Client Error: Conflict for url: https://apidata.googleusercontent.com/caldav/v2/[my-google-calendar-id]@group.calendar.google.com/events/[my-event-id].ics
error: Use `-vdebug` to see the full traceback.
debug:   File "/Users/bytedance/.local/pipx/venvs/vdirsyncer/lib/python3.9/site-packages/vdirsyncer/cli/utils.py", line 75, in handle_cli_error
debug:     raise e
debug:   File "/Users/bytedance/.local/pipx/venvs/vdirsyncer/lib/python3.9/site-packages/vdirsyncer/sync/__init__.py", line 154, in sync
debug:     action.run(a_info, b_info, conflict_resolution, partial_sync)
debug:   File "/Users/bytedance/.local/pipx/venvs/vdirsyncer/lib/python3.9/site-packages/vdirsyncer/sync/__init__.py", line 276, in run
debug:     Update(new_item, b).run(a, b, conflict_resolution, partial_sync)
debug:   File "/Users/bytedance/.local/pipx/venvs/vdirsyncer/lib/python3.9/site-packages/vdirsyncer/sync/__init__.py", line 177, in run
debug:     self._run_impl(a, b)
debug:   File "/Users/bytedance/.local/pipx/venvs/vdirsyncer/lib/python3.9/site-packages/vdirsyncer/sync/__init__.py", line 229, in _run_impl
debug:     meta.etag = self.dest.storage.update(meta.href, self.item, meta.etag)
debug:   File "/Users/bytedance/.local/pipx/venvs/vdirsyncer/lib/python3.9/site-packages/vdirsyncer/storage/base.py", line 13, in inner
debug:     return f(self, *args, **kwargs)
debug:   File "/Users/bytedance/.local/pipx/venvs/vdirsyncer/lib/python3.9/site-packages/vdirsyncer/storage/base.py", line 13, in inner
debug:     return f(self, *args, **kwargs)
debug:   File "/Users/bytedance/.local/pipx/venvs/vdirsyncer/lib/python3.9/site-packages/vdirsyncer/storage/base.py", line 13, in inner
debug:     return f(self, *args, **kwargs)
debug:   File "/Users/bytedance/.local/pipx/venvs/vdirsyncer/lib/python3.9/site-packages/vdirsyncer/storage/dav.py", line 543, in update
debug:     href, etag = self._put(self._normalize_href(href), item, etag)
debug:   File "/Users/bytedance/.local/pipx/venvs/vdirsyncer/lib/python3.9/site-packages/vdirsyncer/storage/dav.py", line 516, in _put
debug:     response = self.session.request(
debug:   File "/Users/bytedance/.local/pipx/venvs/vdirsyncer/lib/python3.9/site-packages/vdirsyncer/storage/dav.py", line 397, in request
debug:     return http.request(method, url, session=self._session, **more)
debug:   File "/Users/bytedance/.local/pipx/venvs/vdirsyncer/lib/python3.9/site-packages/vdirsyncer/http.py", line 174, in request
debug:     r.raise_for_status()
debug:   File "/Users/bytedance/.local/pipx/venvs/vdirsyncer/lib/python3.9/site-packages/requests/models.py", line 960, in raise_for_status
debug:     raise HTTPError(http_error_msg, response=self)
error: 1 out of 2 tasks failed.

[WhyNotHugo]

Do you have a clear path to reproduce this? A defined set of steps to reach this 409 scenario?

409 means that there is a conflict. Would need to research further why this is happening.

[telotortium]

@WhyNotHugo No, I'm not sure of the best way to reproduce this conflict. Google Calendar also throws 403 errors for several individual events, and I'm not sure why those occur either. I would need help from someone more versed in the CalDAV format to attempt to figure it out, and it might also be a Google Calendar-specific issue.

[g0rdonL]

I also have the same issue when syncing from caldav to google calendar. I initially thought the error would show for "already synced" events, but apparently when new events are added to caldav they cant be synced while the error shows.

[g0rdonL]

my current workaround is to delete all events on google calendar everytime i sync

[marc1uk]

OK, this seems to be reproducible (at least with my combination of orage calendar / google calendar) by altering an existing, recurring event locally, then attempting to sync.

• Creating, altering, and deleting single-instance events locally, then syncing, works fine
• Creating and deleting recurring events locally, then syncing, works fine
• But altering recurring events locally gives a conflict error.
• N.B. All remote changes, including altering recurring events, sync fine

The easiest solution when a conflict is generated seems to just be to delete the locally modified event and then sync, which it seems to accept - though of course it removes the event both locally and remotely.
I think I had tried to remove locally altered events before but lost track of changes, in which case the only other option is the nuclear one, for which the steps seem to be:

rm -r ~/.vdirsyncer/status
rm -r ~/.local/share/orage/<remote_calendar_name>      # orage-specific, edit to whatever your local copy is
vdirsyncer discover <remote_calendar_name>
vdirsyncer sync

not ideal, but it's all i had previously.

Hopefully that may help narrow down what's messing this up - presumably editing the recurring event locally alters its ETag, which then does not match the remote one. But I'm not sure what's specific about recurring events altered locally that makes this different from any other kind of event modification. Perhaps @WhyNotHugo may know more...?

[shymega]

I'm seeing this after editing with khal. I think @marc1uk is onto something with ETags.

Should I post a debug log?