[FIXED] Is there a bug in Python 3.8 datetime with DST transitions?

Issue

I'm trying to convert a timezone-aware datetime in Europe/Sofia to the start of the day in Europe/Sofia, but returning the datetime in UTC. Doing this, I encountered a strange problem:

#!/usr/bin/env python

import sys
import pytz
from datetime import datetime, timedelta


def main():
    tz = pytz.timezone('Europe/Sofia')
    dt = pytz.utc.localize(datetime(year=2019, month=3, day=31, hour=14, minute=45))
    print(f'We start with the datetime in UTC: {dt}')
    dt = dt.astimezone(tz)
    print(f'Then convert it to Europe/Sofia timezone: {dt}')

    result = dt.replace(hour=0, minute=0, second=0, microsecond=0)
    print(f'Then make it the start of the day midnight: {result}')
    result = result.astimezone(pytz.utc)
    print(f'Then convert it back to UTC (the final result we want): {result}')
    print(f'But notice the time is wrong now when converted back to Europe/Sofia: {result.astimezone(tz)}')

    result = dt - timedelta(hours=dt.hour, minutes=dt.minute, seconds=dt.second, microseconds=dt.microsecond)
    print(f'Using a timedelta instead of replace has the same result: {result}')
    print(f'The time is still wrong when converted back to Europe/Sofia: {result.astimezone(pytz.utc).astimezone(tz)}')

    return 0


if __name__ == '__main__':
    sys.exit(main())

Which prints:

We start with the datetime in UTC: 2019-03-31 14:45:00+00:00
Then convert it to Europe/Sofia timezone: 2019-03-31 17:45:00+03:00
Then make it the start of the day midnight: 2019-03-31 00:00:00+03:00
Then convert it back to UTC (the final result we want): 2019-03-30 21:00:00+00:00
But notice the time is wrong now when converted back to Europe/Sofia: 2019-03-30 23:00:00+02:00
Using a timedelta instead of replace has the same result: 2019-03-31 00:00:00+03:00
The time is still wrong when converted back to Europe/Sofia: 2019-03-30 23:00:00+02:00

The bug happens when subtracting the timedelta or doing the replace(), because the result should be midnight in Europe/Sofia with +2 offset, however we're seeing an incorrect +3 offset. It should be +2 because at 3 AM on Mar 31 2019 there was a DST transition of +1 hour. [1]

Is this a bug in Python or a known limitation with a known workaround?

Solution

It's a pytz problem, not Python (3.8). It doesn't account for the DST transition when you use replace on the aware datetime object. You could do the timedelta arithmetic with the naive datetime instead, then localize and normalize.

To avoid that rather complicated procedure, an alternative might be to use the zoneinfo module (Python 3.9), which is available via backports also on 3.8:

from datetime import datetime, timedelta
from backports.zoneinfo import ZoneInfo # pip install backports.zoneinfo

tz, utc = ZoneInfo("Europe/Sofia"), ZoneInfo("UTC")

dt = datetime(year=2019, month=3, day=31, hour=14, minute=45, tzinfo=utc)
dt = dt.astimezone(tz)
result = dt.replace(hour=0, minute=0, second=0, microsecond=0)
print(f'Then make it the start of the day midnight: {result}')

result = result.astimezone(ZoneInfo("UTC"))
print(f'Then convert it back to UTC (the final result we want): {result}')
print(f'Converted back to Europe/Sofia: {result.astimezone(tz)}')

result = dt - timedelta(hours=dt.hour, minutes=dt.minute, seconds=dt.second, microseconds=dt.microsecond)
print(f'Using a timedelta instead of replace has the same result: {result}')
print(f'The time is still correct when converted back to Europe/Sofia: {result.astimezone(utc).astimezone(tz)}')

Then make it the start of the day midnight: 2019-03-31 00:00:00+02:00
Then convert it back to UTC (the final result we want): 2019-03-30 22:00:00+00:00
Converted back to Europe/Sofia: 2019-03-31 00:00:00+02:00
Using a timedelta instead of replace has the same result: 2019-03-31 00:00:00+02:00
The time is still wrong when converted back to Europe/Sofia: 2019-03-31 00:00:00+02:00

Answered By - FObersteiner