datetime 'UTCLeapSeconds', 2 second difference across leap sec boundary

I was recently made aware of the leap second capability in the new datetime class of R2014b, so I did some simple tests and did not get the expected results. Am I doing something wrong or is the leap second adjustment wrong?

1st case is two dates surrounding a June 30 that did NOT have a leap second, 2013, and using the 'UTC' for TimeZone:

>> t3 = datetime(2013,6,30,0,0,0)
t3 = 
   30-Jun-2013 00:00:00
>> t3.TimeZone = 'UTC'
t3 = 
   30-Jun-2013 00:00:00
>> t4 = datetime(2013,7,2,0,0,0)
t4 = 
   02-Jul-2013 00:00:00
>> t4.TimeZone = 'UTC'
t4 = 
   02-Jul-2013 00:00:00
>> seconds(t4-t3) - 2*86400
ans =
     0

Exactly two 86400 second days as expected. All well and good.

2nd case is two dates surrounding a June 30 that DID have a leap second, 2012, and using the 'UTCLeapSeconds' for TimeZone:

>> t1 = datetime(2012,6,30,0,0,0)
t1 = 
   30-Jun-2012 00:00:00
>> t1.TimeZone = 'UTCLeapSeconds'
t1 = 
   2012-06-30T00:00:24.000Z
>> t2 = datetime(2012,7,2,0,0,0)
t2 = 
   02-Jul-2012 00:00:00
>> t2.TimeZone = 'UTCLeapSeconds'
t2 = 
   2012-07-02T00:00:25.000Z
>> seconds(t2-t1) - 2*86400
ans =
     2

So the 24.000 and 25.000 that are showing up in the date displays match the total number of leap seconds that have been applied through those dates (e.g., see http://en.wikipedia.org/wiki/Leap_second). So that seems to match. But when I look at the difference I get 2 seconds instead of the expected 1 second across this applied leap second boundary. What am I missing here? Am I building the dates or applying the UTCLeapSeconds TimeZone incorrectly? Or what?

3rd case is two dates surrounding a June 30 that DID have a leap second, 2015, but is announced in the future, again using the 'UTCLeapSeconds' for TimeZone:

>> t5 = datetime(2015,6,30,0,0,0)
t5 = 
   30-Jun-2015 00:00:00
>> t5.TimeZone = 'UTCLeapSeconds'
t5 = 
   2015-06-30T00:00:25.000Z
>> t6 = datetime(2015,7,2,0,0,0)
t6 = 
   02-Jul-2015 00:00:00
>> t6.TimeZone = 'UTCLeapSeconds'
t6 = 
   2015-07-02T00:00:25.000Z
>> seconds(t6-t5) - 2*86400
ans =
     0

So the announced leap second is not accounted for here (obvious from looking at the 25.000 in the t6 date instead of the expected 26.000). So what is the process for keeping the datetime function up-to-date for leap second adjustments? This needs to be a table look up of some sort in the background, so how does one get/keep this updated? I didn't see anything in the doc that mentions this.

Answer 1

[Peter Perkins]

James, in the second instance, you're subtracting 2012-06-30T00:00:24.000Z from 2012-07-02T00:00:25.000Z. There really is 86401+86400+1 seconds difference, i.e. 2*86400 + 2, because of the one leap second and the 1s diffference in the times of day. I think the confusion stems from the diff between what you did and this:

>> t1 = datetime(2012,6,30,0,0,0,'TimeZone','UTCLeapSeconds')
t1 = 
   2012-06-30T00:00:00.000Z
>> t2 = datetime(2012,7,2,0,0,0,'TimeZone','UTCLeapSeconds')
t2 = 
   2012-07-02T00:00:00.000Z
>> t2 - t1
ans = 
   48:00:01

That's 48h and 1s. I will make a note to look into the conversion from "unzoned" to 'UTCLeapSeconds', and why you get those extra seconds.

In the third case, you're running 14b, which was released before the 2015 leap second was even announced. There was a patch posted in early January, and the recently-released 15a has the patch already in it. You're right, there needs to be more communication about how to update, although given the current state of affairs, it's not even clear that there will ever be any more leap seconds.

Hope this helps.

[James Tursa]

Thanks for the reply. Indeed, it looks like I am misinterpreting what is going on when converting from unzoned ('') to 'UTCLeapSeconds' and what the resulting display of the date & time means. I am not sure how to make sense of what it is doing for this conversion (I need to think about this some more), but now that I know what the display is telling me the subtraction result at least makes sense if you start with the converted numbers.

When going from unzoned to UTCLeapSeconds I would have expected the unzoned to be interpreted as whatever the new time zone is (in this case UTCLeapSeconds, i.e., just slap the new time zone onto the datetime with no time of day adjustment), or perhaps UTC, but clearly neither is the case. Interestingly, it does work this way for other time zones such as 'UTC' or 'PST' etc ... it just doesn't work this way for 'UTCLeapSeconds' for some reason (making the UTCLeapSeconds case inconsistent with the other cases IMO). E.g.,

>> t1 = datetime(2012,6,30,0,0,0)
t1 = 
   30-Jun-2012 00:00:00
>> t1.TimeZone = 'UTC'
t1 = 
   30-Jun-2012 00:00:00   % <-- Note, no time of day adjustment
>> tp = datetime(2012,6,30,0,0,0)
tp = 
   30-Jun-2012 00:00:00
>> tp.TimeZone = 'PST'
tp = 
   30-Jun-2012 00:00:00  % <-- Again, no time of day adjustment
>> hours(tp-t1)
ans =
     7   % <-- The expected result

Apparently the proper way of attaching a UTCLeapSeconds time zone to an unzoned datetime without causing a time of day adjustment is something like this instead:

>> t1 = datetime(2012,6,30,0,0,0)
t1 = 
   30-Jun-2012 00:00:00
>> t1 = datetime(datevec(t1),'TimeZone','UTCLeapSeconds')
t1 = 
   2012-06-30T00:00:00.000Z
>> t2 = datetime(2012,7,2,0,0,0)
t2 = 
   02-Jul-2012 00:00:00
>> t2 = datetime(datevec(t2),'TimeZone','UTCLeapSeconds')
t2 = 
   2012-07-02T00:00:00.000Z
>> seconds(t2-t1) - 2*86400
ans =
     1

But why should I have to do something special for this conversion when I didn't have to do anything special for the other conversions?

[Peter Perkins]

The additional 24s in this

>> t1 = datetime(2012,6,30,0,0,0)
t1 = 
   30-Jun-2012 00:00:00
>> t1.TimeZone = 'UTCLeapSeconds'
t1 = 
   2012-06-30T00:00:24.000Z

is just a bug. You should see a bug report in a couple days. Thanks for pointing it out.

It's obviously common to want to convert from one (civil) time zone to another, and not uncommon to want to convert from unzoned to one of the "no leap seconds" time zones. But it's probably unusual to want to convert UTCLeapSeconds to or from anything else, because you're probably working in some scientific context that always requires leap seconds.

[James Tursa]

" ...But it's probably unusual to want to convert UTCLeapSeconds to or from anything else ..."

Well, maybe uncommon, but from my perspective not unusual. If I am given UTC times (which are not leap second adjusted) and want to convert them to GPS times, I would think it natural to make use of a UTCLeapSeconds conversion to get the leap seconds necessary for the conversion. E.g.,

>> GPS0 = datetime(1980,1,6,0,0,0,'TimeZone','UTCLeapSeconds') % GPS time 0
GPS0 = 
   1980-01-06T00:00:00.000Z
>> t2 = datetime(2012,7,2,0,0,0,'TimeZone','UTC') % arbitrary UTC date & time
t2 = 
   02-Jul-2012 00:00:00
>> t2_GPStime = seconds(datetime(datevec(t2),'TimeZone','UTCLeapSeconds') - GPS0) % GPS time (in seconds) for t2
t2_GPStime =
     1.025222416000000e+09
>> mod(t2_GPStime,86400)
ans =
    16   % <-- matches expected leap seconds since GPS time 0

[Steve Allen]

The 24 seconds is not a bug, it is a matter of interpretation. If a computer had counted every second in the internationally recommended radio broadcast time scales since 1970 including the leap seconds then according to the POSIX rules its time would be 24 seconds ahead of the current value of UTC as of 2012 June. In contrast, if a Unix-like system that had counted all of the seconds were to use the "right" zoneinfo files from IANA tz (timezone) then those zoneinfo files would know about the 24 seconds and compensate for them when producing a formatted date/time string. The same would apply to a system clock set using LORAN-C broadcasts (if they had not been discontinued), for the Time of Convergence points in LORAN-C counted all the leap seconds.

[James Tursa]

I think the only question is the behavior of the conversion when going from unzoned ('') to 'UTCLeapSeconds' when changing the TimeZone property directly by assignment. Apparently in all other cases (e.g., 'UTC', 'PST', etc) the new TimeZone is simply applied with no time-of-day adjustment. To be consistent with this behavior, applying 'UTCLeapSeconds' should do the same thing ... simply tack on the new time zone with no time-of-day adjustment. I think that is why Peter called it a bug because currently it is not consistent with all the other cases. Sure, one can simply make a rule in the doc that this case is different. But IMO that simply confuses things and serves very little purpose.

[Peter Perkins]

James, my point about "unusual" was just that if you're getting those times from outside MATLAB, then you can create them directly as UTCLeapSeconds datetimes, and not have to create them unzoned and then convert.

Your implementation of the GPS conversion does cry out for real support in MATLAB for GPS times.

[James Tursa]

Peter: I agree with all your points. It would be a welcome addition if a 'GPS' zone capability or conversion of some sort were added.

[James Tursa]

Peter, Follow-Up question:

In a comment above I had to employ datevec to convert a 'UTC' TimeZone to a 'UTCLeapSeconds' TimeZone:

datetime(datevec(t2),'TimeZone','UTCLeapSeconds')

Why can't I just do this:

>> t2 = datetime(2012,7,2,0,0,0,'TimeZone','UTC')
t2 = 
   02-Jul-2012 00:00:00
>> datetime(t2,'TimeZone','UTCLeapSeconds')
ans = 
Unable to format datetime string using the format 'dd-MMM-uuuu HH:mm:ss': invalid ISO8601 format string.

That syntax works for other time zones, and seems like a natural way to convert a datetime to a different TimeZone. E.g.,

>> datetime(t2,'TimeZone','PST')
ans = 
   01-Jul-2012 17:00:00

But I can't do it with 'UTCLeapSeconds' for some reason.

[Peter Perkins]

James, I only just saw this, sorry for the long wait. If I understand the question correctly, the problem is that 'UTCLeapSeconds' currently requires a specific display format. You might do one of two things:

1) Call the datetime constructor to do the conversion in one line:

>> t = datetime(2012,7,2,0,0,0,'TimeZone','UTC')
t = 
   02-Jul-2012 00:00:00
>> tl = datetime(t,'Format','uuuu-MM-dd''T''HH:mm:ss.SSS''Z''','TimeZone','UTCLeapSeconds')
tl = 
   2012-07-02T00:00:00.000Z

2) Or set the properties one at a time:

>> t = datetime(2012,7,2,0,0,0,'TimeZone','UTC')
t = 
   02-Jul-2012 00:00:00
>> t.Format = 'uuuu-MM-dd''T''HH:mm:ss.SSS''Z''';
>> t.TimeZone = 'UTCLeapSeconds'
t = 
   2012-07-02T00:00:00.000Z

Hope this helps. I would think that if you were working in leap seconds, you'd pretty much start off there and stay there. I'd be interested in hearing about why you're converting back and forth. Not saying you're doing anything wrong, just trying to understand.

Answer 2

[Steve Allen]

The 24 seconds is not a time of day adjustment. The question is whether the underlying time scale has been counting all of the elapsed SI seconds in the radio broadcasts that correspond to TAI, or whether it has been ignoring some SI seconds so as to stay synchronized with calendar days of mean solar time that are counted by the face value of UTC. If the underlying time scale is counting all of the SI seconds then it must disagree with the POSIX-mandated interpretation of the face value of UTC before and after a leap second. Yes, it is inevitably confusing because there is no international standard for the IANA tz "right" timezones, but on the other hand that "right" interpretation is the only one which actually corresponds with the actual history of the seconds which have been broadcast in the internationally-recommended time signals when counted since all civil clocks had a face value of 1970-01-01T00:00:00.

[James Tursa]

"... the underlying time scale ..."

That's the point, there is no underlying time scale. It is unzoned (''). The only question is what to do with this when applying a new time zone directly by assignment. To be consistent with the behavior of all other time zones, applying 'UTCLeapSeconds' should not do a time of day adjustment.

Answer 3

[Andrew Janke]

Hi folks. I'm a software developer coming from Java/C#/Joda-Time/Noda-Time, and this behavior is confusing me.

Is there a central documentation page for how the Matlab datetime API handles leap seconds? I couldn't find it anywhere looking through the doco.

[Peter Perkins]

Andrew, leaps seconds in MATLAB are "opt in". My experience is that most people don't want their time calculations to consider them at all, and unless you do opt in, MATLAB behaves as if they never happened. You opt in by setting the TimeZone property to 'UTCLeapSeconds' (This isn't really a "time zone" in the same sense that America/New_York is, but that's how it's done). Other than that, there's nothing special.