15
$\begingroup$

I am writing an API to crawl a popular website, eksisozluk, kind of like Turkey's reddit, but more text based.

On this website, you can get a list of the popular posts from the path /basliklar/populer. While looking at the source, I noticed something unusual, an element that goes

...
<ul class="topic-list partial" data-timestamp="635847618951557008">
      <li> 
...

The data-timestamp attribute lists an integer, but it is unusually large compared to what we should expect from the current timestamp, 1449158148. You can also try for yourself from the link above.

I was curious, and decided to write a script to collect more data. It would download the page every few seconds, and write the value of the unusual timestamp along with the current true timestamp. It is written in Python 3:

import urllib.request
from bs4 import BeautifulSoup
from datetime import datetime

from time import sleep

unixTime = datetime(1970,1,1)

def timestampFromDatetime(dt):
    timestamp = round((dt - unixTime).total_seconds())
    return timestamp

def getTimestamp():
    url = "https://eksisozluk.com/basliklar/populer"

    page = urllib.request.urlopen(url)
    html = page.read()
    soup = BeautifulSoup(html, "html.parser")

    result = int(soup.find("ul", class_="topic-list").attrs["data-timestamp"])
    return result

for i in range(50):
    fromSite = getTimestamp()
    now = timestampFromDatetime(datetime.now())
    print(fromSite, now)
    sleep(5)

This gives me 50 points:

# Weird timestamp  Actual timestamp
635847598460202879 1449163046
635847598513998011 1449163051
635847598564898858 1449163057
635847598619779187 1449163062
635847598672122901 1449163067
635847598724154127 1449163073
635847598774759606 1449163078
635847598826951319 1449163083
635847598882122760 1449163088
635847598933678706 1449163094
635847598987122702 1449163099
635847599037905893 1449163104
635847599089941378 1449163109
635847599143685105 1449163114
635847599194481078 1449163120
635847599246516550 1449163125
635847599299020788 1449163130
635847599352747462 1449163135
635847599403247975 1449163141
635847599455595969 1449163146
635847599507787698 1449163151
635847599561809825 1449163156
635847599613997298 1449163162
635847599666184777 1449163167
635847599718372221 1449163172
635847599770403445 1449163177
635847599821719358 1449163182
635847599874067356 1449163188
635847599925946549 1449163193
635847599979465814 1449163198
635847600031965774 1449163203
635847600082677978 1449163209
635847600136340701 1449163214
635847600186905168 1449163219
635847600239102634 1449163224
635847600291450609 1449163229
635847600345090574 1449163235
635847600395677810 1449163240
635847600447713278 1449163245
635847600499748728 1449163250
635847600551784190 1449163255
635847600605090407 1449163261
635847600656965384 1449163266
635847600709465344 1449163271
635847600759926040 1449163276
635847600813371543 1449163281
635847600864153228 1449163287
635847600916344950 1449163292
635847600969933930 1449163297
635847601021808910 1449163302

The relationship between the columns appears to be linear. With c1 being the first column and c2 being the second, I plotted c2 versus c1 and fit the linear function

$$y(x) = m x + n$$

to get

enter image description here

The actual relationship is

m = 1.000000e+7
n = 6.21355315553082e+17

It is clear that the number of seconds is multiplied by 10^7 for some reason. But there is also the shift n. I wanted to understand how many seconds that shift is equal to. Since the number of seconds is multiplied with 10^7, the actual shift should be 6.21355315553082e+17 / 1e7 = 6.21355315553082e+10 seconds.

I found out using google that

6.21355315553082e+10 seconds = 1 968.99824 years

which is 15 hours short of 1969 years.

This is as far as I could come. The Unix epoch starts at 1.1.1970, which clearly plays a role somewhere. I want to find out if there is a reason behind this weirdness, whether

  • Is it intentional, for obfuscation?
  • Is it related to an integer overflow bug?
  • Is it just bad coding?

I could only guess, and I don't think I could reach the site's admins. If this insignificantest of mysteries seems attractive to you, you can take a shot.

$\endgroup$
  • $\begingroup$ is the offset always 0 or 1? $\endgroup$ – dmg Dec 3 '15 at 16:46
  • $\begingroup$ My first thought went to leap seconds. $\endgroup$ – Raystafarian Dec 3 '15 at 16:47
  • 3
    $\begingroup$ Your conversion from seconds to years probably used a year length of 365.2425 days, which is the average length of a year in the modern calendar, but not the correct length of any particular year. $\endgroup$ – Ben Voigt Dec 3 '15 at 17:38
  • $\begingroup$ Also, timezones. (The epoch could be in some timezone other than the one you expect) $\endgroup$ – user253751 Dec 4 '15 at 9:21
21
$\begingroup$

The timestamp is using 01/01/01 as it's base, instead of the UNIX epoch: [Example]http://www.epochconverter.com/epoch/seconds-days-since-year-0.php

Look down the page to Seconds Since 0001/01/01 AD to see. I believe the extra digits beyond what the above link shows are just additional precision.

$\endgroup$
  • $\begingroup$ Great, I compared using this and the difference is around 0.1 seconds. $\endgroup$ – osolmaz Dec 3 '15 at 16:58
  • $\begingroup$ I imagine that difference would be due to latency. $\endgroup$ – Irishpanda Dec 3 '15 at 17:17
  • 1
    $\begingroup$ Note that this is January 1, 1 AD according to the proleptic Gregorian calendar, not the Kalends of January, 754 A.U.C. according to the then-used Julian calendar. (There's a difference of 2 days). $\endgroup$ – Henning Makholm Dec 5 '15 at 16:59
8
$\begingroup$

The Wikipedia page for Epoch lists some commonly used epoch dates. Specifically Microsoft .NET, Go, REXX and Rata Die are listed as using January 1, AD 1 as their epoch date. Therefore I assume one of these languages was used to implement this site.

$\endgroup$
  • $\begingroup$ IIRC they are using .NET $\endgroup$ – osolmaz Dec 4 '15 at 16:30
4
$\begingroup$

They are using .NET's built-in representation of time, the DateTime structure. In particular, they are storing the internal representation in Ticks:

A single tick represents one hundred nanoseconds or one ten-millionth of a second. There are 10,000 ticks in a millisecond, or 10 million [107] ticks in a second.
The value of this property represents the number of 100-nanosecond intervals that have elapsed since 12:00:00 midnight, January 1, 0001 (0:00:00 UTC on January 1, 0001, in the Gregorian calendar), which represents DateTime.MinValue. It does not include the number of ticks that are attributable to leap seconds.

This matches with the epoch and tick size determined by the previous answerers.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.