new time on page

[ruslandoga]

Changes

This PR changes how Time on Page statistic is calculated.

Before it was:

sum(duration(page)) / count(unique(page, next_page, session))

Now it is:

avg(sum(duration(page) per user) > 0)

TODOs

• filter by "active visitors" sum(duration(sessions) per user) > 0 and not "active pages" sum(duration(page)) per user > 0?

Tests

• Automated tests have been updated

Changelog

• Entry has been added to changelog

Documentation

• This change does not need a documentation update

Dark mode

• This PR does not change the UI

[ruslandoga]

New SQL:

SELECT 
  pathname,
  avg(duration) 
FROM (
  SELECT
    pathname,
    sum(next_timestamp - timestamp) AS duration
  FROM (
    SELECT
      leadInFrame(timestamp) OVER (
        PARTITION BY session_id
        ORDER BY timestamp
        ROWS BETWEEN CURRENT ROW AND 1 FOLLOWING
      ) AS next_timestamp,
      timestamp,
      pathname,
      user_id
    FROM events_v2 SAMPLE 20000000 
    WHERE -- {...site and date range filter...}
  )
  WHERE next_timestamp != 0 AND -- {...some page filter...}
  HAVING duration > 0
  GROUP BY pathname, user_id
)
GROUP BY pathname

[ruslandoga]

Previous SQL:

SELECT 
  pathname,
  sum(duration)/countIf(transition)
FROM (
  SELECT
    pathname,
    next_pathname != pathname AS transition,
    sum(next_timestamp - timestamp) AS duration
  FROM (
    SELECT
      leadInFrame(timestamp) OVER event_horizon AS next_timestamp,
      leadInFrame(pathname) OVER event_horizon AS next_pathname,
      timestamp,
      pathname,
      session_id
    FROM events_v2 SAMPLE 20000000
    WHERE -- {...site and date range filter...}
    WINDOW event_horizon AS (
      PARTITION BY session_id
      ORDER BY timestamp
      ROWS BETWEEN CURRENT ROW AND 1 FOLLOWING
    )
  )
  WHERE next_timestamp != 0 AND -- {...some page filter...}
  GROUP BY pathname, next_pathname, session_id
)
GROUP BY pathname

which tried to recreate the original neighbour SQL:

SELECT
  pathname,
  round(sum(duration)/count(case when next_pathname != pathname then 1 end))
FROM (
  SELECT
    pathname,
    next_pathname,
    sum(next_timestamp-timestamp) AS duration
  FROM (
    SELECT
      *,
      neighbor(timestamp, 1) as next_timestamp,
      neighbor(pathname, 1) as next_pathname,
      neighbor(session_id, 1) as next_session_id
    FROM (
      SELECT
        pathname,
        timestamp,
        session_id
      FROM events_v2 SAMPLE 20000000
      WHERE -- {...site and date range filter...}
      ORDER BY session_id, timestamp
    )
  )
  WHERE session_id=next_session_id AND -- {...some page filter...}
  GROUP BY pathname, next_pathname, session_id
)
GROUP BY pathname

[ruslandoga]

I think this imported_pages query might have been under-reporting time_on_page as it was dividing by visits rather than visitors which I assume is lower. The reason I'm changing it to visitors is because that's how google seems to calculate time_on_page on its dashboards and it also maps better to avg(time_on_page across sessions) which the events_v2 query does.

[ruslandoga]

The new approach supports single page visits / applications. The previous approach was dividing by zero in that case (since there were no page transitions).