time-intro.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(time-intro)=\n",
    "# Introduction to Time\n",
    "\n",
    "In this section, we'll introduce the tools you need to manipulate time... well, in Python at least. In this chapter, we'll cover times, dates, datetimes, time zones, and differences in datetimes.\n",
    "\n",
    "This chapter has benefitted from the [Python Data Science Handbook](https://jakevdp.github.io/PythonDataScienceHandbook/) by Jake VanderPlas, and [strftime.org](https://strftime.org/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Python's built-in datetime\n",
    "\n",
    "The datetime object is the fundamental time object in, for want of a better description, 'base' Python. It's useful to know about these before moving on to datetime operations using **pandas** (which you're far more likely to use in practice). It combines information on date *and* time, capturing as it does the year, month, day, hour, second, and microsecond. Let's import the class that deals with datetimes (whose objects are of type `datetime.datetime`) and take a look at it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from datetime import datetime\n",
    "\n",
    "now = datetime.now()\n",
    "print(now)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Most people will be more used to working with day-month-year, while some people even have month-day-year, which clearly makes no sense at all! But note datetime follows [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601), the international standard for datetimes that has year-month-day-hrs:mins:seconds, with hours in the 24 hour clock format. This is the format you should use when coding too.\n",
    "\n",
    "As ever, the excellent [**rich**](https://github.com/willmcgugan/rich) library can give us a good idea of what properties and methods are available for objects of type `datetime.datetime` via its `inspect()` method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from rich import inspect\n",
    "\n",
    "inspect(now)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that the variable we created has methods such as `year`, `month`, `day`, and so on, down to `microsecond`. When calling these methods on the `now` object we created, they will return the relevant detail. \n",
    "\n",
    "```{admonition} Exercise\n",
    "Try calling the year, month, and day functions on an instance of `datetime.now()`.\n",
    "```\n",
    "\n",
    "Note that, once created, `now` does not refresh itself: it's frozen at the time that it was made.\n",
    "\n",
    "To create a datetime using given information the command is:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "specific_datetime = datetime(2019, 11, 28)\n",
    "print(specific_datetime)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To make clearer and more readable code, you can also call this using keyword arguments: `datetime(year=2019, month=11, day=28)`. Many of the operations you'd expect to just work with datetimes, do for example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "now > specific_datetime"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Datetimes and strings\n",
    "\n",
    "One of the most common transformations you're likely to need to do when it comes to times is the one from a string, like \"4 July 2002\", to a datetime. You can do this using `datetime.strptime()`. Here's an example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "date_string = \"16 February in 2002\"\n",
    "datetime.strptime(date_string, \"%d %B in %Y\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What's going on? The pattern of the datestring is \"day month 'in' year\". Python's `strptime` function has codes for the different parts of a datetime (and the different ways they can be expressed). For example, if you had the short version of month instead of the long it would be:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "date_string = \"16 Feb in 2002\"\n",
    "datetime.strptime(date_string, \"%d %b in %Y\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What about turning a datetime into a string? We can do that too, courtesy of the same codes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "now.strftime(\"%A, %m, %Y\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Of course, you don't always want to have to worry about the ins and outs of what you're passing in, and the built-in `dateutil` is here for flexible parsing of formats should you need that (explicit is better than implicit though!):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from dateutil.parser import parse\n",
    "\n",
    "date_string = \"03 Feb 02\"\n",
    "print(parse(date_string))\n",
    "date_string = \"3rd February 2002\"\n",
    "print(parse(date_string))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can find a close-to-comprehensive list of `strftime` codes at [https://strftime.org/](https://strftime.org/), but they're reproduced in the table below for convenience. \n",
    "\n",
    "| Code | Meaning | Example |\n",
    "|-|-|-|\n",
    "| %a | Weekday as locale’s abbreviated name. | Mon |\n",
    "| %A | Weekday as locale’s full name. | Monday |\n",
    "| %w | Weekday as a decimal number, where 0 is Sunday and 6 is Saturday. | 1 |\n",
    "| %d | Day of the month as a zero-padded decimal number. | 30 |\n",
    "| %-d | Day of the month as a decimal number. (Platform specific) | 30 |\n",
    "| %b | Month as locale’s abbreviated name. | Sep |\n",
    "| %B | Month as locale’s full name. | September |\n",
    "| %m | Month as a zero-padded decimal number. | 09 |\n",
    "| %-m | Month as a decimal number. (Platform specific) | 9 |\n",
    "| %y | Year without century as a zero-padded decimal number. | 13 |\n",
    "| %Y | Year with century as a decimal number. | 2013 |\n",
    "| %H | Hour (24-hour clock) as a zero-padded decimal number. | 07 |\n",
    "| %-H | Hour (24-hour clock) as a decimal number. (Platform specific) | 7 |\n",
    "| %I | Hour (12-hour clock) as a zero-padded decimal number. | 07 |\n",
    "| %-I | Hour (12-hour clock) as a decimal number. (Platform specific) | 7 |\n",
    "| %p | Locale’s equivalent of either AM or PM. | AM |\n",
    "| %M | Minute as a zero-padded decimal number. | 06 |\n",
    "| %-M | Minute as a decimal number. (Platform specific) | 6 |\n",
    "| %S | Second as a zero-padded decimal number. | 05 |\n",
    "| %-S | Second as a decimal number. (Platform specific) | 5 |\n",
    "| %f | Microsecond as a decimal number, zero-padded on the left. | 000000 |\n",
    "| %z | UTC offset in the form +HHMM or -HHMM (empty string if the the object is naive). |  |\n",
    "| %Z | Time zone name (empty string if the object is naive). |  |\n",
    "| %j | Day of the year as a zero-padded decimal number. | 273 |\n",
    "| %-j | Day of the year as a decimal number. (Platform specific) | 273 |\n",
    "| %U | Week number of the year (Sunday as the first day of the week) as a zero padded decimal number. | 39 |\n",
    "| %W | Week number of the year (Monday as the first day of the week) as a decimal number. | 39 |\n",
    "| %c | Locale’s appropriate date and time representation. | Mon Sep 30 07:06:05 2013 |\n",
    "| %x | Locale’s appropriate date representation. | 09/30/13 |\n",
    "| %X | Locale’s appropriate time representation. | 07:06:05 |\n",
    "| %% | A literal '%' character. | % |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### From time to time\n",
    "\n",
    "As well as recording a *single* datetime, there are plenty of occasions when we'll be interested in *differences* in datetimes. Let's create one and then check its type."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "time_diff = now - datetime(year=2020, month=1, day=1)\n",
    "print(time_diff)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is in the format of days, hours, minutes, seconds, and microseconds. Let's check the type, and more, with `inspect()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "inspect(time_diff)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is of type `datetime.timedelta`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### In the zone\n",
    "\n",
    "Date and time objects may be categorized as aware or naive depending on whether or not they include timezone information; an aware object can locate itself relative to other aware objects, but a naive object does not contain enough information to unambiguously locate itself relative to other date/time objects. So far we've been working with naive datetime objects.\n",
    "\n",
    "The **pytz** package can help us work with time zones. It has two main use cases: i) localise timezone-naive datetimes so that they become aware, ie have a timezone and ii) convert a datetimne in one timezone to another timezone.\n",
    "\n",
    "The default timezone for coding is UTC. ‘UTC’ is Coordinated Universal Time. It is a successor to, but distinct from, Greenwich Mean Time (GMT) and the various definitions of Universal Time. UTC is now the worldwide standard for regulating clocks and time measurement.\n",
    "\n",
    "All other timezones are defined relative to UTC, and include offsets like UTC+0800 - hours to add or subtract from UTC to derive the local time. No daylight saving time occurs in UTC, making it a useful timezone to perform date arithmetic without worrying about the confusion and ambiguities caused by daylight saving time transitions, your country changing its timezone, or mobile computers that roam through multiple timezones.\n",
    "\n",
    "Let's create a couple of time zone aware datetimes and look at their difference."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pytz\n",
    "from pytz import timezone\n",
    "\n",
    "aware = datetime(tzinfo=pytz.UTC, year=2020, month=1, day=1)\n",
    "unaware = datetime(year=2020, month=1, day=1)\n",
    "\n",
    "us_tz = timezone(\"US/Eastern\")\n",
    "us_aware = us_tz.localize(unaware)\n",
    "\n",
    "print(us_aware - aware)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So we find that there's a five hour difference between UTC and the time on the East Coast of the USA. In the above, we used the `localize()` method to make convert a naive datetime into an aware one, and we also initiated an aware datetime directly.\n",
    "\n",
    "For data where time really matters, such as some types of financial data, using timezone aware datetimes could prevent some nasty (and expensive) mistakes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```{admonition} Exercise\n",
    "Using `datetime.now()` and `localize()`, what is the time in the 'Australia/Melbourne' time zone?\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A More User-Friendly Approach to Datetimes: **arrow**\n",
    "\n",
    "While Python's standard library has near-complete date, time and timezone functionality, it's not the most user-friendly. The [**arrow**](https://arrow.readthedocs.io/en/latest/) package attempts to offer a sensible and human-friendly approach to creating, manipulating, formatting and converting dates, times and timestamps. Let's take a quick look at some of the functionality of **arrow**.\n",
    "\n",
    "Import arrow, create a datetime, and find the current datetime."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import arrow\n",
    "\n",
    "dt = arrow.get(\"2013-05-11T21:23:00\")\n",
    "print(dt)\n",
    "dt2 = arrow.now()\n",
    "dt2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use **arrow** to shift a datetime back by an hour and a day."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dt.shift(hours=-1, days=-1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Convert to a different datetime:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dt2.to(\"US/Pacific\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Give simpler, human readable datetimes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dt2.shift(hours=-1).humanize()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Vectorised Datetimes \n",
    "\n",
    "Now we come to vectorised operations on datetimes using the powerful **numpy** packages (and this is what is used by **pandas**). **numpy** has its own version of datetime, called `np.datetime64`, and it's very efficient at scale. Let's see it in action:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "date = np.array(\"2020-01-01\", dtype=np.datetime64)\n",
    "date"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The 'D' tells us that the smallest unit here is days. We can easily create a vector of dates from this object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "date + range(32)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note how the last day rolls over into the next month.\n",
    "\n",
    "If you are creating a datetime with more precision than day, **numpy** will figure it out from the input, for example this gives resolution down to seconds."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.datetime64(\"2020-01-01 09:00\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One word of warning with **numpy** and datetimes though: the more precise you go, and you can go down to femtoseconds ($10^{-15}$ seconds) if you wish, the smaller the range of dates you can hit. A popular choice of precision is `datetime64[ns]`, which can encode times from 1678 AD to 2262 AD. Working with seconds gets you 2.9$\\times 10^9$ BC to 2.9$\\times 10^9$ AD.\n",
    "\n",
    "We'll be seeing much more of **numpy** datetimes in the next chapter, on time series."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.8.13 ('codeforecon')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  },
  "vscode": {
   "interpreter": {
    "hash": "c4570b151692b3082981c89d172815ada9960dee4eb0bedb37dc10c95601d3bd"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}