###############################
the cost of erlang:send_after/3
###############################


When you want to do something "after N seconds" in Erlang, the best way to do it is said to be ``erlang:send_after/3`` . But how many memory does it take? How many computation cost does it take?

# Entry point

``erlang:send_after/3`` can be found as bif at [bif.tab](https://github.com/erlang/otp/blob/OTP-17.0/erts/emulator/beam/bif.tab#L203) . The entry point is at [erl_bif_timer.c](https://github.com/erlang/otp/blob/OTP-17.0/erts/emulator/beam/erl_bif_timer.c#L493) . This function just calls ``setup_bif_timer()``, which is [in the same file](https://github.com/erlang/otp/blob/OTP-17.0/erts/emulator/beam/erl_bif_timer.c#L390). This allocates ``ErtsBifTimer`` ::

	btm = (ErtsBifTimer *) erts_alloc(ERTS_ALC_T_LL_BIF_TIMER,
					 sizeof(ErtsBifTimer));


which is defined in [the same file](https://github.com/erlang/otp/blob/OTP-17.0/erts/emulator/beam/erl_bif_timer.c#L41) . This does not include such a big struct other than ``ErlTimer tm;`` ( [defined in erl_time.h](https://github.com/erlang/otp/blob/OTP-17.0/erts/emulator/beam/erl_time.h#L33) ). Given the struct size is 256B, then registering 10^6 timers costs about 256MB (re-calculate it, please!) . Needs calculation, but and, it is put into timer wheel.::

    tab_insert(btm);
    ASSERT(btm == tab_find(ref));
    btm->tm.active = 0; /* MUST be initalized */
    erts_set_timer(&btm->tm,
		  (ErlTimeoutProc) bif_timer_timeout,
		  (ErlCancelProc) bif_timer_cleanup,
		  (void *) btm,
		  timeout);


``bif_timer_timeout`` is a callback of timeout, and ``btm->tm`` is ``ErlTimer`` struct.

# Before/After setting timer

[erts_set_timer in time.c](https://github.com/erlang/otp/blob/OTP-17.0/erts/emulator/beam/time.c#L397) is very interesting. Just before setting timer, it moves clock forward by calling ``erts_deliver_time()`` . ::


  void
  erts_set_timer(ErlTimer* p, ErlTimeoutProc timeout, ErlCancelProc cancel,
	      void* arg, Uint t)
  {

    erts_deliver_time();
    erts_smp_mtx_lock(&tiw_lock);
    if (p->active) { /* XXX assert ? */
	erts_smp_mtx_unlock(&tiw_lock);
	return;
    }
    p->timeout = timeout;
    p->cancel = cancel;
    p->arg = arg;
    p->active = 1;
    insert_timer(p, t);
    erts_smp_mtx_unlock(&tiw_lock);
  #if defined(ERTS_SMP)
    if (t <= (Uint) ERTS_SHORT_TIME_T_MAX)
	erts_sys_schedule_interrupt_timed(1, (erts_short_time_t) t);
  #endif
  }


In ``insert_timer(p, t);`` actually timer is registered, but right after it the erts scheduler is interrupted by [writing "!"](https://github.com/erlang/otp/blob/OTP-17.0/erts/emulator/sys/common/erl_poll.c#L472) ``write(2)`` with into scheduler interruption pipe.

# Setting timer actually

``insert_timer`` is in [time.c](https://github.com/erlang/otp/blob/OTP-17.0/erts/emulator/beam/time.c#L347) . ``tiw`` might be read as "timer wheel" ? maybe. It's actually timer wheel, which has 65536 slots (actually ``TIW_SLOTS`` ) . Each slot is for single millisecond and keeps all timers that should be invoked in the single milliseconds. Each slot is just a linked list via pointer and generic prev/next pointer linking. ::


   /* calculate slot */
    tm = (ticks + tiw_pos) % TIW_SIZE;
    p->slot = (Uint) tm;
    p->count = (Uint) (ticks / TIW_SIZE);

    /* insert at head of list at slot */
    p->next = tiw[tm];
    p->prev = NULL;
    if (p->next != NULL)
	p->next->prev = p;
    tiw[tm] = p;


# Invoking the timer

``erts_bump_timer`` is the function where these timers are triggered. This function is called by several scheduling logic incling ``schedule()`` like [here](https://github.com/erlang/otp/blob/OTP-17.0/erts/emulator/beam/erl_process.c#L3012)::

        dt = erts_do_time_read_and_reset();
	if (dt) erts_bump_timer(dt);


``erts_bump_timer_internal`` is the core part of popping out timed out timers from the timer wheel slot [timer.c](https://github.com/erlang/otp/blob/OTP-17.0/erts/emulator/beam/time.c#L227). Popping out all timer entries to ``timeout_head`` and calls all callbacks.::

    /* Call timedout timers callbacks */
    while (timeout_head) {
	p = timeout_head;
	timeout_head = p->next;
	/* Here comes hairy use of the timer fields!
	 * They are reset without having the lock.
	 * It is assumed that no code but this will
	 * accesses any field until the ->timeout
	 * callback is called.
	 */
	p->next = NULL;
	p->prev = NULL;
	p->slot = 0;
	(*p->timeout)(p->arg);
    }


For ``erlang:send_after/3`` the callback is [bif_timer_timeout](https://github.com/erlang/otp/blob/OTP-17.0/erts/emulator/beam/erl_bif_timer.c#L302) . This function calls ``erts_queue_message``, which finally stacks the message into target process's msg_q.



# Conclusion

The memory consumption is in order of ~100Bytes per timer. The computation cost is in order of traversing the linked list, whose length is in the same order with Timers stored in each milliseconds.

Suppose 10K timers continuously stored on memory then the length of each queue is 10K / 65536 by average. This is because each timer slot can be accessed by offset.


## Note

In ``*`` nix [gettimeofday(2) is used for clock.] (https://github.com/erlang/otp/blob/OTP-17.0/erts/emulator/sys/unix/erl_unix_sys.h#L162) This clock cannot be trusted so much because it leaps back but Erlang remembers the former answer of it and if the time diff is negative, the wrong time is ignored and nothing changes.