Timers and time management in the Linux kernel. Part 6.
x86_64 related clock sources
This is sixth part of the chapter which describes timers and time management related stuff in the Linux kernel. In the previous part we saw clockevents
framework and now we will continue to dive into time management related stuff in the Linux kernel. This part will describe implementation of x86 architecture related clock sources (more about clocksource
concept you can read in the second part of this chapter).
First of all we must know what clock sources may be used at x86
architecture. It is easy to know from the sysfs or from content of the /sys/devices/system/clocksource/clocksource0/available_clocksource
. The /sys/devices/system/clocksource/clocksourceN
provides two special files to achieve this:
available_clocksource
- provides information about available clock sources in the system;current_clocksource
- provides information about currently used clock source in the system.
So, let’s look:
$ cat /sys/devices/system/clocksource/clocksource0/available_clocksource
tsc hpet acpi_pm
We can see that there are three registered clock sources in my system:
tsc
- Time Stamp Counter;hpet
- High Precision Event Timer;acpi_pm
- ACPI Power Management Timer.
Now let’s look at the second file which provides best clock source (a clock source which has the best rating in the system):
$ cat /sys/devices/system/clocksource/clocksource0/current_clocksource
tsc
For me it is Time Stamp Counter. As we may know from the second part of this chapter, which describes internals of the clocksource
framework in the Linux kernel, the best clock source in a system is a clock source with the best (highest) rating or in other words with the highest frequency.
Frequency of the ACPI power management timer is 3.579545 MHz
. Frequency of the High Precision Event Timer is at least 10 MHz
. And the frequency of the Time Stamp Counter depends on processor. For example On older processors, the Time Stamp Counter
was counting internal processor clock cycles. This means its frequency changed when the processor’s frequency scaling changed. The situation has changed for newer processors. Newer processors have an invariant Time Stamp counter
that increments at a constant rate in all operational states of processor. Actually we can get its frequency in the output of the /proc/cpuinfo
. For example for the first processor in the system:
$ cat /proc/cpuinfo
...
model name : Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz
...
And although Intel manual says that the frequency of the Time Stamp Counter
, while constant, is not necessarily the maximum qualified frequency of the processor, or the frequency given in the brand string, anyway we may see that it will be much more than frequency of the ACPI PM
timer or High Precision Event Timer
. And we can see that the clock source with the best rating or highest frequency is current in the system.
You can note that besides these three clock source, we don’t see yet another two familiar us clock sources in the output of the /sys/devices/system/clocksource/clocksource0/available_clocksource
. These clock sources are jiffy
and refined_jiffies
. We don’t see them because this filed maps only high resolution clock sources or in other words clock sources with the CLOCK_SOURCE_VALID_FOR_HRES flag.
As I already wrote above, we will consider all of these three clock sources in this part. We will consider it in order of their initialization or:
hpet
;acpi_pm
;tsc
.
We can make sure that the order is exactly like this in the output of the dmesg util:
$ dmesg | grep clocksource
[ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns
[ 0.000000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 133484882848 ns
[ 0.094369] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns
[ 0.186498] clocksource: Switched to clocksource hpet
[ 0.196827] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
[ 1.413685] tsc: Refined TSC clocksource calibration: 3999.981 MHz
[ 1.413688] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x73509721780, max_idle_ns: 881591102108 ns
[ 2.413748] clocksource: Switched to clocksource tsc
The first clock source is the High Precision Event Timer, so let’s start from it.
High Precision Event Timer
The implementation of the High Precision Event Timer
for the x86 architecture is located in the arch/x86/kernel/hpet.c source code file. Its initialization starts from the call of the hpet_enable
function. This function is called during Linux kernel initialization. If we will look into start_kernel
function from the init/main.c source code file, we will see that after the all architecture-specific stuff initialized, early console is disabled and time management subsystem already ready, call of the following function:
if (late_time_init)
late_time_init();
which does initialization of the late architecture specific timers after early jiffy counter already initialized. The definition of the late_time_init
function for the x86
architecture is located in the arch/x86/kernel/time.c source code file. It looks pretty easy:
static __init void x86_late_time_init(void)
{
x86_init.timers.timer_init();
tsc_init();
}
As we may see, it does initialization of the x86
related timer and initialization of the Time Stamp Counter
. The seconds we will see in the next paragraph, but now let’s consider the call of the x86_init.timers.timer_init
function. The timer_init
points to the hpet_time_init
function from the same source code file. We can verify this by looking on the definition of the x86_init
structure from the arch/x86/kernel/x86_init.c:
struct x86_init_ops x86_init __initdata = {
...
...
...
.timers = {
.setup_percpu_clockev = setup_boot_APIC_clock,
.timer_init = hpet_time_init,
.wallclock_init = x86_init_noop,
},
...
...
...
The hpet_time_init
function does setup of the programmable interval timer if we can not enable High Precision Event Timer
and setups default timer IRQ for the enabled timer:
void __init hpet_time_init(void)
{
if (!hpet_enable())
setup_pit_timer();
setup_default_timer_irq();
}
First of all the hpet_enable
function check we can enable High Precision Event Timer
in the system by the call of the is_hpet_capable
function and if we can, we map a virtual address space for it:
int __init hpet_enable(void)
{
if (!is_hpet_capable())
return 0;
hpet_set_mapping();
}
The is_hpet_capable
function checks that we didn’t pass hpet=disable
to the kernel command line and the hpet_address
is received from the ACPI HPET table. The hpet_set_mapping
function just maps the virtual address spaces for the timer registers:
hpet_virt_address = ioremap_nocache(hpet_address, HPET_MMAP_SIZE);
As we can read in the IA-PC HPET (High Precision Event Timers) Specification:
The timer register space is 1024 bytes
So, the HPET_MMAP_SIZE
is 1024
bytes too:
#define HPET_MMAP_SIZE 1024
After we mapped virtual space for the High Precision Event Timer
, we read HPET_ID
register to get number of the timers:
id = hpet_readl(HPET_ID);
last = (id & HPET_ID_NUMBER) >> HPET_ID_NUMBER_SHIFT;
We need to get this number to allocate correct amount of space for the General Configuration Register
of the High Precision Event Timer
:
cfg = hpet_readl(HPET_CFG);
hpet_boot_cfg = kmalloc((last + 2) * sizeof(*hpet_boot_cfg), GFP_KERNEL);
After the space is allocated for the configuration register of the High Precision Event Timer
, we allow to main counter to run, and allow timer interrupts if they are enabled by the setting of HPET_CFG_ENABLE
bit in the configuration register for all timers. In the end we just register new clock source by the call of the hpet_clocksource_register
function:
if (hpet_clocksource_register())
goto out_nohpet;
which just calls already familiar
clocksource_register_hz(&clocksource_hpet, (u32)hpet_freq);
function. Where the clocksource_hpet
is the clocksource
structure with the rating 250
(remember rating of the previous refined_jiffies
clock source was 2
), name - hpet
and read_hpet
callback for the reading of atomic counter provided by the High Precision Event Timer
:
static struct clocksource clocksource_hpet = {
.name = "hpet",
.rating = 250,
.read = read_hpet,
.mask = HPET_MASK,
.flags = CLOCK_SOURCE_IS_CONTINUOUS,
.resume = hpet_resume_counter,
.archdata = { .vclock_mode = VCLOCK_HPET },
};
After the clocksource_hpet
is registered, we can return to the hpet_time_init()
function from the arch/x86/kernel/time.c source code file. We can remember that the last step is the call of the:
setup_default_timer_irq();
function in the hpet_time_init()
. The setup_default_timer_irq
function checks existence of legacy
IRQs or in other words support for the i8259 and setups IRQ0 depends on this.
That’s all. From this moment the High Precision Event Timer clock source registered in the Linux kernel clock source
framework and may be used from generic kernel code via the read_hpet
:
static cycle_t read_hpet(struct clocksource *cs)
{
return (cycle_t)hpet_readl(HPET_COUNTER);
}
function which just reads and returns atomic counter from the Main Counter Register
.
ACPI PM timer
The seconds clock source is ACPI Power Management Timer. Implementation of this clock source is located in the drivers/clocksource/acpi_pm.c source code file and starts from the call of the init_acpi_pm_clocksource
function during fs
initcall.
If we will look at implementation of the init_acpi_pm_clocksource
function, we will see that it starts from the check of the value of pmtmr_ioport
variable:
static int __init init_acpi_pm_clocksource(void)
{
...
...
...
if (!pmtmr_ioport)
return -ENODEV;
...
...
...
This pmtmr_ioport
variable contains extended address of the Power Management Timer Control Register Block
. It gets its value in the acpi_parse_fadt
function which is defined in the arch/x86/kernel/acpi/boot.c source code file. This function parses FADT
or Fixed ACPI Description Table
ACPI table and tries to get the values of the X_PM_TMR_BLK
field which contains extended address of the Power Management Timer Control Register Block
, represented in Generic Address Structure
format:
static int __init acpi_parse_fadt(struct acpi_table_header *table)
{
#ifdef CONFIG_X86_PM_TIMER
...
...
...
pmtmr_ioport = acpi_gbl_FADT.xpm_timer_block.address;
...
...
...
#endif
return 0;
}
So, if the CONFIG_X86_PM_TIMER
Linux kernel configuration option is disabled or something going wrong in the acpi_parse_fadt
function, we can’t access the Power Management Timer
register and return from the init_acpi_pm_clocksource
. In other way, if the value of the pmtmr_ioport
variable is not zero, we check rate of this timer and register this clock source by the call of the:
clocksource_register_hz(&clocksource_acpi_pm, PMTMR_TICKS_PER_SEC);
function. After the call of the clocksource_register_hs
, the acpi_pm
clock source will be registered in the clocksource
framework of the Linux kernel:
static struct clocksource clocksource_acpi_pm = {
.name = "acpi_pm",
.rating = 200,
.read = acpi_pm_read,
.mask = (cycle_t)ACPI_PM_MASK,
.flags = CLOCK_SOURCE_IS_CONTINUOUS,
};
with the rating - 200
and the acpi_pm_read
callback to read atomic counter provided by the acpi_pm
clock source. The acpi_pm_read
function just executes read_pmtmr
function:
static cycle_t acpi_pm_read(struct clocksource *cs)
{
return (cycle_t)read_pmtmr();
}
which reads value of the Power Management Timer
register. This register has following structure:
+-------------------------------+----------------------------------+
| | |
| upper eight bits of a | running count of the |
| 32-bit power management timer | power management timer |
| | |
+-------------------------------+----------------------------------+
31 E_TMR_VAL 24 TMR_VAL 0
Address of this register is stored in the Fixed ACPI Description Table
ACPI table and we already have it in the pmtmr_ioport
. So, the implementation of the read_pmtmr
function is pretty easy:
static inline u32 read_pmtmr(void)
{
return inl(pmtmr_ioport) & ACPI_PM_MASK;
}
We just read the value of the Power Management Timer
register and mask its 24
bits.
That’s all. Now we move to the last clock source in this part - Time Stamp Counter
.
Time Stamp Counter
The third and last clock source in this part is - Time Stamp Counter clock source and its implementation is located in the arch/x86/kernel/tsc.c source code file. We already saw the x86_late_time_init
function in this part and initialization of the Time Stamp Counter starts from this place. This function calls the tsc_init()
function from the arch/x86/kernel/tsc.c source code file.
At the beginning of the tsc_init
function we can see check, which checks that a processor has support of the Time Stamp Counter
:
void __init tsc_init(void)
{
u64 lpj;
int cpu;
if (!cpu_has_tsc) {
setup_clear_cpu_cap(X86_FEATURE_TSC_DEADLINE_TIMER);
return;
}
...
...
...
The cpu_has_tsc
macro expands to the call of the cpu_has
macro:
#define cpu_has_tsc boot_cpu_has(X86_FEATURE_TSC)
#define boot_cpu_has(bit) cpu_has(&boot_cpu_data, bit)
#define cpu_has(c, bit) \
(__builtin_constant_p(bit) && REQUIRED_MASK_BIT_SET(bit) ? 1 : \
test_cpu_cap(c, bit))
which check the given bit (the X86_FEATURE_TSC_DEADLINE_TIMER
in our case) in the boot_cpu_data
array which is filled during early Linux kernel initialization. If the processor has support of the Time Stamp Counter
, we get the frequency of the Time Stamp Counter
by the call of the calibrate_tsc
function from the same source code file which tries to get frequency from the different source like Model Specific Register, calibrate over programmable interval timer and etc, after this we initialize frequency and scale factor for the all processors in the system:
tsc_khz = x86_platform.calibrate_tsc();
cpu_khz = tsc_khz;
for_each_possible_cpu(cpu) {
cyc2ns_init(cpu);
set_cyc2ns_scale(cpu_khz, cpu);
}
because only first bootstrap processor will call the tsc_init
. After this we check hat Time Stamp Counter
is not disabled:
if (tsc_disabled > 0)
return;
...
...
...
check_system_tsc_reliable();
and call the check_system_tsc_reliable
function which sets the tsc_clocksource_reliable
if bootstrap processor has the X86_FEATURE_TSC_RELIABLE
feature. Note that we went through the tsc_init
function, but did not register our clock source. Actual registration of the Time Stamp Counter
clock source occurs in the:
static int __init init_tsc_clocksource(void)
{
if (!cpu_has_tsc || tsc_disabled > 0 || !tsc_khz)
return 0;
...
...
...
if (boot_cpu_has(X86_FEATURE_TSC_RELIABLE)) {
clocksource_register_khz(&clocksource_tsc, tsc_khz);
return 0;
}
function. This function called during the device
initcall. We do it to be sure that the Time Stamp Counter
clock source will be registered after the High Precision Event Timer clock source.
After these all three clock sources will be registered in the clocksource
framework and the Time Stamp Counter
clock source will be selected as active, because it has the highest rating among other clock sources:
static struct clocksource clocksource_tsc = {
.name = "tsc",
.rating = 300,
.read = read_tsc,
.mask = CLOCKSOURCE_MASK(64),
.flags = CLOCK_SOURCE_IS_CONTINUOUS | CLOCK_SOURCE_MUST_VERIFY,
.archdata = { .vclock_mode = VCLOCK_TSC },
};
That’s all.
Conclusion
This is the end of the sixth part of the chapter that describes timers and timer management related stuff in the Linux kernel. In the previous part got acquainted with the clockevents
framework. In this part we continued to learn time management related stuff in the Linux kernel and saw a little about three different clock sources which are used in the x86 architecture. The next part will be last part of this chapter and we will see some user space related stuff, i.e. how some time related system calls implemented in the Linux kernel.
If you have questions or suggestions, feel free to ping me in twitter 0xAX, drop me email or just create issue.
Please note that English is not my first language and I am really sorry for any inconvenience. If you found any mistakes please send me PR to linux-insides.