This adds kexec() support for SH.
Signed-off-by: kogiidena <kogiidena@eggplant.ddo.jp>
Signed-off-by: Paul Mundt <lethal@linux-sh.org>
Cc: <fastboot@lists.osdl.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Semaphore to mutex conversion.
The conversion was generated via scripts, and the result was validated
automatically via a script as well.
Signed-off-by: Arjan van de Ven <arjan@infradead.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Dave Jones <davej@redhat.com>
If optimizing for size (CONFIG_CC_OPTIMIZE_FOR_SIZE), allow gcc4 compilers
to decide what to inline and what not - instead of the kernel forcing gcc
to inline all the time. This requires several places that require to be
inlined to be marked as such, previous patches in this series do that.
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Arjan van de Ven <arjan@infradead.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Arjan van de Ven <arjan@infradead.org>
Mark a number of functions as 'must inline'. The functions affected by this
patch need to be inlined because they use knowledge that their arguments are
constant so that most of the function optimizes away. At this point this
patch does not change behavior, it's for documentation only (and for future
patches in the inline series)
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Arjan van de Ven <arjan@infradead.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
This patch is the first in a series that tries to optimize the kernel in terms
of size (and thus cache behavior, both cpu and pagecache).
This first patch changes __always_inline to be a forced inline instead of the
"regular" inline it was on everything except alpha. This forced inline
matches the intention of the define better as a matter of documentation.
There is no change in behavior by this patch, since "inline" currently is
mapped to a forced inline anyway.
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Arjan van de Ven <arjan@infradead.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
No need for a file argument. If we'd really need it it's in vma->vm_file
already. gbefb and sgivwfb used to set vma->vm_file to the file argument, but
the kernel alrady did that.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Antonino Daplas <adaplas@pol.net>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
The ioctl and file arguments to ->fb_mmap are totally unused and there's not
reason a driver should need them.
Also update the ->fb_compat_ioctl prototype to be the same as ->fb_mmap.
Signed-off-by: Antonino Daplas <adaplas@pol.net>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Remove the remaining kmalloc() wrapper bits from fs/smbfs/.
Signed-off-by: Pekka Enberg <penberg@cs.helsinki.fi>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
The problem, reported in:
http://bugzilla.kernel.org/show_bug.cgi?id=5859
and by various other email messages and lkml posts is that the cpuset hook
in the oom (out of memory) code can try to take a cpuset semaphore while
holding the tasklist_lock (a spinlock).
One must not sleep while holding a spinlock.
The fix seems easy enough - move the cpuset semaphore region outside the
tasklist_lock region.
This required a few lines of mechanism to implement. The oom code where
the locking needs to be changed does not have access to the cpuset locks,
which are internal to kernel/cpuset.c only. So I provided a couple more
cpuset interface routines, available to the rest of the kernel, which
simple take and drop the lock needed here (cpusets callback_sem).
Signed-off-by: Paul Jackson <pj@sgi.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
finish_arch_switch needs to update the user cpu time as well, not just the
system cpu time. Otherwise the partial user cpu time of a process that is
stored in the lowcore will be (mis-)accounted to the next process.
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Anything that writes into a tmpfs filesystem is liable to disproportionately
decrease the available memory on a particular node. Since there's no telling
what sort of application (e.g. dd/cp/cat) might be dropping large files
there, this lets the admin choose the appropriate default behavior for their
site's situation.
Introduce a tmpfs mount option which allows specifying a memory policy and
a second option to specify the nodelist for that policy. With the default
policy, tmpfs will behave as it does today. This patch adds support for
preferred, bind, and interleave policies.
The default policy will cause pages to be added to tmpfs files on the node
which is doing the writing. Some jobs expect a single process to create
and manage the tmpfs files. This results in a node which has a
significantly reduced number of free pages.
With this patch, the administrator can specify the policy and nodes for
that policy where they would prefer allocations.
This patch was originally written by Brent Casavant and Hugh Dickins. I
added support for the bind and preferred policies and the mpol_nodelist
mount option.
Signed-off-by: Brent Casavant <bcasavan@sgi.com>
Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Robin Holt <holt@sgi.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Some people apparently run CONFIG_NUMA without CONFIG_SWAP. The migration
code currently depends on swap. This patch provides a set of inline
fallback functions so that the kernel properly compiles. However, calls to
migration functions will fail.
Signed-off-by: Christoph Lameter <clameter@sgi.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Add a new SCHED_BATCH (3) scheduling policy: such tasks are presumed
CPU-intensive, and will acquire a constant +5 priority level penalty. Such
policy is nice for workloads that are non-interactive, but which do not
want to give up their nice levels. The policy is also useful for workloads
that want a deterministic scheduling policy without interactivity causing
extra preemptions (between that workload's tasks).
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Cc: Michael Kerrisk <mtk-manpages@gmx.net>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Add driver support for a 2 port PCI IOC3-based serial card on Altix boxes:
This is a re-submission. On the original submission I was asked to
organize the code so that the MIPS ioc3 ethernet and serial parts could be
used with this driver. Stanislaw Skowronek was kind enough to provide the
shim layer for this - thanks Stanislaw. This patch includes the shim layer
and the Altix PCI ioc3 serial driver. The MIPS merged ioc3 ethernet and
serial support is forthcoming.
Signed-off-by: Patrick Gefre <pfg@sgi.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
A Christoph suggested that the /proc/devices file be converted to use the
seq_file interface. This patch does that.
I've obxerved one or two installation that had sufficiently large sans that
they overran the 4k limit on /proc/devices.
Signed-off-by: Neil Horman <nhorman@redhat.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Add software recognition for the new LSI Logic Fibre Channel controller.
Signed-off-by: Eric Moore <Eric.Moore@lsil.com>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
gcc4 generates warnings when a non-FASTCALL function pointer is assigned to a
FASTCALL one. Perhaps it has taste.
Cc: David Brownell <david-b@pacbell.net>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
This makes the SPI core and its users access transfers in the SPI message
structure as linked list not as an array, as discussed on LKML.
From: David Brownell <dbrownell@users.sourceforge.net>
Updates including doc, bugfixes to the list code, add
spi_message_add_tail(). Plus, initialize things _before_ grabbing the
locks in some cases (in case it grows more expensive). This also merges
some bitbang updates of mine that didn't yet make it into the mm tree.
Signed-off-by: Vitaly Wool <vwool@ru.mvista.com>
Signed-off-by: Dmitry Pervushin <dpervushin@gmail.com>
Signed-off-by: David Brownell <dbrownell@users.sourceforge.net>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
This was originally a driver for the ST M25P80 SPI flash. It's been
updated slightly to handle other M25P series chips.
For many of these chips, the specific type could be probed, but for now
this just requires static setup with flash_platform_data that lists the
chip type (size, format) and any default partitioning to use.
Signed-off-by: David Brownell <dbrownell@users.sourceforge.net>
Cc: Mike Lavender <mike@steroidmicros.com>
Cc: David Brownell <dbrownell@users.sourceforge.net>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
This adds a bitbanging spi master, hooking up to board/adapter-specific glue
code which knows how to set and read the signals (gpios etc).
This code kicks in after the glue code creates a platform_device with the
right platform_data. That data includes I/O loops, which will usually
come from expanding an inline function (provided in the header). One goal
is that the I/O loops should be easily optimized down to a few GPIO register
accesses, in common cases, for speed and minimized overhead.
This understands all the currently defined protocol tweaking options in the
SPI framework, and might eventually serve as as reference implementation.
- different word sizes (1..32 bits)
- differing clock rates
- SPI modes differing by CPOL (affecting chip select and I/O loops)
- SPI modes differing by CPHA (affecting I/O loops)
- delays (usecs) after transfers
- temporarily deselecting chips in mid-transfer
A lot of hardware could work with this framework, though common types of
controller can't reach peak performance without switching to a driver
structure that supports pipelining of transfers (e.g. DMA queues) and maybe
controllers (e.g. IRQ driven).
Signed-off-by: David Brownell <dbrownell@users.sourceforge.net>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
This updates the ads7864 driver to use the new "spi_driver" struct, and
includes some minor unrelated cleanup.
Signed-off-by: David Brownell <dbrownell@users.sourceforge.net>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
This includes various updates to the SPI core:
- Fixes a driver model refcount bug in spi_unregister_master() paths.
- The spi_master structures now have wrappers which help keep drivers
from needing class-level get/put for device data or for refcounts.
- Check for a few setup errors that would cause oopsing later.
- Docs say more about memory management. Highlights the use of DMA-safe
i/o buffers, and zero-initializing spi_message and such metadata.
- Provide a simple alloc/free for spi_message and its spi_transfer;
this is only one of the possible memory management policies.
Nothing to break code that already works.
Signed-off-by: David Brownell <dbrownell@users.sourceforge.net>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
This is a refresh of the "Simple SPI Framework" found in 2.6.15-rc3-mm1
which makes the following changes:
* There's now a "struct spi_driver". This increase the footprint
of the core a bit, since it now includes code to do what the driver
core was previously handling directly. Documentation and comments
were updated to match.
* spi_alloc_master() now does class_device_initialize(), so it can
at least be refcounted before spi_register_master(). To match,
spi_register_master() switched over to class_device_add().
* States explicitly that after transfer errors, spi_devices will be
deselected. We want fault recovery procedures to work the same
for all controller drivers.
* Minor tweaks: controller_data no longer points to readonly data;
prevent some potential cast-from-null bugs with container_of calls;
clarifies some existing kerneldoc,
And a few small cleanups.
Signed-off-by: David Brownell <dbrownell@users.sourceforge.net>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
This is a conversion of the AT91rm9200 DataFlash MTD driver to use the
lightweight SPI framework, and no longer be AT91-specific. It compiles
down to less than 3KBytes on ARM.
The driver allows board-specific init code to provide platform_data with
the relevant MTD partitioning information, and hotplugs.
This version has been lightly tested. Its parent at91_dataflash driver has
been pretty well banged on, although kernel.org JFFS2 dataflash support was
acting broken the last time I tried it.
Signed-off-by: David Brownell <dbrownell@users.sourceforge.net>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
This is a driver for the ADS7846 touchscreen sensor, derived from
the corgi_ts and omap_ts drivers. Key differences from those two:
- Uses the new SPI framework (minimalist version)
- <linux/spi/ads7846.h> abstracts board-specific touchscreen info
- Sysfs attributes for the temperature and voltage sensors
- Uses fewer ARM-specific IRQ primitives
The temperature and voltage sensors show up in sysfs like this:
$ pwd
/sys/devices/platform/omap-uwire/spi2.0
$ ls
bus@ input:event0@ power/ temp1 vbatt
driver@ modalias temp0 vaux
$ cat modalias
ads7846
$ cat temp0
991
$ cat temp1
1177
$
So far only basic testing has been done. There's a fair amount of hardware
that uses this sensor, and which also runs Linux, which should eventually
be able to use this driver.
One portability note may be of special interest. It turns out that not all
SPI controllers are happy issuing requests that do things like "write 8 bit
command, read 12 bit response". Most of them seem happy to handle various
word sizes, so the issue isn't "12 bit response" but rather "different rx
and tx write sizes", despite that being a common MicroWire convention. So
this version of the driver no longer reads 12 bit native-endian words; it
reads 16-bit big-endian responses, then byteswaps them and shifts the
results to discard the noise.
Signed-off-by: David Brownell <david-b@pacbell.net>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
This is the core of a small SPI framework, implementing the model of a
queue of messages which complete asynchronously (with thin synchronous
wrappers on top).
- It's still less than 2KB of ".text" (ARM). If there's got to be a
mid-layer for something so simple, that's the right size budget. :)
- The guts use board-specific SPI device tables to build the driver
model tree. (Hardware probing is rarely an option.)
- This version of Kconfig includes no drivers. At this writing there
are two known master controller drivers (PXA/SSP, OMAP MicroWire)
and three protocol drivers (CS8415a, ADS7846, DataFlash) with LKML
mentions of other drivers in development.
- No userspace API. There are several implementations to compare.
Implement them like any other driver, and bind them with sysfs.
The changes from last version posted to LKML (on 11-Nov-2005) are minor,
and include:
- One bugfix (removes a FIXME), with the visible effect of making device
names be "spiB.C" where B is the bus number and C is the chipselect.
- The "caller provides DMA mappings" mechanism now has kerneldoc, for
DMA drivers that want to be fancy.
- Hey, the framework init can be subsys_init. Even though board init
logic fires earlier, at arch_init ... since the framework init is
for driver support, and the board init support uses static init.
- Various additional spec/doc clarifications based on discussions
with other folk. It adds a brief "thank you" at the end, for folk
who've helped nudge this framework into existence.
As I've said before, I think that "protocol tweaking" is the main support
that this driver framework will need to evolve.
From: Mark Underwood <basicmark@yahoo.com>
Update the SPI framework to remove a potential priority inversion case by
reverting to kmalloc if the pre-allocated DMA-safe buffer isn't available.
Signed-off-by: David Brownell <dbrownell@users.sourceforge.net>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
There are errors and inconsistency in the display of NIP6 strings.
ie: net/ipv6/ip6_flowlabel.c
There are errors and inconsistency in the display of NIPQUAD strings too.
ie: net/netfilter/nf_conntrack_ftp.c
This patch:
adds NIP6_FMT to kernel.h
changes all code to use NIP6_FMT
fixes net/ipv6/ip6_flowlabel.c
adds NIPQUAD_FMT to kernel.h
fixes net/netfilter/nf_conntrack_ftp.c
changes a few uses of "%u.%u.%u.%u" to NIPQUAD_FMT for symmetry to NIP6_FMT
Signed-off-by: Joe Perches <joe@perches.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
semaphore to mutex conversion.
the conversion was generated via scripts, and the result was validated
automatically via a script as well.
build and boot tested.
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Add bus_type probe, remove and shutdown methods to replace the
corresponding methods in struct device_driver. This matches
the way we handle the suspend/resume methods.
Since the bus methods override the device_driver methods, warn
if a device driver is registered whose methods will not be
called.
The long-term idea is to remove the device_driver methods entirely.
Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
On PowerPC, we want to be able to provide an AT_PLATFORM aux table
entry to userspace, so that glibc can choose optimized libraries for
the processor we're running on. Unfortunately that would be the 21st
aux table entry on powerpc, meaning that the aux table including the
terminating null entry would overflow the mm->saved_auxv[] array,
leading to userland programs segfaulting.
This increases the size of the mm->saved_auxv array to be large enough
to accommodate an AT_PLATFORM entry on powerpc.
Signed-off-by: Paul Mackerras <paulus@samba.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Add support to the proc_device_tree file for removing
and updating properties. Remove just removes the
proc file, update changes the data pointer within
the proc file. The remainder of the device-tree
changes occur elsewhere.
Signed-off-by: Dave Boutcher <sleddog@us.ibm.com>
Signed-off-by: Paul Mackerras <paulus@samba.org>
This monster-patch tries to do the best job for unifying the data
structures and backend interfaces for the three evil clones ip_tables,
ip6_tables and arp_tables. In an ideal world we would never have
allowed this kind of copy+paste programming... but well, our world
isn't (yet?) ideal.
o introduce a new x_tables module
o {ip,arp,ip6}_tables depend on this x_tables module
o registration functions for tables, matches and targets are only
wrappers around x_tables provided functions
o all matches/targets that are used from ip_tables and ip6_tables
are now implemented as xt_FOOBAR.c files and provide module aliases
to ipt_FOOBAR and ip6t_FOOBAR
o header files for xt_matches are in include/linux/netfilter/,
include/linux/netfilter_{ipv4,ipv6} contains compatibility wrappers
around the xt_FOOBAR.h headers
Based on this patchset we're going to further unify the code,
gradually getting rid of all the layer 3 specific assumptions.
Signed-off-by: Harald Welte <laforge@netfilter.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
The license header in each file now more clearly state that this
code is licensed under a dual BSD/GPL. Before this was only
evident if you looked at the MODULE_LICENSE line in core.c.
Signed-off-by: Per Liden <per.liden@nospam.ericsson.com>
Restored the old tipc_config.h to get a cleaner division between the
interfaces used by normal TIPC users and TIPC administration utilities.
Signed-off-by: Per Liden <per.liden@nospam.ericsson.com>
TIPC (Transparent Inter Process Communication) is a protocol designed for
intra cluster communication. For more information see
http://tipc.sourceforge.net
Signed-off-by: Per Liden <per.liden@nospam.ericsson.com>
Make the driver produce the string used by phy_connect and have board specific
code pass the integer mii bus id and phy device id for the specific controller
instance.
Signed-off-by: Kumar Gala <galak@kernel.crashing.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Jeff Garzik <jgarzik@pobox.com>
Add the PHY_ID_FMT macro to ensure that the format of the id string used by a
driver to match to its specific phy is consistent between the mdio_bus and the
driver.
Signed-off-by: Kumar Gala <galak@kernel.crashing.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Jeff Garzik <jgarzik@pobox.com>
We can now have the gianfar mii platform device have a proper resource for the
IO memory region for its registers. Previously we passed this information
that the platform_data structure because we couldn't handle overlapping memory
regions for platform devices.
Signed-off-by: Kumar Gala <galak@kernel.crashing.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Jeff Garzik <jgarzik@pobox.com>
There's a problem with the REQ_BLOCK_PC handling as well (bad ->data_len
handling) where it could actually complete a request ahead of time. I
suggest we just back this out for now, I will resubmit it later when I'm
fully confident in it.
This reverts commit 8672d57138
Signed-off-by: Jens Axboe <axboe@suse.de>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Adding defines for RAID10 and RAID50 levels, in preparation
of adding RAID Transport support in the mpt fusion drivers.
(BTW: IME is RAID10, and IM is RAID1).
Signed-off-by: Eric Moore <Eric.Moore@lsil.com>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
Patchset annotates arch/* uses of ->thread_info. Ones that really are about
access of thread_info of given process are simply switched to
task_thread_info(task); ones that deal with access to objects on stack are
switched to new helper - task_stack_page(). A _lot_ of the latter are
actually open-coded instances of "find where pt_regs are"; those are
consolidated into task_pt_regs(task) (many architectures actually have such
helper already).
Note that these annotations are not mandatory - any code not converted to
these helpers still works. However, they clean up a lot of places and have
actually caught a number of bugs, so converting out of tree ports would be a
good idea...
As an example of breakage caught by that stuff, see i386 pt_regs mess - we
used to have it open-coded in a bunch of places and when back in April Stas
had fixed a bug in copy_thread(), the rest had been left out of sync. That
required two followup patches (the latest - just before 2.6.15) _and_ still
had left /proc/*/stat eip field broken. Try ps -eo eip on i386 and watch the
junk...
This patch:
new helper - task_stack_page(task). Returns pointer to the memory object
containing task stack; usually thread_info of task sits in the beginning
of that object.
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
)
From: Nick Piggin <nickpiggin@yahoo.com.au>
Track the last waker CPU, and only consider wakeup-balancing if there's a
match between current waker CPU and the previous waker CPU. This ensures
that there is some correlation between two subsequent wakeup events before
we move the task. Should help random-wakeup workloads on large SMP
systems, by reducing the migration attempts by a factor of nr_cpus.
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Nick Piggin <npiggin@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
)
From: Ingo Molnar <mingo@elte.hu>
This is the latest version of the scheduler cache-hot-auto-tune patch.
The first problem was that detection time scaled with O(N^2), which is
unacceptable on larger SMP and NUMA systems. To solve this:
- I've added a 'domain distance' function, which is used to cache
measurement results. Each distance is only measured once. This means
that e.g. on NUMA distances of 0, 1 and 2 might be measured, on HT
distances 0 and 1, and on SMP distance 0 is measured. The code walks
the domain tree to determine the distance, so it automatically follows
whatever hierarchy an architecture sets up. This cuts down on the boot
time significantly and removes the O(N^2) limit. The only assumption
is that migration costs can be expressed as a function of domain
distance - this covers the overwhelming majority of existing systems,
and is a good guess even for more assymetric systems.
[ People hacking systems that have assymetries that break this
assumption (e.g. different CPU speeds) should experiment a bit with
the cpu_distance() function. Adding a ->migration_distance factor to
the domain structure would be one possible solution - but lets first
see the problem systems, if they exist at all. Lets not overdesign. ]
Another problem was that only a single cache-size was used for measuring
the cost of migration, and most architectures didnt set that variable
up. Furthermore, a single cache-size does not fit NUMA hierarchies with
L3 caches and does not fit HT setups, where different CPUs will often
have different 'effective cache sizes'. To solve this problem:
- Instead of relying on a single cache-size provided by the platform and
sticking to it, the code now auto-detects the 'effective migration
cost' between two measured CPUs, via iterating through a wide range of
cachesizes. The code searches for the maximum migration cost, which
occurs when the working set of the test-workload falls just below the
'effective cache size'. I.e. real-life optimized search is done for
the maximum migration cost, between two real CPUs.
This, amongst other things, has the positive effect hat if e.g. two
CPUs share a L2/L3 cache, a different (and accurate) migration cost
will be found than between two CPUs on the same system that dont share
any caches.
(The reliable measurement of migration costs is tricky - see the source
for details.)
Furthermore i've added various boot-time options to override/tune
migration behavior.
Firstly, there's a blanket override for autodetection:
migration_cost=1000,2000,3000
will override the depth 0/1/2 values with 1msec/2msec/3msec values.
Secondly, there's a global factor that can be used to increase (or
decrease) the autodetected values:
migration_factor=120
will increase the autodetected values by 20%. This option is useful to
tune things in a workload-dependent way - e.g. if a workload is
cache-insensitive then CPU utilization can be maximized by specifying
migration_factor=0.
I've tested the autodetection code quite extensively on x86, on 3
P3/Xeon/2MB, and the autodetected values look pretty good:
Dual Celeron (128K L2 cache):
---------------------
migration cost matrix (max_cache_size: 131072, cpu: 467 MHz):
---------------------
[00] [01]
[00]: - 1.7(1)
[01]: 1.7(1) -
---------------------
cacheflush times [2]: 0.0 (0) 1.7 (1784008)
---------------------
Here the slow memory subsystem dominates system performance, and even
though caches are small, the migration cost is 1.7 msecs.
Dual HT P4 (512K L2 cache):
---------------------
migration cost matrix (max_cache_size: 524288, cpu: 2379 MHz):
---------------------
[00] [01] [02] [03]
[00]: - 0.4(1) 0.0(0) 0.4(1)
[01]: 0.4(1) - 0.4(1) 0.0(0)
[02]: 0.0(0) 0.4(1) - 0.4(1)
[03]: 0.4(1) 0.0(0) 0.4(1) -
---------------------
cacheflush times [2]: 0.0 (33900) 0.4 (448514)
---------------------
Here it can be seen that there is no migration cost between two HT
siblings (CPU#0/2 and CPU#1/3 are separate physical CPUs). A fast memory
system makes inter-physical-CPU migration pretty cheap: 0.4 msecs.
8-way P3/Xeon [2MB L2 cache]:
---------------------
migration cost matrix (max_cache_size: 2097152, cpu: 700 MHz):
---------------------
[00] [01] [02] [03] [04] [05] [06] [07]
[00]: - 19.2(1) 19.2(1) 19.2(1) 19.2(1) 19.2(1) 19.2(1) 19.2(1)
[01]: 19.2(1) - 19.2(1) 19.2(1) 19.2(1) 19.2(1) 19.2(1) 19.2(1)
[02]: 19.2(1) 19.2(1) - 19.2(1) 19.2(1) 19.2(1) 19.2(1) 19.2(1)
[03]: 19.2(1) 19.2(1) 19.2(1) - 19.2(1) 19.2(1) 19.2(1) 19.2(1)
[04]: 19.2(1) 19.2(1) 19.2(1) 19.2(1) - 19.2(1) 19.2(1) 19.2(1)
[05]: 19.2(1) 19.2(1) 19.2(1) 19.2(1) 19.2(1) - 19.2(1) 19.2(1)
[06]: 19.2(1) 19.2(1) 19.2(1) 19.2(1) 19.2(1) 19.2(1) - 19.2(1)
[07]: 19.2(1) 19.2(1) 19.2(1) 19.2(1) 19.2(1) 19.2(1) 19.2(1) -
---------------------
cacheflush times [2]: 0.0 (0) 19.2 (19281756)
---------------------
This one has huge caches and a relatively slow memory subsystem - so the
migration cost is 19 msecs.
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Ashok Raj <ashok.raj@intel.com>
Signed-off-by: Ken Chen <kenneth.w.chen@intel.com>
Cc: <wilder@us.ibm.com>
Signed-off-by: John Hawkes <hawkes@sgi.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Miklos Szeredi