You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Mel Gorman 2457aec637 mm: non-atomically mark page accessed during page cache allocation where possible ... aops->write_begin may allocate a new page and make it visible only to have mark_page_accessed called almost immediately after. Once the page is visible the atomic operations are necessary which is noticable overhead when writing to an in-memory filesystem like tmpfs but should also be noticable with fast storage. The objective of the patch is to initialse the accessed information with non-atomic operations before the page is visible. The bulk of filesystems directly or indirectly use grab_cache_page_write_begin or find_or_create_page for the initial allocation of a page cache page. This patch adds an init_page_accessed() helper which behaves like the first call to mark_page_accessed() but may called before the page is visible and can be done non-atomically. The primary APIs of concern in this care are the following and are used by most filesystems. find_get_page find_lock_page find_or_create_page grab_cache_page_nowait grab_cache_page_write_begin All of them are very similar in detail to the patch creates a core helper pagecache_get_page() which takes a flags parameter that affects its behavior such as whether the page should be marked accessed or not. Then old API is preserved but is basically a thin wrapper around this core function. Each of the filesystems are then updated to avoid calling mark_page_accessed when it is known that the VM interfaces have already done the job. There is a slight snag in that the timing of the mark_page_accessed() has now changed so in rare cases it's possible a page gets to the end of the LRU as PageReferenced where as previously it might have been repromoted. This is expected to be rare but it's worth the filesystem people thinking about it in case they see a problem with the timing change. It is also the case that some filesystems may be marking pages accessed that previously did not but it makes sense that filesystems have consistent behaviour in this regard. The test case used to evaulate this is a simple dd of a large file done multiple times with the file deleted on each iterations. The size of the file is 1/10th physical memory to avoid dirty page balancing. In the async case it will be possible that the workload completes without even hitting the disk and will have variable results but highlight the impact of mark_page_accessed for async IO. The sync results are expected to be more stable. The exception is tmpfs where the normal case is for the "IO" to not hit the disk. The test machine was single socket and UMA to avoid any scheduling or NUMA artifacts. Throughput and wall times are presented for sync IO, only wall times are shown for async as the granularity reported by dd and the variability is unsuitable for comparison. As async results were variable do to writback timings, I'm only reporting the maximum figures. The sync results were stable enough to make the mean and stddev uninteresting. The performance results are reported based on a run with no profiling. Profile data is based on a separate run with oprofile running. async dd 3.15.0-rc3 3.15.0-rc3 vanilla accessed-v2 ext3 Max elapsed 13.9900 ( 0.00%) 11.5900 ( 17.16%) tmpfs Max elapsed 0.5100 ( 0.00%) 0.4900 ( 3.92%) btrfs Max elapsed 12.8100 ( 0.00%) 12.7800 ( 0.23%) ext4 Max elapsed 18.6000 ( 0.00%) 13.3400 ( 28.28%) xfs Max elapsed 12.5600 ( 0.00%) 2.0900 ( 83.36%) The XFS figure is a bit strange as it managed to avoid a worst case by sheer luck but the average figures looked reasonable. samples percentage ext3 86107 0.9783 vmlinux-3.15.0-rc4-vanilla mark_page_accessed ext3 23833 0.2710 vmlinux-3.15.0-rc4-accessed-v3r25 mark_page_accessed ext3 5036 0.0573 vmlinux-3.15.0-rc4-accessed-v3r25 init_page_accessed ext4 64566 0.8961 vmlinux-3.15.0-rc4-vanilla mark_page_accessed ext4 5322 0.0713 vmlinux-3.15.0-rc4-accessed-v3r25 mark_page_accessed ext4 2869 0.0384 vmlinux-3.15.0-rc4-accessed-v3r25 init_page_accessed xfs 62126 1.7675 vmlinux-3.15.0-rc4-vanilla mark_page_accessed xfs 1904 0.0554 vmlinux-3.15.0-rc4-accessed-v3r25 init_page_accessed xfs 103 0.0030 vmlinux-3.15.0-rc4-accessed-v3r25 mark_page_accessed btrfs 10655 0.1338 vmlinux-3.15.0-rc4-vanilla mark_page_accessed btrfs 2020 0.0273 vmlinux-3.15.0-rc4-accessed-v3r25 init_page_accessed btrfs 587 0.0079 vmlinux-3.15.0-rc4-accessed-v3r25 mark_page_accessed tmpfs 59562 3.2628 vmlinux-3.15.0-rc4-vanilla mark_page_accessed tmpfs 1210 0.0696 vmlinux-3.15.0-rc4-accessed-v3r25 init_page_accessed tmpfs 94 0.0054 vmlinux-3.15.0-rc4-accessed-v3r25 mark_page_accessed [akpm@linux-foundation.org: don't run init_page_accessed() against an uninitialised pointer] Signed-off-by: Mel Gorman <mgorman@suse.de> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Vlastimil Babka <vbabka@suse.cz> Cc: Jan Kara <jack@suse.cz> Cc: Michal Hocko <mhocko@suse.cz> Cc: Hugh Dickins <hughd@google.com> Cc: Dave Hansen <dave.hansen@intel.com> Cc: Theodore Ts'o <tytso@mit.edu> Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Rik van Riel <riel@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Tested-by: Prabhakar Lad <prabhakar.csengg@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>		11 years ago
..
tests	Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs	11 years ago
Kconfig	Btrfs: fix btrfs boot when compiled as built-in	11 years ago
Makefile	Btrfs: fix btrfs boot when compiled as built-in	11 years ago
acl.c	btrfs: remove dead code	11 years ago
async-thread.c	btrfs: fix crash in remount(thread_pool=) case	11 years ago
async-thread.h	btrfs: Add trace for btrfs_workqueue alloc/destroy	11 years ago
backref.c	Btrfs: remove transaction from send	11 years ago
backref.h	…
btrfs_inode.h	arch: Mass conversion of smp_mb__*()	11 years ago
check-integrity.c	Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs	11 years ago
check-integrity.h	block: submit_bio_wait() conversions	11 years ago
compression.c	mm + fs: prepare for non-page entries in page cache radix trees	11 years ago
compression.h	…
ctree.c	Btrfs: hold the commit_root_sem when getting the commit root during send	11 years ago
ctree.h	Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs	11 years ago
delayed-inode.c	btrfs: Cleanup the "_struct" suffix in btrfs_workequeue	11 years ago
delayed-inode.h	Btrfs: introduce the delayed inode ref deletion for the single link inode	11 years ago
delayed-ref.c	Btrfs: fix race when updating existing ref head	11 years ago
delayed-ref.h	Btrfs: attach delayed ref updates to delayed ref heads	11 years ago
dev-replace.c	Btrfs: don't flush all delalloc inodes when we doesn't get s_umount lock	11 years ago
dev-replace.h	…
dir-item.c	Btrfs: convert printk to btrfs_ and fix BTRFS prefix	11 years ago
disk-io.c	Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs	11 years ago
disk-io.h	…
export.c	…
export.h	…
extent-tree.c	Btrfs: correctly set profile flags on seqlock retry	11 years ago
extent_io.c	mm: non-atomically mark page accessed during page cache allocation where possible	11 years ago
extent_io.h	Btrfs: don't clear uptodate if the eb is under IO	11 years ago
extent_map.c	Btrfs: more efficient btrfs_drop_extent_cache	11 years ago
extent_map.h	Btrfs: more efficient btrfs_drop_extent_cache	11 years ago
file-item.c	Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs	11 years ago
file.c	mm: non-atomically mark page accessed during page cache allocation where possible	11 years ago
free-space-cache.c	Btrfs: convert printk to btrfs_ and fix BTRFS prefix	11 years ago
free-space-cache.h	…
hash.c	Btrfs: fix btrfs boot when compiled as built-in	11 years ago
hash.h	Btrfs: fix btrfs boot when compiled as built-in	11 years ago
inode-item.c	btrfs: cleanup: removed unused 'btrfs_get_inode_ref_index'	11 years ago
inode-map.c	Btrfs: fix inode caching vs tree log	11 years ago
inode-map.h	…
inode.c	arch: Mass conversion of smp_mb__*()	11 years ago
ioctl.c	Merge branch 'locking-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip into next	11 years ago
locking.c	…
locking.h	…
lzo.c	Btrfs: convert printk to btrfs_ and fix BTRFS prefix	11 years ago
math.h	…
ordered-data.c	Btrfs: split the global ordered extents mutex	11 years ago
ordered-data.h	btrfs: Cleanup the "_struct" suffix in btrfs_workequeue	11 years ago
orphan.c	btrfs: expand btrfs_find_item() to include find_orphan_item functionality	11 years ago
print-tree.c	Btrfs: don't use ram_bytes for uncompressed inline items	11 years ago
print-tree.h	…
props.c	Btrfs: add support for inode properties	11 years ago
props.h	Btrfs: add support for inode properties	11 years ago
qgroup.c	btrfs: Cleanup the "_struct" suffix in btrfs_workequeue	11 years ago
raid56.c	Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs	11 years ago
raid56.h	…
rcu-string.h	…
reada.c	btrfs: Cleanup the "_struct" suffix in btrfs_workequeue	11 years ago
relocation.c	Btrfs: do not reset last_snapshot after relocation	11 years ago
root-tree.c	btrfs: Use PTR_ERR_OR_ZERO	11 years ago
scrub.c	Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs	11 years ago
send.c	Btrfs: send, fix incorrect ref access when using extrefs	11 years ago
send.h	…
struct-funcs.c	…
super.c	Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs	11 years ago
sysfs.c	btrfs: add simple debugfs interface	11 years ago
sysfs.h	btrfs: add simple debugfs interface	11 years ago
transaction.c	Btrfs: remove transaction from send	11 years ago
transaction.h	Btrfs: remove transaction from send	11 years ago
tree-defrag.c	…
tree-log.c	Btrfs: stop joining the log transaction if sync log fails	11 years ago
tree-log.h	Btrfs: just wait or commit our own log sub-transaction	11 years ago
ulist.c	Btrfs: do not export ulist functions	11 years ago
ulist.h	Btrfs: do not export ulist functions	11 years ago
uuid-tree.c	Btrfs: convert printk to btrfs_ and fix BTRFS prefix	11 years ago
volumes.c	Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs	11 years ago
volumes.h	btrfs: Cleanup the "_struct" suffix in btrfs_workequeue	11 years ago
xattr.c	Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs	11 years ago
xattr.h	btrfs: use generic posix ACL infrastructure	11 years ago
zlib.c	Btrfs: convert printk to btrfs_ and fix BTRFS prefix	11 years ago