I am getting more or less reproducible crashes from the CAPI subsystem
using the fcdsl driver:
Unable to handle kernel NULL pointer dereference at virtual address 00000010
printing eip:
c39bbca4
*pde = 00000000
Oops: 0000 [#1]
Modules linked in: netconsole capi capifs 3c59x mii fcdsl kernelcapi uhci_hcd usbcore ide_cd cdrom
CPU: 0
EIP: 0060:[<c39bbca4>] Tainted: P VLI
EFLAGS: 00010202 (2.6.16.11 #3)
EIP is at handle_minor_send+0x17a/0x241 [capi]
eax: c24abbc0 ebx: c0b4c980 ecx: 00000010 edx: 00000010
esi: c1679140 edi: c2783016 ebp: 0000c28d esp: c0327e24
ds: 007b es: 007b ss: 0068
Process swapper (pid: 0, threadinfo=c0326000 task=c02e1300)
Stack: <0>000005b4 c1679180 00000000 c28d0000 c1ce04e0 c2f69654 c221604e c1679140
c39bc19a 00000038 c20c0400 c075c560 c1f2f800 00000000 c01dc9b5 c1e96a40
c075c560 c2ed64c0 c1e96a40 c01dcd3b c2fb94e8 c075c560 c0327f00 c1e96a40
Call Trace:
[<c39bc19a>] capinc_tty_write+0xda/0xf3 [capi]
[<c01dc9b5>] ppp_sync_push+0x52/0xfe
[<c01dcd3b>] ppp_sync_send+0x1f5/0x204
[<c01d9bc1>] ppp_push+0x3e/0x9c
[<c01dacd4>] ppp_xmit_process+0x422/0x4cc
[<c01daf3f>] ppp_start_xmit+0x1c1/0x1f6
[<c0213ea5>] qdisc_restart+0xa7/0x135
[<c020b112>] dev_queue_xmit+0xba/0x19e
[<c0223f69>] ip_output+0x1eb/0x236
[<c0220907>] ip_forward+0x1c1/0x21a
[<c021fa6c>] ip_rcv+0x38e/0x3ea
[<c020b4c2>] netif_receive_skb+0x166/0x195
[<c020b55e>] process_backlog+0x6d/0xd2
[<c020a30f>] net_rx_action+0x6a/0xff
[<c0112909>] __do_softirq+0x35/0x7d
[<c0112973>] do_softirq+0x22/0x26
[<c0103a9d>] do_IRQ+0x1e/0x25
[<c010255a>] common_interrupt+0x1a/0x20
[<c01013c5>] default_idle+0x2b/0x53
[<c0101426>] cpu_idle+0x39/0x4e
[<c0328386>] start_kernel+0x20b/0x20d
Code: c0 e8 b3 b6 77 fc 85 c0 75 10 68 d8 c8 9b c3 e8 82 3d 75 fc 8b 43 60 5a eb 50 8d 56 50 c7 00 00 00 00 00 66 89 68 04 eb 02 89
ca <8b> 0a 85 c9 75 f8 89 02 89 da ff 46 54 8b 46 10 e8 30 79 fd ff
<0>Kernel panic - not syncing: Fatal exception in interrupt
That oops took me to the "ackqueue" implementation in capi.c. The crash
occured in capincci_add_ack() (auto-inlined by the compiler).
I read the code a bit and finally decided to replace the custom linked list
implementation (struct capiminor->ackqueue) by a struct list_head. That
did not solve the crash, but produced the following interresting oops:
Unable to handle kernel paging request at virtual address 00200200
printing eip:
c39bb1f5
*pde = 00000000
Oops: 0002 [#1]
Modules linked in: netconsole capi capifs 3c59x mii fcdsl kernelcapi uhci_hcd usbcore ide_cd cdrom
CPU: 0
EIP: 0060:[<c39bb1f5>] Tainted: P VLI
EFLAGS: 00010246 (2.6.16.11 #3)
EIP is at capiminor_del_ack+0x18/0x49 [capi]
eax: 00200200 ebx: c18d41a0 ecx: c1385620 edx: 00100100
esi: 0000d147 edi: 00001103 ebp: 0000d147 esp: c1093f3c
ds: 007b es: 007b ss: 0068
Process events/0 (pid: 3, threadinfo=c1092000 task=c1089030)
Stack: <0>c2a17580 c18d41a0 c39bbd16 00000038 c18d41e0 00000000 d147c640 c29e0b68
c29e0b90 00000212 c29e0b68 c39932b2 c29e0bb0 c10736a0 c0119ef0 c399326c
c10736a8 c10736a0 c10736b0 c0119f93 c011a06e 00000001 00000000 00000000
Call Trace:
[<c39bbd16>] handle_minor_send+0x1af/0x241 [capi]
[<c39932b2>] recv_handler+0x46/0x5f [kernelcapi]
[<c0119ef0>] run_workqueue+0x5e/0x8d
[<c399326c>] recv_handler+0x0/0x5f [kernelcapi]
[<c0119f93>] worker_thread+0x0/0x10b
[<c011a06e>] worker_thread+0xdb/0x10b
[<c010c998>] default_wake_function+0x0/0xc
[<c011c399>] kthread+0x90/0xbc
[<c011c309>] kthread+0x0/0xbc
[<c0100a65>] kernel_thread_helper+0x5/0xb
Code: 7e 02 89 ee 89 f0 5a f7 d0 c1 f8 1f 5b 21 f0 5e 5f 5d c3 56 53 8b 48 50 89 d6 89 c3 8b 11 eb 2f 66 39 71 08 75 25 8b 41 04 8b 11 <89> 10 89 42 04 c7 01 00 01 10 00 89 c8 c7 41 04 00 02 20 00 e8
The interresting part of it is the "virtual address 00200200", which is
LIST_POISON2. I thought about some race condition, but as this is an UP
system, it leads to questions on how it can happen. If we look at EFLAGS:
00010202, we see that interrupts are enabled at the time of the crash
(eflags & 0x200).
Finally, I don't understand all the capi code, but I think that
handle_minor_send() is racing somehow against capi_recv_message(), which
call both capiminor_del_ack(). So if an IRQ occurs in the middle of
capiminor_del_ack() and another instance of it is invoked, it leads to
linked list corruption.
I came up with the following patch. With this, I could not reproduce the
crash anymore. Clearly, this is not the correct fix for the issue. As this
seems to be some locking issue, there might be more locking issues in that
code. For example, doesn't the whole struct capiminor have to be locked
somehow?
Cc: Carsten Paeth <calle@calle.de>
Cc: Kai Germaschewski <kai.germaschewski@gmx.de>
Cc: Karsten Keil <kkeil@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
I am having the bug FATAL: Error inserting capi ([..]/capi.ko): Device or
resource busy when I try to reload capi after loading it. in dmesg:
capi20: unable to get major 68
Fix the issue which is caused by setting the major to zero when registering
the chrdev succeeded.
(akpm: this means that we can again not use `major=0' (dynamic major
allocation) for this driver).
Cc: Karsten Keil <kkeil@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
If the user specified `major=0' (odd thing to do), capi.c will use dynamic
allocation. We need to pick up that major for subsequent unregister_chrdev().
Acked-by: Karsten Keil <kkeil@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
The API and code have been through various bits of initial review by
serial driver people but they definitely need to live somewhere for a
while so the unconverted drivers can get knocked into shape, existing
drivers that have been updated can be better tuned and bugs whacked out.
This replaces the tty flip buffers with kmalloc objects in rings. In the
normal situation for an IRQ driven serial port at typical speeds the
behaviour is pretty much the same, two buffers end up allocated and the
kernel cycles between them as before.
When there are delays or at high speed we now behave far better as the
buffer pool can grow a bit rather than lose characters. This also means
that we can operate at higher speeds reliably.
For drivers that receive characters in blocks (DMA based, USB and
especially virtualisation) the layer allows a lot of driver specific
code that works around the tty layer with private secondary queues to be
removed. The IBM folks need this sort of layer, the smart serial port
people do, the virtualisers do (because a virtualised tty typically
operates at infinite speed rather than emulating 9600 baud).
Finally many drivers had invalid and unsafe attempts to avoid buffer
overflows by directly invoking tty methods extracted out of the innards
of work queue structs. These are no longer needed and all go away. That
fixes various random hangs with serial ports on overflow.
The other change in here is to optimise the receive_room path that is
used by some callers. It turns out that only one ldisc uses receive room
except asa constant and it updates it far far less than the value is
read. We thus make it a variable not a function call.
I expect the code to contain bugs due to the size alone but I'll be
watching and squashing them and feeding out new patches as it goes.
Because the buffers now dynamically expand you should only run out of
buffering when the kernel runs out of memory for real. That means a lot of
the horrible hacks high performance drivers used to do just aren't needed any
more.
Description:
tty_insert_flip_char is an old API and continues to work as before, as does
tty_flip_buffer_push() [this is why many drivers dont need modification]. It
does now also return the number of chars inserted
There are also
tty_buffer_request_room(tty, len)
which asks for a buffer block of the length requested and returns the space
found. This improves efficiency with hardware that knows how much to
transfer.
and tty_insert_flip_string_flags(tty, str, flags, len)
to insert a string of characters and flags
For a smart interface the usual code is
len = tty_request_buffer_room(tty, amount_hardware_says);
tty_insert_flip_string(tty, buffer_from_card, len);
More description!
At the moment tty buffers are attached directly to the tty. This is causing a
lot of the problems related to tty layer locking, also problems at high speed
and also with bursty data (such as occurs in virtualised environments)
I'm working on ripping out the flip buffers and replacing them with a pool of
dynamically allocated buffers. This allows both for old style "byte I/O"
devices and also helps virtualisation and smart devices where large blocks of
data suddenely materialise and need storing.
So far so good. Lots of drivers reference tty->flip.*. Several of them also
call directly and unsafely into function pointers it provides. This will all
break. Most drivers can use tty_insert_flip_char which can be kept as an API
but others need more.
At the moment I've added the following interfaces, if people think more will
be needed now is a good time to say
int tty_buffer_request_room(tty, size)
Try and ensure at least size bytes are available, returns actual room (may be
zero). At the moment it just uses the flipbuf space but that will change.
Repeated calls without characters being added are not cumulative. (ie if you
call it with 1, 1, 1, and then 4 you'll have four characters of space. The
other functions will also try and grow buffers in future but this will be a
more efficient way when you know block sizes.
int tty_insert_flip_char(tty, ch, flag)
As before insert a character if there is room. Now returns 1 for success, 0
for failure.
int tty_insert_flip_string(tty, str, len)
Insert a block of non error characters. Returns the number inserted.
int tty_prepare_flip_string(tty, strptr, len)
Adjust the buffer to allow len characters to be added. Returns a buffer
pointer in strptr and the length available. This allows for hardware that
needs to use functions like insl or mencpy_fromio.
Signed-off-by: Alan Cox <alan@redhat.com>
Cc: Paul Fulghum <paulkf@microgate.com>
Signed-off-by: Hirokazu Takata <takata@linux-m32r.org>
Signed-off-by: Serge Hallyn <serue@us.ibm.com>
Signed-off-by: Jeff Dike <jdike@addtoit.com>
Signed-off-by: John Hawkes <hawkes@sgi.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Signed-off-by: Adrian Bunk <bunk@stusta.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
The previous patch adding the ability to nest struct class_device
changed the paramaters to the call class_device_create(). This patch
fixes up all in-kernel users of the function.
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Another large rollup of various patches from Adrian which make things static
where they were needlessly exported.
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Initial git repository build. I'm not bothering with the full history,
even though we have it. We can create a separate "historical" git
archive of that later if we want to, and in the meantime it's about
3.2GB when imported into git - space that would just make the early
git days unnecessarily complicated, when we don't have a lot of good
infrastructure for it.
Let it rip!
Jörn Engel