Original post was incorrect as it didn't realize that we already had
a self-referenc due to device_initialize(), and we were really only
missing the put on our own reference. This was hidden by the other bug
which had the midlayer reusing stargets after they were already free,
which was doing too many puts on our rport.
Updating FC transport for:
- Add put in fc_rport_final_delete(), to release the rport.
Prior, we were leaving the rport with a reference, thus the shost
with references, etc. If the driver was unloaded, shosts and rports
remained, along with work threads, etc
- Fix fc_rport_create failure path - too many put's on parent
- Add commenting to easily track ref taking.
Signed-off-by: James Smart <james.smart@emulex.com>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
Updated patch to address comments from Pat Mansfield and Michael Reed:
Bumped max to 600 (10mins). Set default dev_loss_tmo to a value other
than the max (30s).
Signed-off-by: James Smart <James.Smart@emulex.com>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
In a prior posting to linux-scsi on the fc transport and workq
deadlocks, we noted a second error that did not have a patch:
http://marc.theaimsgroup.com/?l=linux-scsi&m=114467847711383&w=2
- There's a deadlock where scsi_remove_target() has to sit behind
scsi_scan_target() due to contention over the scan_lock().
Subsequently we posted a request for comments about the deadlock:
http://marc.theaimsgroup.com/?l=linux-scsi&m=114469358829500&w=2
This posting resolves the second error. Here's what we now understand,
and are implementing:
If the lldd deletes the rport while a scan is active, the sdev's queue
is blocked which stops the issuing of commands associated with the scan.
At this point, the scan stalls, and does so with the shost->scan_mutex held.
If, at this point, if any scan or delete request is made on the host, it
will stall waiting for the scan_mutex.
For the FC transport, we queue all delete work to a single workq.
So, things worked fine when competing with the scan, as long as the
target blocking the scan was the same target at the top of our delete
workq, as the delete workq routine always unblocked just prior to
requesting the delete. Unfortunately, if the top of our delete workq
was for a different target, we deadlock. Additionally, if the target
blocking scan returned, we were unblocking it in the scan workq routine,
which really won't execute until the existing stalled scan workq
completes (e.g. we're re-scheduling it while it is in the midst of its
execution).
This patch moves the unblock out of the workq routines and moves it to
the context that is scheduling the work. This ensures that at some point,
we will unblock the target that is blocking scan. Please note, however,
that the deadlock condition may still occur while it waits for the
transport to timeout an unblock on a target. Worst case, this is bounded
by the transport dev_loss_tmo (default: 30 seconds).
Finally, Michael Reed deserves the credit for the bulk of this patch,
analysis, and it's testing. Thank you for your help.
Note: The request for comments statements about the gross-ness of the
scan_mutex still stand.
Signed-off-by: Michael Reed <mdr@sgi.com>
Signed-off-by: James Smart <james.smart@emulex.com>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
Use ARRAY_SIZE macro instead of sizeof(x)/sizeof(x[0]) and remove
duplicates of the macro.
Signed-off-by: Tobias Klauser <tklauser@nuerscht.ch>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
As previously reported via Michael Reed, the FC transport took a hit
in 2.6.15 (perhaps a little earlier) when we solved a recursion error.
There are 2 deadlocks occurring:
- With scan and the delete items sharing the same workq, flushing the
workq for the delete code was getting it stalled behind a very long
running scan code path.
- There's a deadlock where scsi_remove_target() has to sit behind
scsi_scan_target() due to contention over the scan_lock().
This patch resolves the 1st deadlock and significantly reduces the
odds of the second. So far, we have only replicated the 2nd deadlock
on a highly-parallel SMP system. More on the 2nd deadlock in a following
email.
This patch reworks the transport to:
- Only use the scsi host workq for scanning
- Use 2 other workq's internally. One for deletions, the other for
scheduled deletions. Originally, we tried this with a single workq,
but the occassional flushes of the scheduled queues was hitting the
second deadlock with a slightly higher frequency. In the future, we'll
look at the LLDD's and the transport to see if we can get rid of this
extra overhead.
- When moving to the other workq's we tightened up some object states
and some lock handling.
- Properly syncs adds/deletes
- minor code cleanups
- directly reference fc_host_attrs, rather than through attribute
macros
- flush the right workq on delayed work cancel failures.
Large kudos to Michael Reed who has been working this issue for the last
month.
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
This moves the eh_timed_out functionality from the scsi_host_template
to the transport_template. Given that this is now a transport function,
the EH_RESET_TIMER case no longer caps the timer reschedulings. The
transport guarantees that this is not an infinite condition.
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
In the past I added an host attribute but unfortunately
I forgot to increase FC_HOST_NUM_ATTRS.
This is fixed with the patch. Otherwise an fibre channel
lld might run into
BUG_ON(count > FC_HOST_NUM_ATTRS);
in fc_attach_transport().
Signed-off-by: Andreas Herrmann <aherrman@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
Change the core SCSI code to use kzalloc rather than kmalloc+memset
where possible.
Signed-off-by: Jes Sorensen <jes@sgi.com>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
Current fc_transport consumers initially register rports
with an UNKNOWN role-state and follow-up with a call to
fc_remote_port_rolechg(). Modify code in
fc_remote_port_add() to scan the fc_host_rport_bindings()
array for consistent bindings regardless of role-type.
Original code would only scan bindings array for targets,
causing duplicate fc_remote_ports/rport-X:Y-Z entries to be
created for the yet-to-be-role-changed rports.
Signed-off-by: Andrew Vasquez <andrew.vasquez@qlogic.com>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
When James Smart fixed the issue of the userspace scan atributes
crashing the system with the FC transport class he added a patch to
let the transport class check if the parent is valid for a given
transport class.
When adding support for the integrated raid of fusion sas devices
we ran into a problem with that, as it didn't allow adding virtual
raid volumes without the transport class knowing about it.
So this patch adds a user_scan attribute instead, that takes over from
scsi_scan_host_selected if the transport class sets it and thus lets
the transport class control the user-initiated scanning. As this
plugs the hole about user-initiated scanning the target_parent hook
goes away and we rely on callers of the scanning routines to do
something sensible.
For SAS this meant I had to switch from a spinlock to a mutex to
synchronize the topology linked lists, in FC they were completely
unsynchronized which seems wrong.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
Add fc_host attribute permanent_port_name which is
used to show the port name of the primary port -
the port that initially logged into the fabric.
For a virtual port (registered via the primary port with
FDISC command) it is useful to know not only its (virtual)
port name but also the permanent port name.
Signed-off-by: Andreas Herrmann <aherrman@de.ibm.com>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
In the scenario that a link was broken, the devloss timer for each
rport was expire at roughly the same time, causing lots of "delete"
workqueue items being queued. Depth is dependent upon the number of
rports that were on the link.
The rport target remove calls were calling flush_scheduled_work(),
which would interrupt the stream, and start the next workqueue item,
which did the same thing, and so on until recursion depth was large.
This fix stops the recursion in the initial delete path, and pushes it
off to a host-level work item that reaps the dead rports.
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
patch below marks a few scsi core datastructures as const, so that they end up
in the .rodata section and don't cacheline share with things that get dirtied
Signed-off-by: Arjan van de Ven <arjan@infradead.org>
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
I recently picked up my older work to remove unnecessary #includes of
sched.h, starting from a patch by Dave Jones to not include sched.h
from module.h. This reduces the number of indirect includes of sched.h
by ~300. Another ~400 pointless direct includes can be removed after
this disentangling (patch to follow later).
However, quite a few indirect includes need to be fixed up for this.
In order to feed the patches through -mm with as little disturbance as
possible, I've split out the fixes I accumulated up to now (complete for
i386 and x86_64, more archs to follow later) and post them before the real
patch. This way this large part of the patch is kept simple with only
adding #includes, and all hunks are independent of each other. So if any
hunk rejects or gets in the way of other patches, just drop it. My scripts
will pick it up again in the next round.
Signed-off-by: Tim Schmielau <tim@physik3.uni-rostock.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
We recently went back to implement a board reset. When we perform the
reset, we wanted to tear down the internal data structures and rebuild
them. Unfortunately, when it came to the rport structure, things were
odd. If we deleted them, the scsi targets and sdevs would be
torn down. Not a good thing for a temporary reset. We could block the
rports, but we either maintain the internal structures to keep the
rport reference (perhaps even replicating what's in the transport),
or we have to fatten the fc transport with new search routines to find
the rport (and deal with a case of a dangling rport that the driver
forgets).
It dawned on me that we had actually reached this state incorrectly.
When the fc transport first started, we did the block/unblock first, then
added the rport interface. The purpose of block/unblock is to hide the
temporary disappearance of the rport (e.g. being deleted, then readded).
Why are we making the driver do the block/unblock ? We should be making
the transport have only an rport add/delete, and the let the transport
handle the block/unblock.
So... This patch removes the existing fc_remote_port_block/unblock
functions. It moves the block/unblock functionality into the
fc_remote_port_add/delete functions. Updates for the lpfc driver are
included. Qlogic driver updates are also enclosed, thanks to the
contributions of Andrew Vasquez. [Note: the qla2xxx changes are
relative to the scsi-misc-2.6 tree as of this morning - which does
not include the recent patches sent by Andrew]. The zfcp driver does
not use the block/unblock functions.
One last comment: The resulting behavior feels very clean. The LLDD is
concerned only with add/delete, which corresponds to the physical
disappearance. However, the fact that the scsi target and sdevs are
not immediately torn down after the LLDD calls delete causes an
interesting scenario... the midlayer can call the xxx_slave_alloc and
xxx_queuecommand functions with a sdev that is at the location the
rport used to be. The driver must validate the device exists when it
first enters these functions. In thinking about it, this has always
been the case for the LLDD and these routines. The existing drivers
already check for existence. However, this highlights that simple
validation via data structure dereferencing needs to be watched.
To deal with this, a new transport function, fc_remote_port_chkready()
was created that LLDDs should call when they first enter these two
routines. It validates the rport state, and returns a scsi result
which could be returned. In addition to solving the above, it also
creates consistent behavior from the LLDD's when the block and deletes
are occuring.
Rejections fixed up and
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
Ok, here's a patch to add such a common API for fc transport users.
Relevant LLD changes (lpfc and qla2xxx) also present.
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
This should eliminate (at least in the mid layer) to make numeric
assumptions about any of the enumeration variables. As a side effect,
it will also make all the messages consistent and line us up nicely for
the error logging strategy (if it ever shows itself again).
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
I recently tried to construct a totally generic transport class and
found there were certain features missing from the current abstract
transport class. Most notable is that you have to hang the data on the
class_device but most of the API is framed in terms of the generic
device, not the class_device.
These changes are two fold
- Provide the class_device to all of the setup and configure APIs
- Provide and extra API to take the device and the attribute class and
return the corresponding class_device
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
We have some nasty issues with 2.6.12-rc6. Any request to scan on
the lpfc or qla2xxx FC adapters will oops. What is happening is the
system is defaulting to non-transport registered targets, which
inherit the parent of the scan. On this second scan, performed by
the attribute, the parent becomes the shost instead of the rport.
The slave functions in the 2 FC adapters use starget_to_rport()
routines, which incorrectly map the shost as an rport pointer.
Additionally, this pointed out other weaknesses:
- If the target structure is torn down outside of the transport,
we have no method for it to be regenerated at the proper parent.
- We have race conditions on the target being allocated by both
the midlayer scan (parent=shost) and by the fc transport
(parent=rport).
Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
Initial git repository build. I'm not bothering with the full history,
even though we have it. We can create a separate "historical" git
archive of that later if we want to, and in the meantime it's about
3.2GB when imported into git - space that would just make the early
git days unnecessarily complicated, when we don't have a lot of good
infrastructure for it.
Let it rip!
James Smart