We recently took delivery of two 2540 storage arrays to be used with MySQL and ZFS.  These are great little boxes that offer a lot of bang for the buck.

After getting them on-site and online I started seeing lots of dropped packets behavior between the CAM host and other devices on the same VLAN when CAM was attempting to communicate with the arrays.  Initially we thought this was a bad edge switch, but as it turns out, it looks to be related to the bug described here:

http://sunsolve.sun.com/search/document.do?assetkey=1-66-240105-1

Originally we were only upgrading the arrays to the firmwares that come with the latest release of the 6.0.x branch because a Sun tech was concerned about false positives on broken drives/controllers with the 6.1.whatever CAM software.  However, after running into this network issue, I’ve upgraded CAM to 6.1.2.8.  I’m guessing that the issues he was concerned about are gone since it’s been a few months since the ticket was open.

To get the arrays with more disks to upgrade correctly, I’ve actually had to connect both controllers to sepearate network ports on the back of the CAM host (on different subnets to avoid routing issues).  This method didn’t actually work until I un-registered and re-registered the array within CAM.  What a pain!

I hope this helps someone else out there if they run into this issue.