Hardware vs Software iSCSI initiator

For latest version see: http://vinfrastructure.it/vdesign/vstorage-software-vs-hardware-iscsi/

ESX/ESXi 4 supports both hardware and software iSCSI initiators. Both supports multipathing for higher avaiability.

A best practice is to never mix both type of initators in the same host.

Software iSCSI initiators are appropriate mostly where there are limited number of host PCI slots. A software initiator can use a port of an existing NIC for iSCSI traffic (but it's still strongly recomened to isolate iSCSI traffic for performance and security reasons).

Technical difference

VMware ESX/ESXi support both type of iSCSI and in vSphere the performance difference is quite minimal.

The configuration is quite different, cause in the hardware version (with iSCSI HBA) the configuration is done only under storage adapters, with a software initiator the configuration must be done also on networking and on firewall (in the full ESX version).

For more info about the iSCSI configuration see the official documentation: http://www.vmware.com/pdf/vsphere4/r40_u1/vsp_40_u1_iscsi_san_cfg.pdf

The big technical difference is that an hardware initiator is a way to offload the complete iSCSI and TCP/IP stack on hardware and save host resources (primary CPU and memory resources). But on new CPU this overhead (for software initiator) could be a minimal problem.

Hardware initiator without HBA

A lot of new NICs have new hardware feature to offload part of the iSCSI and/or network stack. For TCP/IP offload this feature is usually called TOE (TCP/IP offload Engine).

Intel NICs use a quite different approach rather than TOE, that is called Intel® I/O Acceleration Technology:

http://download.intel.com/support/network/sb/19127_lad_competeguide_r04.pdf

http://features.techworld.com/storage/1236/intels-i-o-at-is-not-a-toe-killer/

Note that vSphere 4.1 introduces a support for some kind of TOE/iSCSI optiomization, but only on a few NIC cards:

http://www.vmwareinfo.com/2009/01/iscsi-hardware-or-software-how-many.html

http://vbl0g.blogspot.com/2010/07/vsphere-41-with-iscsi-offloading.html

http://www.sanstor.info/5iSCSI%20software%20initiators%20vs.pdf

http://www.dell.com/downloads/global/products/pedge/en/BroadcomDellEnhancedVirualizationFunctionalityWhitePaper.pdf

The problem of this approach is that this kind of optimization will not support jumbo frames...

Software or Hardware iSCSI? No Jumbo Frame on HBA?