interrupt and smp_affinity

Check your interrupt's smp_affinity

[root@localhost src]# cat /proc/irq/106/smp_affinity
7fffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff

Each bit represents 1 CPU core, there are 255 bits in the output above because the kernel is configured with max 255 CPU SMP support.

Interrupt number can be checked with

[root@localhost src]# cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 CPU6 CPU7
0: 615812259 0 0 0 0 0 0 0 IO-APIC-edge timer
1: 3 0 0 0 0 0 0 0 IO-APIC-edge i8042
8: 1 0 0 0 0 0 0 0 IO-APIC-edge rtc
9: 0 0 0 0 0 0 0 0 IO-APIC-level acpi
12: 4 0 0 0 0 0 0 0 IO-APIC-edge i8042
14: 5505606 0 0 0 0 0 0 0 IO-APIC-edge ide0
66: 1803 0 0 0 0 0 0 0 IO-APIC-level uhci_hcd:usb1, uhci_hcd:usb3, ehci_hcd:usb5
74: 0 0 0 0 0 0 0 0 IO-APIC-level uhci_hcd:usb2, uhci_hcd:usb4
90: 408523 0 0 0 0 0 0 0 PCI-MSI eth0
98: 3 0 0 0 0 0 0 0 PCI-MSI eth1
106: 307885 0 0 0 0 0 0 0 PCI-MSI eth2
114: 128 0 0 0 0 0 0 4216539679 PCI-MSI eth3
122: 135 0 0 1727450681 0 0 0 0 PCI-MSI eth4
130: 307878 0 0 0 0 0 0 0 PCI-MSI eth5
169: 1225688 0 0 0 0 0 0 0 IO-APIC-level ioc0
NMI: 6373811 2672059 2908669 507093084 3414440 4144598 3929651 137151686
LOC: 615668730 615672280 615604333 615578017 615668454 615672006 615604038 615577637
ERR: 0
MIS: 0


Change the smp_affinity with:
echo 04 > /proc/irq/106/smp_affinity

On x86_64 system, only 1 CPU core can be tied to 1 eth interrupts. Assigning multiple cores to a single interrupt doesn't work here. The core with lowest id assigned will always be used.