Wed, Jun 03, 2015 at 10:13:16PM CEST, gslavic(a)sox.rs wrote:
Dear Mr Jiri Pirko,
My name is Goran Slavić, and I work for Serbian Open
eXchange. We have setup the teaming on 4x10Gb/sec ports of our routing
server. Unfortunetly load balancing between teamed ports doesn't work very
well (we have extremely UN-balanced traffic on teamed ports). Traffic
analysis suggests that the balancing is only done with MAC addresses and the
rest of configuration directives ("ipv4, ipv6, l4" ...) is ignored/not taken
in consideration when load balancing traffic between ports !
You can find a detailed explanation (what we configured on
ports/teaming software), our configuration details and graph that are the
result of our testing on this link (forum where we first shown the problem
and asked people to help us):
We would highly appreciate any help, advice or suggestion
you can give us regarding this problem.
It's always good idea to cc libteam mailing list for reports such as
TX port selection is done based on hash (0-255) computed by BPF function.
Then, the hash is either used directly to determine tx port
(hash%number_of_ports), or there is a hash-port mapping where every hash
is mapped to a tx port. The second case is yours.
You can see the mapping either in teamd debug output, or if you list
team options directly by "teamnl team0 options". With mapping there are
also statistics for each hash there.
So in your case, there might be 3 issues:
1) your traffic is not differenciated enough, so most of the packets
falls into sigle hash. That would happen for example for one tcp stream
between 2 ip addresses.
2) hash function fails to produce good hash according to the packets
coming through. You could try to dump the BPF function and look at the
inctructions by hand. Or another thing would be to assebmle your own
hash function and push it into kernel manually (using teamnl team0
3) mapping is wrong. Balancer might do the mapping wrongly. To figure this
out, the debug messages or options dump will help you to find out if
this is the case.
Please gather more info so we can proceed to debug this.